Why Developing AI with Strong Mathematical Skills is Crucial for OpenAI

Recently, the event of Sam Altman being fired and quickly rehired by OpenAI has once again brought the debate about the development and use of artificial intelligence (AI) into focus. What is even more unusual is that a prominent theme in media reports is the ability of AI systems to solve mathematical problems.

Clearly, some dramatic changes within OpenAI are related to the development of a new AI algorithm called Q*. This system is seen as a major breakthrough, with one notable feature being its ability to perform mathematical reasoning.

But isn't mathematics the foundation of artificial intelligence? Computers and calculators are capable of performing mathematical tasks, so why would AI systems struggle with mathematical reasoning?

Artificial intelligence is not a singular entity. It is a collection of strategies that can perform calculations without direct human instruction. As we have seen, some AI systems are capable in the field of mathematics.

However, one of the most important current technologies, large language models (LLMs), has consistently struggled with simulating mathematical reasoning. This is because they are designed to focus on language capabilities.

If the company's new Q* algorithm can solve mathematical problems, it could be a significant breakthrough. Mathematics is an ancient form of human reasoning that LLMs have struggled to mimic.

At the time of writing this article, details and capabilities of the Q* algorithm are limited but intriguing. Therefore, various nuances need to be considered before declaring Q* a success.

For example, everyone has varying degrees of involvement with mathematics, and the level of Q*'s mathematical abilities is unclear. However, published academic work has already used other forms of AI to advance research-level mathematics.

These AI systems can be described as mathematically capable. However, Q* may not be intended to assist scholars in their work but rather for another purpose.

Nevertheless, while Q* may not push the boundaries of cutting-edge research, there is likely to be some significance in its construction that could provide enticing opportunities for future developments.

As society progresses, we are increasingly accepting specialized AI experts to solve specific types of problems. For example, digital assistants, facial recognition, and online recommendation systems are familiar to most people. However, the concept of "general artificial intelligence" (AGI), which possesses reasoning abilities equivalent to humans, remains elusive.

Mathematics is a fundamental skill we hope to teach every child, and it can be seen as an important milestone on the path to AGI. So, how can AI systems with strong mathematical abilities help society?

Mathematical thinking is relevant to many applications, such as coding and engineering, making mathematical reasoning an essential skill for both humans and AI. Ironically, AI is fundamentally based on mathematics at a basic level.

For example, many technologies implemented by AI algorithms ultimately boil down to a mathematical field called matrix algebra. A matrix is simply a grid of numbers, and a digital image is a common example. Each pixel is just numerical data.

Large language models are also inherently mathematical. Based on a large corpus of text samples, machines can learn the probability of words following user prompts (or questions) for chatbots. If you want a pre-trained LLM to focus on a specific topic, it can be fine-tuned on mathematical literature or any other learning domain. An LLM can generate text that appears to understand mathematics.

Unfortunately, this approach produces an LLM that is good at bluffing but lacks details. The problem lies in the fact that mathematical statements are defined to have a precise Boolean value (i.e., true or false). Mathematical reasoning is equivalent to logically deducing new mathematical statements from previously established ones.

Naturally, a mathematical reasoning approach relying on language probabilities is bound to deviate from its track. One possible solution to this problem may be integrating some form of verification system into the architecture (i.e., how LLMs are built) that constantly checks the logic behind the leaps made by large language models.

Q* may have already achieved this, as it may reference Q-learning, where models can improve over time by testing and rewarding correct conclusions.

However, building AI with strong mathematical abilities presents several challenges. For example, some of the most interesting mathematics involves highly improbable events. In many cases, people may perceive patterns based on a small set of numbers, but when enough cases are examined, it unexpectedly collapses. This ability is difficult to incorporate into machines.

Another challenge that may come as a surprise is that mathematical research can involve a high degree of creativity. This is necessary because practitioners need to invent new concepts while adhering to the formal rules of the discipline.

Any AI approach trained solely to find patterns in existing mathematics is unlikely to create genuinely new mathematics. Given the connection between mathematics and technology, this seems to rule out the idea of a new technological revolution.

But let's imagine if AI could indeed create new mathematics. The argument against this viewpoint has a flaw, as one could also argue that the greatest human mathematicians are also trained entirely on existing mathematics. Large language models have surprised us before and will continue to do so in the future.