The Foundation of Future Operating Systems: Language

Just a year ago, OpenAI co-founder Ilya Sutskever made a famous statement in a chat with Jensen Huang, the CEO of NVIDIA: "Text is the projection of the world." And contrary to popular belief, ChatGPT does much more than simply learn surface-level statistical correlations.

Fast forward to today, former OpenAI computer scientist Andrej Karpathy talks about the path to Artificial General Intelligence (AGI): "There is a lot of optimization work to be done, and I think, roughly speaking, the direction of development is that everyone is trying to build what I call a Large Language Model Operating System (LLM OS)."

OpenAI has always been on the side of language, believing that text-based models will be the next frontier and the foundation for building more intelligent AI models - or possibly the first step towards AGI.

Karpathy said, "In the past, my feeling about AGI was that its development was not clear. It was very academic, and people would consider different approaches. Now I think it's very clear. Everyone is trying to fill in the gaps."

The current focus is on developing the LLM OS mentioned by Karpathy - an operating system that integrates various modalities such as text, images, and audio. The core or CPU of this system is the LLM Transformer, with the RAM being the context length.

Different Perspectives

Controversy has been ongoing over the past year. Yann LeCun, Chief Scientist at Meta AI, disagrees with Sutskever's viewpoint and may also disagree with Karpathy's assessment today. LeCun believes that "large language models have no knowledge of the underlying reality described by language." He emphasizes that most of human knowledge is non-linguistic.

Gary Marcus and other AI scientists also agree with the view that "large language models cannot reliably understand the world." However, despite such opinions, the main focus of AI research is still based on text and language.

In a similar discussion, Francois Chollet, creator of Keras and a deep learning researcher at Google, expressed his own views. He said, "Language can be seen as the 'operating system' of the mind. It is not the 'substrate' of the mind - without language, you can still think well, just as you can run programs on a computer without an operating system (although it would be more difficult)."

Chollet further states that language greatly simplifies and enhances cognitive abilities, similar to how an operating system simplifies and enhances computational capabilities. It helps create, retain, and inspect thoughts and memories. Without language, connecting complex ideas or recalling distant memories would be challenging but not impossible.

The Debate on AGI Continues

LeCun concludes, "I would say that language is not the operating system of the mind, but its shell. Without the shell, you can still have a fully functional operating system."

Everything seems to make sense. As Karpathy believes, language models may be the operating systems of the future, and experts have unconsciously reached many consensuses.