OpenAI Establishes New Team to Research and Control Superintelligent AI

2023-07-06

OpenAI is assembling a new team led by its Chief Scientist and co-founder, Ilya Sutskever, to develop methods for running and controlling "superintelligent" AI systems. In a blog post released today, Sutskever and Jan Leik, the head of OpenAI's Alignment team, predict that AI surpassing human intelligence may emerge within the next decade. They state that assuming such AI does come to fruition, it may not necessarily be benevolent, and therefore, we need to start researching how to control and constrain it. They write, "Currently, we lack solutions for operating or controlling potential superintelligent AI and preventing it from malfunctioning." "We currently fine-tune AI technologies, such as reinforcement learning based on human feedback, relying on human supervision of AI. However, humans will be unable to supervise AI systems that are smarter than us." To advance progress in the field of "superintelligent alignment," OpenAI is creating a new team called SuperAlignment, co-led by Sutskever and Leik. This team will have access to 20% of the company's current computing resources and will consist of scientists and engineers from OpenAI's former Alignment department, as well as researchers from other departments within the company. Their goal is to address the core technological challenges of controlling superintelligent AI within the next four years. How will they achieve this? By building a "human-level automated alignment searcher" as described by Sutskever and Leik. The overarching objective is to train AI systems using human feedback to assess other AI systems and ultimately construct an AI capable of conducting alignment research. (Here, "alignment research" refers to ensuring that AI systems achieve desired outcomes without deviating from the intended path.) OpenAI's assumption is that AI can advance alignment research faster and better than humans. "As we make progress in this area, our AI systems can handle more and more alignment work and eventually conceive, implement, study, and develop better alignment techniques than what we currently possess," said Leik and his colleagues John Schulman and Jeffrey Wu in a previous blog post. "They will collaborate with humans to ensure their successors are more compatible with humans. Human researchers will increasingly focus on reviewing alignment research conducted by AI systems rather than creating it themselves." Of course, no method is foolproof. Leik, Schulman, and Wu acknowledge many limitations of OpenAI in their article. They state that using AI for evaluation may amplify inconsistencies, biases, or weaknesses present in that AI. Furthermore, it may prove that the most challenging aspect of alignment is not engineering-related. However, Sutskever and Leik believe it is worth the effort. "Superintelligent alignment is fundamentally a machine learning problem, and we believe exceptional machine learning expertise will be key to solving it," they write. "We plan to share the results of this work widely and consider contributing to alignment and protection for non-OpenAI models as an important part of our work." Source: https://biz.crast.net/openai-is-building-a-new-team-to-get-superintelligent-ai-under-control/

RECENT AI NEWS

RECENT AI TOOLS