Former OpenAI Employees and AI Experts Urge the Attorney General to Halt Profit-Making Conversions

A group of 30 artificial intelligence experts, including former OpenAI employees, have issued an open letter urging the attorneys general of California and Delaware to block the company’s reorganization into a "for-profit public benefit corporation."

Nobel laureate and former Google Brain leader Geoffrey Hinton, AI pioneers Yoshua Bengio and Stuart Russell, along with 11 OpenAI employees (including four current anonymous staff and seven former employees), are jointly advocating for the company to avoid potentially unethical business ventures.

Notable signatories include Ramana Kumar, formerly of Google DeepMind, and Neel Nanda, a current DeepMind researcher and ex-Anthropic employee. They stand alongside Elon Musk, who has also expressed opposition, stating that transitioning from a nonprofit to a for-profit model was not part of the mission he helped fund, prompting him to file a lawsuit.

Musk's personal motives are somewhat questionable since he owns a competing AI firm. Perhaps for this reason, Hinton remarked today: “I’m glad someone is trying to ensure OpenAI fulfills its mission without involving Elon Musk.”

In the letter, the signatories argue that the development of Artificial General Intelligence (AGI) should not be controlled by a profit-driven company. AGI refers to AI reaching or surpassing human-level intelligence, which they believe should serve the public good rather than private interests.

OpenAI’s founding mission was precisely built on this principle—“the goal of building safe and beneficial AGI for the benefit of humanity.” In 2017, when OpenAI CEO Sam Altman spoke in London, he emphasized this point, saying: “We never want to make decisions that benefit shareholders. The only people we want to be accountable to are all of humanity.”

The signatories acknowledge that the benefits of AI are “unprecedented,” but they caution that if it falls into the wrong hands, it could worsen “existing inequalities, authoritarianism, and misinformation,” while even posing a risk of human extinction. They reference warnings from experts, AI firms, and governments about this potentially existential challenge.

“AI companies have strong economic incentives to evade effective oversight, and we believe tailored corporate governance structures are insufficient to change that,” the letter adds. OpenAI has grown into a giant, currently valued at $300 billion.

The authors also highlight that “extensive non-disclosure agreements” hinder current and former employees from voicing their concerns. They argue that the company faces little pressure to share information with governments or the public, effectively placing world-changing technology in the hands of a few. “We cannot rely on them to voluntarily share,” they state.

In response, OpenAI stated that any changes to the company “will aim to ensure broader public access to the benefits of AI.” It added: “This structure will continue to ensure that as the for-profit organization succeeds and grows, the nonprofit will also succeed and grow, enabling us to fulfill our mission.”