Google Missed Out on $62 Billion AI Opportunity

2023-09-01

The last of the eight authors of Google's "Transformer" paper has also left Google to start his own company.

Google overlooked the Transformer paper in 2017, resulting in a loss of $6.2 billion - the total valuation of all AI startups founded by the Foundation team based on Transformer. Now, the last of the eight authors of this critical paper has also left the tech giant to start his own company. Llion Jones stated that he will co-found Sakana AI with his colleague and former Google research scientist David Ha.

The paper titled "Attention Is All You Need" was born during Google's mealtime discussions, when the authors were discussing how to efficiently generate text using computers. After five months of testing, the team wrote this paper, unaware of what would happen next. Their work rendered the previously dominant recurrent neural networks (RNNs) largely obsolete in the field of natural language processing.

It is worth noting that the Transformer architecture is the basis for ChatGPT and similar chatbots. The 2017 paper laid the foundation for the generative AI wave initiated by OpenAI, enabling the GPT model to be accessible to the public. Jones and his co-authors finally received the recognition they deserved, as their $1 billion paper was misjudged by Google six years ago.

"It wasn't until recently that I felt... famous?" Jones recently told The Washington Post. "No one knows my face or my name, but I take five seconds to explain, 'I'm part of the team that created the 'T' in ChatGPT'."

A $6.2 billion failure

Cohere, founded by Aidan N Gomez, Ivan Zhang, and Nick Frosst in 2019, is an NLP-focused startup that is currently valued at $2.2 billion. Reports indicate that the initial investor, Tiger Global, will sell its shares and make a 40% premium based on the current valuation, reflecting the rise of this startup.

Another one-year-old startup, "Adept," not only achieved a valuation of $1 billion but also successfully raised $350 million in venture capital this year. Founded by primary authors Ashish Vaswani and Niki Parmar, this startup conceals its profound impact with 25 minimalist employees. The entrepreneurial spirit of the two goes beyond Adept. This undisclosed startup is still in stealth mode and has raised $8 million in funding led by Thrive Capital.

Noam Shazeer, another author of this groundbreaking paper, is also a co-founder of "Character.AI." The valuation of this chatbot startup has risen to $1 billion. Earlier this year, a $150 million funding round led by Andreessen Horowitz solidified the company's unicorn status.

The story doesn't end there. In 2017, Illia Polosukhin founded the high-performance blockchain "NEAR Protocol," valued at $2 billion. The latest $350 million funding is the second nine-figure financing for Polosukhin within four months.

Jakob Uszkoreit, who proposed the name Transformer, founded Inception in 2021, using deep learning and learning life language. During his 13-year tenure as a senior software engineer, he built Google Assistant's language understanding team and was involved in Google Translate in its early stages. The current value of Inception remains undisclosed. As people gradually realize this, Lukasz Kaiser, one of the eight authors of this paper, found a position with OpenAI's technical staff two years ago, which is a fitting ending.

All they needed was attention

Back in 2017, Google had no idea that its paper would eventually be cited by around 90,000 researchers, and the Transformer architecture would support ChatGPT, Midjourney, and many other companies. Although Google shared this discovery with the world, the company itself did not immediately put content generation technology into use. Google kept it dormant until CEO Sundar Pichai announced LaMDA at Google I/O 2021.

The Washington Post pointed out that the company's expansion strategy went awry. According to Bloomberg, out of over 140,000 employees, more than 7,000 are engaged in AI research. In comparison, OpenAI has far fewer employees, with approximately 150 AI researchers out of about 375 employees by 2023.

Several former employees revealed that five years ago, the company's massive workforce added certain layers to approve ideas. Additionally, Polosukhin stated in an interview that the search giant had a high threshold for turning ideas into products. The company's inertia and intolerance for dead ends and repeated experiments were another reason why the Transformer team decided to exit. Furthermore, there were more exciting innovations outside Google's headquarters, leading to talent outflow.

Although the company has been experimenting with models like PaLM, LaMDA, and Bard, there is still no sign of a significant product launch. While this technology was shelved by Google, startups utilized it to eliminate language barriers between humans and machines. This overlooked breakthrough paper made Google a sacrifice for its own success.

RECENT AI NEWS

RECENT AI TOOLS