Like OpenAI, this search engine giant is reversing decades of open publishing in the field of AI research.
A year ago, when Google AI scientists announced an important new project called Pathways Language Model (PaLM), they spent hundreds of words in a technical paper describing the important new AI technology used to implement the project.
Last week, when introducing PaLM's successor, PaLM 2, Google revealed almost no information. In the appendix behind the 92-page "technical report," Google scholars briefly described the reason why they wouldn't tell the world anything this time:
PaLM-2 is a new state-of-the-art language model. We have small, medium, and large variants that use stacked layers based on the Transformer architecture, with different parameters depending on the model size. More details about model size and architecture are not disclosed.
Intentionally refusing to disclose the so-called architecture of PaLM 2 - the way the program is built - is not only different from the previous PaLM paper, but also different from the entire history of AI releases, which are mainly based on open-source software code and usually include a lot of details about program architecture.
This shift is clearly a response to OpenAI, one of Google's biggest competitors. In April of this year, OpenAI refused to disclose details about its latest "generative AI" project, GPT-4, shocking the research community. Prominent AI researchers warned that OpenAI's surprising choice could have a chilling effect on disclosure in the entire industry, and the PaLM 2 paper is the first major sign that their views may be correct.
Like GPT-4, PaLM 2 is a generative AI program that can generate clusters of text based on prompts to perform tasks such as question answering and software coding.
Like OpenAI, Google is also changing the openness of AI research for decades. In a 2017 research paper, "Attention is All You Need," Google detailed a breakthrough project called The Transformer. The program was quickly adopted by the AI research community and industry to develop natural language processing programs.
OpenAI launched the ChatGPT program in the fall, which caused global excitement about ChatGPT.
The authors of the original paper, including Ashish Vaswani, were not listed as authors of PaLM 2.
Therefore, in a sense, by disclosing in one sentence that PaLM 2 is a descendant of The Transformer and refusing to disclose any other information, the company's researchers have made it clear their contributions to the field and their intention to end the tradition of sharing breakthrough research.
They wrote that by finding the right balance between program size and training data volume, the authors were able to make the PaLM 2 program leaner, so the program itself is much smaller than the original PaLM program. Considering the recent trend of AI towards larger scales, this seems significant.
As the authors wrote, "The largest model in the PaLM 2 series, PaLM 2-L, is noticeably smaller than the largest PaLM model but uses more training compute. Our evaluation results show that the PaLM 2 model outperforms PaLM significantly on various tasks including natural language generation, translation, and inference. These results suggest that model scaling is not the only way to improve performance. Instead, performance can be unlocked through careful data selection and efficient architecture/objectives. Additionally, smaller but higher-quality models can significantly improve inference efficiency, reduce serving costs, and make downstream applications of the model available to more applications and users."
The authors of PaLM 2 say there is an optimal balance between program size and training data volume. Compared to PaLM, the PaLM 2 program shows significantly improved accuracy in benchmark tests, as summarized by the authors in a single table.
In this way, they are building observations of actual research on AI project scales over the past two years.
For example, work by Jordan Hoffman and colleagues at Google's DeepMind last year, widely cited, created the so-called Chinchilla rule of thumb, a formula for balancing training data volume and program size.
The numbers proposed by PaLM 2 scientists differ slightly from Hoffman and his team, but it validates what the paper says. They present their results head-on with Chinchilla's work in a single scaling table.
This approach of focusing on an optimal point deviates slightly from the original PaLM. Through this model, Google emphasizes the training scale of the project, noting that it is "the largest TPU-based system configuration used for training to date," referring to Google's TPU computer chip.
Although little is revealed in the new PaLM 2 work, you can say that it does confirm the trend towards more thoughtful scales and capabilities to handle size rather than just size.
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.