CEO of Hugging Face Predicts Small-Scale Language Models Will Dominate in 2024

For artificial intelligence, 2024 will be a crucial turning point. With the rise of small language models (SLMs), companies are pushing the limits of efficiency, cost-effectiveness, and accessibility.

SLMs are compact and powerful AI models that can be trained and fine-tuned for specific tasks. They have fewer parameters, faster inference speed, and lower memory and storage requirements, allowing them to run on ordinary hardware and devices. Their customizability enables them to adapt to different application needs.

According to Clam Delangue, co-founder and CEO of Hugging Face, 2024 will be the year of small AI models. He announced on LinkedIn that Microsoft AI's Phi-2 is the most popular model on Hugging Face.

Mistral, a French AI startup, has also released an open-source SLM called Mixtral 8x7B, which is comparable to GPT-3.5 on some benchmarks but only requires a computer with 100GB of memory to run. It uses a "sparse expert mixture" model that combines multiple small models to improve efficiency.

Microsoft is also keeping up by releasing its self-developed SLM, Phi-2. This model has only 2.7 billion parameters, making it very compact and capable of running even on smartphones, demonstrating the industry's commitment to reducing model size.

In contrast, large language models (LLMs) like GPT-3 have 175 billion parameters. While they can generate human-like text, answer questions, and summarize documents, they also have many drawbacks such as low efficiency, high cost, and poor customizability, creating opportunities for the rise of SLMs.

The rise of SLMs signifies a transformative era for artificial intelligence. 2024 will be the year of small AI models, where innovation and accessibility will redefine the possibilities of AI.