Stability AI Unveils 120 Billion Parameter Stable LM 2 Model

Stability AI continues to launch new models at an astonishing pace, and recently announced the good news of the Stable LM 2 12B model update.

While Stability AI is well-known for its impressive text-to-image generative AI model, Stable Diffusion, its vision extends far beyond the field of image generation, with a range of more diverse models.

Stable LM made its debut in April 2023 as a large-scale language model (LLM) for text content, and was updated to the more advanced Stable LM 2 1.6B model in January of this year.

The newly launched Stable LM 2 12B model greatly enhances the capabilities of Stable LM 2 by increasing parameters and improving performance. The company proudly claims that in certain benchmark tests, it even outperforms larger models like Llama 2 70B.

Stable LM 2 12B includes a base version and an instruction-tuned variant, aiming to enhance conversational abilities in seven languages (English, Spanish, German, Italian, French, Portuguese, and Dutch). This new model has been made available to commercial users through Stability AI membership, which is the company's main source of revenue.

"The instruction model is specifically designed and trained to interact with users in a conversational manner," said Carlos Riquelme, Head of Language Team at Stability AI. "In addition, we have put a lot of effort into ensuring its security."

This update comes less than a month after co-founder and CEO Emad Mostaque resigned amid allegations of mismanagement, but it demonstrates that the company is still moving forward and continuing to release new model updates under the leadership of interim co-CEOs Shan Shan Wong and Christian Laforte.

Stability AI strives for a balance of performance and accessibility in Stable LM 2

Stability AI claims that Stable LM 2 12B achieves an ideal balance between functionality, accuracy, and accessibility.

This model, with 12 billion parameters, is capable of handling various tasks that are typically limited to models with higher parameter and computational requirements. Benchmark test results show that Stable LM 2 12B exhibits powerful performance comparable to significantly larger models.

It is worth noting that the general approach used by Stability AI in building the smaller 1.6B model also applies to the new 12B model.

"We foresee a future where models will no longer be used in isolation, but rather as part of a broader system, where one or several language models interact with each other and leverage external software tools," said Riquelme. "Therefore, in this direction, the 12B model has also been trained to play this leading role by connecting and invoking various functionalities and APIs to meet the needs of different users and organizations."

Continual optimization of the 1.6B model

Stability AI has not only increased the parameters of Stable LM 2, but also made improvements to the previously released 1.6B version.

Riquelme pointed out that smaller models like the 1.6B Stable LM can be used for more specific and narrow tasks, while the larger 12B model will have more capabilities. Nevertheless, he emphasized that the updated Stable LM 2 1.6B model has improved its conversational style, is more secure, and can better connect with other software tools.

"Its size remains the same, so it is equally fast and lightweight," said Riquelme. "On the other hand, the 12B model excels in performance and reliability, but requires more computational power."

He explained that different models will offer different trade-offs depending on use cases and available resources such as response time, memory, and budget.

"We believe that there is no one-size-fits-all model, so we provide two distinctly different model sizes," he said.