Databricks Invests in Mistral and Integrates LLM Models

The data infrastructure company Databricks, led by Ali Ghodsi, has announced a partnership with the French startup Mistral. Mistral has gained widespread attention in the global AI community with its growing family of high-performance large language models (LLMs), many of which are open source.

As part of this collaboration, Databricks has made an undisclosed investment in Mistral to support its Series A funding and plans to integrate Mistral's LLMs into its own data intelligence platform.

This integration will enable direct integration of the models, providing convenience for enterprise users to easily combine data with models for generative AI applications, while maintaining the security, privacy, and governance features of the Databricks platform.

This development marks another notable distribution partnership for Mistral, which has been actively expanding its business through industry collaborations. Recently, it also announced similar partnerships with Snowflake, a competitor to Databricks through its data cloud product, and with Microsoft, which sparked interest from enterprise controls and regulatory agencies.

Native integration of specific models

In a recent blog post, Databricks confirmed that its collaboration with Mistral will drive the native integration of two text generation models: Mistral 7B and Mixtral 8x7B, both of which are open source.

The former is a small transformer model with 7 billion parameters, trained with an 8k context length and high service efficiency. Meanwhile, the latter is a sparse expert model ensemble (SMoE) that supports a 32k context length and can handle English, French, Italian, German, and Spanish. Mixtral 8x7B has even outperformed Meta's Llama 2 70B (its training base) and OpenAI's GPT-3.5 in multiple benchmark tests, including GSM-8K and MMLU, while also having faster inference speed.

Users of the Databricks data intelligence platform can now find these two models in the platform marketplace, which includes complete information about their capabilities and different usage scenarios.

Databricks states that users can experiment with these models using the Mosaic AI Playground provided by the platform console, use them as optimized model endpoints through Mosaic AI Model Serving, or customize them using proprietary data hosted by the platform (Mosaic AI Foundation Model Adaptation) to meet specific use case requirements.

"Mistral AI models can now be used and customized in multiple ways on Databricks, which provides the most comprehensive toolkit for building, testing, and deploying end-to-end generative AI applications. There are multiple quick-start options, whether starting with side-by-side comparisons of pre-trained models or using models on a pay-per-token basis," said a spokesperson for Databricks.

While this news is good for Databricks customers who want to build generative AI applications and solutions using their data assets on the platform, it is worth noting that the partnership does not mention Mistral Large, Mistral's latest model that performs just below GPT-4 and surpasses Anthropic's Claude 2, Google's Gemini Pro, and GPT-3.5. This model has native fluency in five languages and a context window of 32K tokens. Snowflake has integrated this model with Mixtral 8x7B and Mistral 7B into its Cortex service in the data cloud.

When asked about the integration of Mistral Large by VentureBeat, a spokesperson for Databricks stated that they are currently unable to share any related information. Other open-source and commercially available models offered by Databricks include Meta's Llama-2, CodeLlama, Stable Diffusion XL, and Mosaic's MPT series.

Mistral continues to expand its partnerships

Databricks and Snowflake are not the only partners of Mistral.

The company raised the largest seed funding in European history in June 2023 and quickly completed a large Series A funding round, focusing on strengthening collaborations with the industry to expand its influence and consolidate its trustworthiness among suppliers in the AI field dominated by OpenAI, Anthropic, and Google.

Just a few weeks ago, the company received a $16 million investment from Microsoft to add its models to the Azure cloud platform. This deal makes Mistral the second company, after OpenAI, to offer models on the Microsoft platform.

In addition, Mistral has signed separate partnerships with IBM, enabling Mistral 8x7B to be used on WatsonX, and with Perplexity and Amazon. It remains to be seen which other partners Mistral will collaborate with in the future to expand its influence and drive the development of AI use cases across industries.