Databricks Unveils Open-Source DBRX for Enterprise-Grade Large Models, Outperforming LLaMA2 in Performance

Databricks officially announced and launched DBRX, an advanced large-scale language model that outperforms existing open-source models in multiple benchmark tests. The company aims to provide high-quality and personalized artificial intelligence solutions for enterprises that want to harness the power of generative AI. DBRX has demonstrated exceptional capabilities in various tests, particularly in language understanding, programming, mathematics, and logical tasks, surpassing models such as LLaMA2-70B, Mistral, and Grok-1. According to Databricks' open-source benchmark test Gauntlet, DBRX consistently ranks among the top performers in over 30 recent benchmark tests, showcasing the continuous improvement in the quality of open-source models. Notably, DBRX even outperforms GPT-3.5 in most benchmark tests, which holds significant implications. As more and more enterprises lean towards using open-source alternatives to proprietary models for increased efficiency and control, the emergence of DBRX injects new vitality into this trend. Databricks has observed this trend among its 12,000+ customers, with many achieving higher quality and faster speed by customizing open-source models for specific tasks. Another remarkable aspect of DBRX is its hybrid expert (MoE) architecture based on MegaBlocks research and open-source projects. This design allows the model to maintain a relatively small count of 36 billion active parameters while achieving faster token generation, with a total parameter count of a staggering 132 billion. The MoE architecture enables training of larger models with higher throughput, striking a perfect balance between speed and performance. Databricks trained DBRX using a context window of up to 32K tokens and built it entirely on their platform. Throughout this process, they utilized Unity Catalog for data governance, Apache Spark™ and Lilac AI for data processing and cleaning, and Mosaic AI Training service for large-scale model training and fine-tuning. DBRX has been meticulously designed by Databricks to be easily customizable, helping enterprises enhance the quality of their AI applications. Starting today, businesses can interact with DBRX on the Databricks platform, leveraging its long-context capabilities in the RAG system and building custom DBRX models on their private data. The model is now publicly available through Databricks' GitHub repository and Hugging Face (DBRX Base, DBRX Instruct). You can also try out the guided model in the Hugging Face space below. The weights for the base model (DBRX Base) and fine-tuned model (DBRX Instruct) are provided on Hugging Face under an open license. Starting today, Databricks customers can use DBRX via API and utilize the same tools and science Databricks used to build DBRX to pretrain their own DBRX-like models from scratch or continue training on top of a Databricks checkpoint.