EPFL researchers develop open-source large-scale language model for healthcare

EPFL researchers have presented Meditron, an open-source and open-access large language model (LLM) specifically tailored for the medical field. It is trained on Meta's Llama 2 and is available under the Apache 2.0 license.

MEDITRON from EPFL stands out for its open-source nature, offering both 7B and 70B versions. As described in the paper, these models are carefully trained using curated medical data, including PubMed literature and clinical guidelines from various sources.

This model comes with the Llama 2 Community License Agreement and the Apache 2.0 commercial use license. The model can also be accessed on Hugging Face.

Evaluations against medical benchmarks show that its performance surpasses existing open-source models, as well as closed-source models like GPT-3.5 and Med-PaLM.

Zeming Chen, the lead author and a PhD candidate, emphasizes the competitiveness of MEDITRON-70B, with a performance gap of less than 5% compared to GPT-4 and less than 10% compared to Med-PaLM-2. Professor Martin Jaggi highlights the importance of MEDITRON's transparency, providing training code and model weights. The open-source approach allows researchers to enhance the model's reliability and robustness through stress testing.

Dr. Mary-Anne Hartley, a medical doctor, emphasizes the secure design of MEDITRON, encoding medical knowledge from transparent and high-quality sources into the model. Collaboration with the International Committee of the Red Cross integrates their clinical practice guidelines into the model to meet the needs of humanitarian contexts.

Dr. Javier Elkin from the International Committee of the Red Cross expresses excitement about the initiative, noting the rarity of health tools sensitive to humanitarian needs. A workshop held in Geneva, funded by the Humanitarian Action Challenge Grant, will explore the potential, limitations, and risks of MEDITRON, focusing on its unique functionalities.

Professor Antoine Bosselut, the lead researcher, outlines MEDITRON's goal of making access to medical knowledge a universal right. This release aligns with the mission of EPFL's Artificial Intelligence Center, emphasizing responsible and effective use of AI for societal benefit. The center promotes interdisciplinary engagement in AI research, education, and innovation, fostering partnerships across departments.

While general models serve multiple tasks, specialized models like those in the medical field are more easily accessible. Previous attempts at medical LLMs, such as Med-PaLM 2 and GPT-4, were either closed-source or limited in scale. Although medAlpaca is open-source, it is only used for medical question-answering.