Meditron

Meditron is a suite of open-source medical language models (7B and 70B parameters) developed by EPFL (Swiss Federal Institute of Technology) based on Llama 2, achieving state-of-the-art performance on medical question-answering benchmarks among open-source models — using a novel GAP-Replay continual learning technique to train on PubMed articles, medical guidelines, and clinical textbooks without catastrophically forgetting the general English knowledge required for coherent medical conversations.

What Is Meditron?

• Definition: Open-source medical LLMs fine-tuned from Llama 2 on curated medical corpora — designed for clinical decision support, medical education, and diagnostic assistance, with particular focus on accessibility for low-resource healthcare settings where commercial AI APIs are prohibitively expensive.
• GAP-Replay: A continual learning method that replays a small percentage of general-purpose training data alongside medical data during fine-tuning — preventing "catastrophic forgetting" where the model loses ability to hold coherent conversations while gaining medical knowledge.
• Medical Data Sources: PubMed (biomedical literature), clinical practice guidelines (WHO, NIH), medical textbooks, and curated Q&A datasets — carefully filtered for accuracy and relevance.
• Low-Resource Design: The 7B model runs on a single consumer GPU (16 GB VRAM) — enabling hospitals in developing nations to deploy local medical AI assistants without sending private patient data to cloud APIs.

Performance on Medical Benchmarks

Key Features

• Clinical Reasoning: Meditron generates step-by-step diagnostic reasoning — presenting differential diagnoses with supporting evidence from medical literature, not just single-word answers.
• Multilingual Medical: Built on Llama 2's multilingual foundation, enabling medical assistance in languages underserved by English-centric medical AI.
• Safety Design: Trained with conservative refusal patterns for high-risk scenarios — directing patients to seek professional medical care rather than providing definitive diagnoses.
• Reproducible Research: Full training code, data processing pipeline, and evaluation scripts publicly available — enabling the medical AI research community to build upon and improve the methodology.

Meditron vs. Other Medical LLMs

Meditron is the leading open-source medical language model — demonstrating that domain-specialized fine-tuning with continual learning techniques can produce clinically useful AI assistants accessible to healthcare institutions worldwide, including resource-limited settings where data privacy and cost make commercial APIs infeasible.