Vicuna

Vicuna is one of the most influential early open-source chatbots, developed by LMSYS (the team behind Chatbot Arena) by fine-tuning LLaMA on 70,000 user-shared ChatGPT conversations from ShareGPT — demonstrating that open-source models could achieve over 90% of ChatGPT's quality as rated by GPT-4, proving the power of knowledge distillation and launching the era of competitive open-source chat models.

What Is Vicuna?

• Definition: A fine-tuned language model (March 2023) created by LMSYS Org (UC Berkeley, CMU, Stanford, UCSD) — taking Meta's LLaMA-13B base model and fine-tuning it on approximately 70,000 conversations shared by users on ShareGPT.com, a platform where people shared their ChatGPT conversations.
• ShareGPT Data: The training data came from real ChatGPT conversations that users voluntarily shared publicly — providing diverse, high-quality instruction-following examples that captured the breadth of tasks people actually use chatbots for.
• GPT-4 as Judge: LMSYS pioneered using GPT-4 to evaluate chatbot quality — having GPT-4 compare Vicuna's responses against ChatGPT's on 80 diverse questions, finding Vicuna-13B achieved over 90% of ChatGPT's quality score.
• Chatbot Arena: Vicuna was the flagship model for LMSYS's Chatbot Arena — a crowdsourced evaluation platform where users chat with two anonymous models side-by-side and vote for the better response, creating the most trusted LLM ranking system.

Why Vicuna Matters

• Distillation Proof: Vicuna proved that fine-tuning a smaller model (LLaMA-13B) on outputs from a larger model (ChatGPT) could transfer most of the larger model's capabilities — a technique now called "knowledge distillation" that became the standard approach for creating open-source chat models.
• 90% Quality Claim: The "90% of ChatGPT quality" finding electrified the open-source community — showing that the gap between open and closed models was much smaller than expected and could be closed with relatively little data and compute.
• Chatbot Arena Legacy: The evaluation methodology (GPT-4 as judge, human preference voting) that LMSYS developed for Vicuna became the standard for evaluating language models — Chatbot Arena's ELO rankings are now the most cited LLM benchmark.
• Training Cost: Vicuna was trained for approximately $300 in compute — demonstrating that creating a competitive chatbot didn't require millions of dollars, democratizing access to chat model development.

Vicuna is the model that proved open-source chatbots could approach ChatGPT quality through knowledge distillation — by fine-tuning LLaMA on 70K ShareGPT conversations for just $300, LMSYS demonstrated that the gap between open and closed models was bridgeable, launching both the competitive open-source chat model ecosystem and the Chatbot Arena evaluation platform that now defines how the industry measures LLM quality.