Llama

LLaMA (Large Language Model Meta AI) is Metas open-source foundation model family that democratized LLM research. Significance: First truly capable open-weights LLM, enabled explosion of open-source AI research and applications. LLaMA 1 (Feb 2023): 7B, 13B, 33B, 65B parameters. Trained on public data only. Matched GPT-3 quality at smaller sizes. Architecture: Standard decoder-only transformer with pre-normalization (RMSNorm), SwiGLU activation, rotary embeddings (RoPE), no bias terms. Training data: 1.4T tokens from CommonCrawl, C4, GitHub, Wikipedia, Books, ArXiv, StackExchange. Efficiency focus: Designed for inference efficiency, smaller models matching larger ones through better data and training. Open ecosystem: Spawned Alpaca, Vicuna, and hundreds of fine-tuned variants. Research impact: Enabled academic research on LLM behavior, fine-tuning, alignment. Limitations: Original release research-only license, limited commercial use. Legacy: Changed the landscape of open AI, proved open models could compete with proprietary ones.