| Version | Release Date | Parameter Range | Key Innovations and Features |
|---|---|---|---|
| LLaMA 1 | February 24, 2023 | 7B, 13B, 33B, 65B | Dense text-only decoder models using RMSNorm, SwiGLU, and RoPE. Trained on approximately 1.0–1.4 trillion tokens, depending on model size. 2,048-token context. Distributed under a non-commercial research license.[11] |
| Llama 2 | July 18, 2023 | 7B, 13B, 70B | Pretrained and Llama 2-Chat versions; approximately two trillion pre-training tokens; supervised fine-tuning and RLHF for chat. 4,096-token context. The 70B model used Grouped-Query Attention, while the 7B and 13B models retained conventional multi-head attention.[14][13] |
| Code Llama | August 24, 2023; 70B added January 2024 |
7B, 13B, 34B, 70B | Code-specialized continuation of Llama 2. General, Python, and instruction-tuned variants; trained on sequences of 16K tokens and evaluated on inputs of up to 100K. Selected sizes support code infilling.[17][18] |
| Llama 3 | April 18, 2024 | 8B, 70B | Text-only models trained on more than 15 trillion tokens from publicly available sources. 8K context, a tokenizer vocabulary of approximately 128,000 tokens, and GQA in both sizes.[19] |
| Llama 3.1 | July 23, 2024 | 8B, 70B, 405B | Dense, text-only multilingual models with 128K context, tool-use support, and eight officially supported languages. The 405B model was trained using more than 16,000 H100 GPUs. Contrary to some descriptions, Llama 3.1 did not itself accept image input.[20][25][26] |
| Llama 3.2 | September 25, 2024 | Text: 1B, 3B; Vision: 11B, 90B |
Lightweight text models for mobile and edge deployment, together with the first publicly released Llama vision-language models. The 11B and 90B variants accept text and image input and produce text output. 128K context.[3][27] |
| Llama 3.3 | December 6, 2024 | 70B | Instruction-tuned, text-only multilingual model with a 128K context window. Meta positioned it as providing performance near the much larger Llama 3.1 405B on several tasks while requiring fewer resources to serve.[2][28] |
| Llama 4 Scout | April 5, 2025 | 109B total / 17B active; 16 experts |
Native text-and-image model using sparse MoE layers and early multimodal fusion. Approximately 40 trillion multimodal training tokens, 12 officially supported languages, and a nominal context window of up to 10 million tokens.[7][29] |
| Llama 4 Maverick | April 5, 2025 | 400B total / 17B active; 128 experts |
Larger native multimodal MoE model with approximately 22 trillion training tokens and a one-million-token context window. Available in BF16 and FP8 weight formats.[7] |
| Llama 4 Behemoth | Previewed April 5, 2025; not released |
Nearly 2T total / 288B active; 16 experts |
Previewed multimodal teacher model used during the development of Maverick. Its public release was delayed and subsequently reported abandoned.[4][8] |