Llama Models (Meta) Cheat Sheet

Updated 2026-05-21

🧠Study flashcards on this topic82 cards · spaced repetition→

Meta's Llama (Large Language Model Meta AI) is a family of open-weight large language models that has evolved from a research-only release in 2023 into one of the most widely deployed model families in the world. Llama models range from compact 1B-parameter edge models to massive mixture-of-experts architectures exceeding 400B total parameters, enabling deployment on a single smartphone all the way to multi-GPU server clusters. What makes the family distinctive is open weights under a commercial-friendly community license, allowing developers to fine-tune, self-host, and build products without vendor lock-in. Understanding the family requires tracking several parallel axes at once: model generation (3.1, 3.2, 3.3, 4), size tier, modality (text-only vs. vision), and variant type (base vs. instruct) — each combination has distinct capabilities, prompt formats, and deployment requirements.

What This Cheat Sheet Covers

This topic spans 15 focused tables and 98 indexed concepts, 82 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Model Generations and Release TimelineTable 2: Model Sizes and Parameter TiersTable 3: Base vs. Instruct VariantsTable 4: Context WindowsTable 5: Architecture and Technical DesignTable 6: Llama 3.x Prompt Format and Special TokensTable 7: Llama 4 Prompt Format and Special TokensTable 8: Vision and Multimodal CapabilitiesTable 9: Code Llama FamilyTable 10: Llama Guard — Safety ClassificationTable 11: Purple Llama Safety SuiteTable 12: Hosting and Deployment OptionsTable 13: Quantization FormatsTable 14: Fine-Tuning MethodsTable 15: License and Commercial Terms

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Model Generations and Release Timeline

Each Llama generation introduced a major architectural or capability leap. Knowing which generation a model belongs to immediately signals its context window, multimodal support, and license terms.

Model	Example	Description
Llama 3 (April 2024)	`meta-llama/Meta-Llama-3-8B-Instruct`	• 8B and 70B dense decoder-only transformers • 128K-token vocabulary (up from 32K in Llama 2) • 8K context • trained on ~15T tokens • GQA across all sizes • strong reasoning and code
Llama 3.1 (July 2024)	`meta-llama/Llama-3.1-405B-Instruct`	• Adds 405B, expands context to 128K tokens, multilingual support (8 languages), native tool/function calling • 405B intended as teacher model for distillation • same dense architecture as Llama 3.
Llama 3.2 (September 2024)	`meta-llama/Llama-3.2-11B-Vision-Instruct`	• Adds 1B and 3B lightweight edge models + 11B and 90B vision models • first multimodal Llama • 128K context • vision models use cross-attention adapter architecture
Llama 3.3 (December 2024)	`meta-llama/Llama-3.3-70B-Instruct`	• Single-size 70B release matching near-405B performance at 70B compute cost • 128K context • 8-language support • text-only • released Dec 6, 2024.
Llama 4 Scout (April 2025)	`meta-llama/Llama-4-Scout-17B-16E-Instruct`	• First MoE Llama • 17B active / 109B total params • 16 experts • 10M-token context window (iRoPE architecture) • natively multimodal • fits on single H100 with INT4.
Llama 4 Maverick (April 2025)	`meta-llama/Llama-4-Maverick-17B-128E-Instruct`	• 17B active / 400B total params • 128 routed experts • 1M-token context • alternating dense and MoE layers • beats GPT-4o and Gemini 2.0 Flash on key benchmarks
Llama 4 Behemoth (announced, unreleased)	`N/A — still training as of 2025`	• 288B active / ~2T total params • 16 experts • used as teacher model for Scout and Maverick co-distillation • STEM benchmarks exceed GPT-4.5 and Claude 3.7 Sonnet • no public release date
Llama 1 (February 2023)	Sizes: 7B, 13B, 33B, 65B	• Original research-only release • 2K context window • non-commercial license • 1–1.4T token training corpus • weights initially leaked, later superseded by Llama 2.
Llama 2 (July 2023)	Sizes: 7B, 13B, 70B	• First commercially permissive release • 4K context window • trained on 2T tokens • introduced instruction-tuned chat variants (RLHF/SFT) • 40% more training data than Llama 1.

Table 2: Model Sizes and Parameter Tiers

Llama covers a wide range of sizes, each targeting a different deployment context. Matching model size to hardware and latency budget is the first practical decision when choosing a Llama model.

Model	Example	Description
1B (Llama 3.2)	`meta-llama/Llama-3.2-1B-Instruct`	• Designed for on-device / mobile inference • fits in ~2GB VRAM (FP16) • competitive with larger models on summarization and instruction following on-device

The rest of this section is part of the full cheat sheet

Table 3: Base vs. Instruct Variants

Every Llama size ships as both a base (pretrained) and instruct (fine-tuned) variant. Choosing the wrong one is a common source of poor results in production.

Variant	Example	Description
Base model	`meta-llama/Meta-Llama-3-8B`	• Pretrained on raw text • continues text rather than following instructions • used as a starting point for custom fine-tuning or few-shot prompting experiments

The rest of this section is part of the full cheat sheet

Table 4: Context Windows

Context window size has grown from 2K in Llama 1 to 10M in Llama 4 Scout. The practical limit at inference time depends on both the model's trained maximum and available KV-cache memory.

Model	Example	Description
128K tokens (Llama 3.1/3.2/3.3)	`max_new_tokens=131072`	• Standard context for all Llama 3.x text and vision models • enabled by high RoPE base frequency and GQA • KV cache for 128K at FP16: ~16GB (8B), ~39GB (70B), ~123GB (405B).

The rest of this section is part of the full cheat sheet

Table 1: Model Generations and Release Timeline

Each Llama generation introduced a major architectural or capability leap. Knowing which generation a model belongs to immediately signals its context window, multimodal support, and license terms.

Model	Example	Description
Llama 3 (April 2024)	`meta-llama/Meta-Llama-3-8B-Instruct`	• 8B and 70B dense decoder-only transformers • 128K-token vocabulary (up from 32K in Llama 2) • 8K context • trained on ~15T tokens • GQA across all sizes • strong reasoning and code
Llama 3.1 (July 2024)	`meta-llama/Llama-3.1-405B-Instruct`	• Adds 405B, expands context to 128K tokens, multilingual support (8 languages), native tool/function calling • 405B intended as teacher model for distillation • same dense architecture as Llama 3.
Llama 3.2 (September 2024)	`meta-llama/Llama-3.2-11B-Vision-Instruct`	• Adds 1B and 3B lightweight edge models + 11B and 90B vision models • first multimodal Llama • 128K context • vision models use cross-attention adapter architecture
Llama 3.3 (December 2024)	`meta-llama/Llama-3.3-70B-Instruct`	• Single-size 70B release matching near-405B performance at 70B compute cost • 128K context • 8-language support • text-only • released Dec 6, 2024.
Llama 4 Scout (April 2025)	`meta-llama/Llama-4-Scout-17B-16E-Instruct`	• First MoE Llama • 17B active / 109B total params • 16 experts • 10M-token context window (iRoPE architecture) • natively multimodal • fits on single H100 with INT4.
Llama 4 Maverick (April 2025)	`meta-llama/Llama-4-Maverick-17B-128E-Instruct`	• 17B active / 400B total params • 128 routed experts • 1M-token context • alternating dense and MoE layers • beats GPT-4o and Gemini 2.0 Flash on key benchmarks
Llama 4 Behemoth (announced, unreleased)	`N/A — still training as of 2025`	• 288B active / ~2T total params • 16 experts • used as teacher model for Scout and Maverick co-distillation • STEM benchmarks exceed GPT-4.5 and Claude 3.7 Sonnet • no public release date
Llama 1 (February 2023)	Sizes: 7B, 13B, 33B, 65B	• Original research-only release • 2K context window • non-commercial license • 1–1.4T token training corpus • weights initially leaked, later superseded by Llama 2.
Llama 2 (July 2023)	Sizes: 7B, 13B, 70B	• First commercially permissive release • 4K context window • trained on 2T tokens • introduced instruction-tuned chat variants (RLHF/SFT) • 40% more training data than Llama 1.

Table 2: Model Sizes and Parameter Tiers

Llama covers a wide range of sizes, each targeting a different deployment context. Matching model size to hardware and latency budget is the first practical decision when choosing a Llama model.

Model	Example	Description
1B (Llama 3.2)	`meta-llama/Llama-3.2-1B-Instruct`	• Designed for on-device / mobile inference • fits in ~2GB VRAM (FP16) • competitive with larger models on summarization and instruction following on-device

Table 3: Base vs. Instruct Variants

Every Llama size ships as both a base (pretrained) and instruct (fine-tuned) variant. Choosing the wrong one is a common source of poor results in production.

Variant	Example	Description
Base model	`meta-llama/Meta-Llama-3-8B`	• Pretrained on raw text • continues text rather than following instructions • used as a starting point for custom fine-tuning or few-shot prompting experiments

Table 4: Context Windows

Context window size has grown from 2K in Llama 1 to 10M in Llama 4 Scout. The practical limit at inference time depends on both the model's trained maximum and available KV-cache memory.

Model	Example	Description
128K tokens (Llama 3.1/3.2/3.3)	`max_new_tokens=131072`	• Standard context for all Llama 3.x text and vision models • enabled by high RoPE base frequency and GQA • KV cache for 128K at FP16: ~16GB (8B), ~39GB (70B), ~123GB (405B).