In-context learning (ICL) allows large language models to adapt to new tasks by embedding demonstrations directly within the input prompt, eliminating the need for parameter updates or fine-tuning. This paradigm enables models to learn from examples at inference time, making it a cornerstone technique for prompt engineering and rapid task adaptation. Unlike traditional training, ICL leverages the model's pre-existing knowledge to generalize from a small number of contextual examples—and as context windows have expanded into the millions of tokens, the many-shot regime now approaches fine-tuning performance without any weight updates. The effectiveness of ICL depends critically on example selection, ordering, formatting, and reasoning structure—subtle variations can dramatically impact model performance, making prompt design as consequential as model architecture itself. Theoretically, ICL is understood as an implicit Bayesian inference process, where the model selects or approximates a task-solving function from its pre-training distribution given the provided demonstrations.
What This Cheat Sheet Covers
This topic spans 13 focused tables and 121 indexed concepts, 121 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
A jump-to index of every table row in this cheat sheet.
An interactive map of every table and concept in this topic.
Table 1: Core ICL Approaches
The foundational shot-count spectrum of ICL—from no examples to thousands—determines the trade-off between annotation cost, prompt length, and task performance; knowing when each regime is appropriate is the starting point for any ICL practitioner.
| Technique | Example | Description | |
|---|---|---|---|
"Happy" → Positive"Terrible" → Negative"Excited" → Positive"Amazing" → | • Includes 2–10 labeled examples in the prompt to guide the model's understanding of the task • the standard ICL approach for most applications. | ||
Classify: "I loved this movie!" → Sentiment: | • Direct task instruction without any demonstrations • relies entirely on the model's pre-training knowledge and instruction-following capability. | ||
Q: "What is 2+2?" A: "4"Q: "What is 5+3?" A: | • Provides exactly one example to demonstrate the desired input-output pattern • useful when task format is novel but the pattern is simple. | ||
Hundreds to thousands of examples within extended context windows (e.g., 100K+ tokens) | • Leverages extremely long context windows to provide dozens or hundreds of demonstrations • approaches fine-tuning performance without weight updates and overrides pre-training biases • gains plateau around 50–70 examples per class for classification; open-ended generation tasks see minimal improvement. | ||
Provide 50 demonstrations where chain-of-thought rationales are auto-generated by the model via zero-shot CoT rather than human-written | • Uses model-generated rationales instead of costly human-annotated ones in many-shot prompts • effectively breaks the human-annotation bottleneck while maintaining reasoning quality. | ||
Provide hundreds of raw input-output pairs with no explanations or reasoning steps | • Removes rationales entirely • surprisingly effective in the many-shot regime as the model extracts task patterns from raw I/O pairs alone • requires no annotation. | ||
Schedule tasks with human-inspired spacing (distributed practice) across prompts; interleave examples from earlier tasks with new ones to prevent forgetting | • Extends ICL to sequential multitask settings without parameter updates by leveraging task scheduling and prompt rearrangement • benefits from a spacing "sweet spot" analogous to human distributed practice, reducing cross-task interference. |