In-context Learning Cheat Sheet

In-context learning (ICL) allows large language models to adapt to new tasks by embedding demonstrations directly within the input prompt, eliminating the need for parameter updates or fine-tuning. This paradigm enables models to learn from examples at inference time, making it a cornerstone technique for prompt engineering and rapid task adaptation. Unlike traditional training, ICL leverages the model's pre-existing knowledge to generalize from a small number of contextual examples—and as context windows have expanded into the millions of tokens, the many-shot regime now approaches fine-tuning performance without any weight updates. The effectiveness of ICL depends critically on example selection, ordering, formatting, and reasoning structure—subtle variations can dramatically impact model performance, making prompt design as consequential as model architecture itself. Theoretically, ICL is understood as an implicit Bayesian inference process, where the model selects or approximates a task-solving function from its pre-training distribution given the provided demonstrations.