Hallucinations in large language models are confident but factually incorrect, nonsensical, or ungrounded responses—a fundamental challenge that emerges from the probabilistic nature of token-by-token prediction in transformer architectures. Preventing hallucinations requires grounding outputs in verifiable sources, constraining generation behavior, and implementing multi-layered verification rather than relying solely on the model's training. Modern RAG pipelines have advanced substantially, with GraphRAG, speculative RAG, active retrieval (FLARE), and multimodal grounding significantly extending what's achievable in 2026. The key insight: effective hallucination prevention is an orchestration problem, combining prompt design, retrieval mechanisms, sampling strategies, and post-generation validation into a coherent system where each layer compensates for the others' weaknesses.
What This Cheat Sheet Covers
This topic spans 12 focused tables and 98 indexed concepts, 98 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
Table 1: Grounding TechniquesTable 2: Prompt Engineering StrategiesTable 3: Sampling and Decoding ParametersTable 4: Post-Generation VerificationTable 5: Training and Fine-Tuning ApproachesTable 6: Architectural and Model-Level StrategiesTable 7: Confidence and Uncertainty EstimationTable 8: Human-in-the-Loop and OversightTable 9: Evaluation and Detection MetricsTable 10: Advanced Detection MethodsTable 11: Monitoring and Production StrategiesTable 12: Testing and Red-Teaming
Quick IndexSubscribe to unlock
A jump-to index of every table row in this cheat sheet.
Mind MapSubscribe to unlock
An interactive map of every table and concept in this topic.
Table 1: Grounding Techniques
Grounding keeps model outputs anchored to real, retrievable evidence rather than parametric knowledge alone. Choosing the right retrieval architecture — basic RAG, hybrid, graph-based, or active — is the single highest-leverage decision in any hallucination-reduction system.
| Technique | Example | Description | |
|---|---|---|---|
query = "Tesla revenue 2025"docs = retriever.search(query)prompt = f"Based on: {docs}, answer: {query}" | • Retrieves relevant documents from external knowledge base before generation, anchoring responses in retrieved facts • reduces hallucination rate by over 40% when properly implemented. | ||
results_kw = bm25.search(query, top_k=20)results_vec = vector_db.search(embed(query), top_k=20)fused = reciprocal_rank_fusion(results_kw, results_vec) | • Combines sparse keyword (BM25) and dense semantic search via Reciprocal Rank Fusion (RRF) • BM25 catches exact terms and IDs that embeddings miss; dense search catches paraphrases keyword search misses. | ||
candidates = retrieve_top_k(query, k=50)reranked = cross_encoder.score(query, candidates)context = reranked[:5] | • Uses a cross-encoder model to re-score retrieved candidates against the query, selecting highest-precision passages • narrows a large recall pool to the most relevant context before generation. | ||
# Model decides when to retrieveoutput = self_rag_model.generate(prompt)# Reflection tokens: [Retrieve], [IsREL], [IsSUP] | • Fine-tuned LLM decides on-demand whether to retrieve and critiques its own output via learned reflection tokens • outperforms static RAG by skipping retrieval when unnecessary and self-verifying generated claims. | ||
score = retrieval_evaluator(query, docs)if score < threshold: docs = web_search(query)answer = llm(query, docs) | • Lightweight retrieval evaluator scores retrieved documents; if confidence is low, falls back to web search • applies a decompose-then-recompose algorithm to strip noise before passing context to the LLM. | ||
tokens, probs = llm.generate_with_probs(prompt)if min(probs) < threshold: new_docs = retrieve(form_query(tokens)) regenerate(prompt, new_docs) | • Forward-Looking Active Retrieval — iteratively predicts upcoming tokens and triggers new retrieval when low-confidence tokens are detected during generation • unlike static RAG, retrieves multiple times throughout long-form generation only when needed. | ||
chunk_with_context = add_document_context(chunk)embed_and_store(chunk_with_context)retrieve_with_full_context(query) | • Prepends document-level context to each chunk before embedding, improving retrieval accuracy by ensuring chunks retain meaning when retrieved in isolation • reduces loss of context from chunking. | ||
small_chunks = split(doc, size=100)for chunk in small_chunks: store_child(chunk, parent_id=doc.id)# Retrieve child, return parent | • Embeds small child chunks for high-precision retrieval, but returns the larger parent document for richer context • combines surgical recall precision with the broad context the LLM needs to avoid fabrication. | ||
variants = llm(f"Generate 5 rephrasings of: {query}")results = [retriever.search(v) for v in variants]context = deduplicate(results) | • Generates multiple semantically equivalent query variants to increase retrieval recall, bridging wording gaps between user question and document vocabulary • especially effective for vague queries in BM25/keyword search. | ||
graph = extract_kg(documents)communities = build_community_hierarchy(graph)answer = llm(query, community_summaries) | • Extracts a knowledge graph with entity relationships and builds community hierarchy summaries for global understanding • superior to vector RAG for multi-hop reasoning and queries requiring connections across multiple documents or entities. | ||
hyp_doc = llm(f"Write a passage answering: {query}")hyp_embed = embed(hyp_doc)results = vector_db.search(hyp_embed, k=5) | • Generates a hypothetical answer, embeds it, and retrieves real documents nearest to that embedding • bridges the query-document semantic gap for vague or sparse queries. | ||
drafts = [specialist_lm(docs_subset) for docs_subset in doc_partitions]scores = generalist_lm.score(drafts)return drafts[argmax(scores)] | • Uses a smaller specialist LM to generate draft answers in parallel from document subsets, then a larger generalist LM verifies and selects the best draft • achieves up to 12.97% accuracy gain while reducing latency by 51% vs. standard RAG. | ||
kg = build_knowledge_graph(query)context = kg.get_connected_facts()answer = llm(query, context) | • Leverages structured relational data from knowledge graphs for explicit entity-relationship context • particularly effective for multi-hop reasoning where facts must be connected across nodes. | ||
embedding = embed(query)chunks = vector_db.similarity_search(embedding, k=5)context = "\n".join(chunks) | • Uses semantic similarity search over embedded document chunks to find relevant context • enables real-time knowledge retrieval from proprietary or frequently updated data sources. | ||
results = web_search_api.query(question)facts = extract_snippets(results)answer = llm(question, facts) | • Fetches current, up-to-date information from the internet at inference time • critical for time-sensitive queries where training data is outdated. | ||
results = retriever.search(query, filters={"date": ">2024", "source": "verified"})context = results[:5] | • Applies structured metadata rules (source, date, document type, region) to restrict retrieval before or alongside semantic search • prevents stale or off-scope documents from entering the prompt, reducing irrelevant-context hallucination. | ||
# ColPali: embed page as imagepage_vecs = colpali.embed(render_page(pdf, 34))hits = index.maxsim_search(query_vec, page_vecs) | • Treats images, tables, charts, and diagrams as first-class retrieval objects alongside text • eliminates hallucination caused by text-only parsing losing visual information (chart values, schematic connections). | ||
cache_key = embed(query)if cache.exists(cache_key): return cache.get(cache_key) | • Stores verified responses indexed by semantic similarity • reuses proven-accurate answers for similar questions, bypassing generation entirely. |