What you'll have at the end

A noise-sensitivity score for two different top-k settings, showing which one gives correct answers.

Before you start

You need

A RAG pipeline where you can change top-k independently of everything else, plus a small set of real questions each with one clear correct answer.

Not covered

Whether to change your embedding model or chunking strategy instead of top-k; this only measures the top-k change you're already weighing.

Leans on

RAG (Retrieval Augmented Generation) Cheat SheetDefines the noise-sensitivity metric and its 0-to-1 scale, the exact number this recipe computes by hand.
RAG Evaluation Cheat SheetLists the top-k tuning technique this recipe runs, plus the reranking option the final decision can route to.
AI-LLM App Evaluation Cheat SheetThis step's own topic sheet; the wider set of RAG-specific metrics for when noise sensitivity alone doesn't explain a bad answer.

Where to go next

Build a golden test set instead of eyeballing prompt changes

start there first if you don't yet have a small set of real questions with known-correct answers to run this comparison against

Score your RAG's retrieval and generation as two separate problems

once a wide top-k is hurting more than one claim per answer, this splits every failure into a retrieval problem or a generation problem instead of one noise number

Add a reranker so RAG stops burying the right chunk

if the wider top-k's recall gain is worth keeping, this adds the reranker that keeps noisy chunks from reaching the generator in the first place

Measure your RAG's noise sensitivity before you raise top-k

Before you start

Where to go next

Measure your RAG's noise sensitivity before you raise top-k

Before you start

Where to go next