Your RAG's retrieval hit rate has never been measured

I keep tweaking my RAG pipeline and have no idea if retrieval actually got better or worse.

Intermediate to advancedRead 5 minDoing: an afternoon to build the labeled set and measure once, a few minutes each time after that8 steps

▶ Start

What you'll have at the end

A written hit rate score for your RAG's top-5 retrieval, measured on 15 real questions

Before you start

You need

A working RAG pipeline whose retrieval step you can call directly with a single test question and get back an ordered list of chunks, outside of the full chat flow.

Not covered

Whether the generated answer is accurate or well-cited once the right chunk is retrieved; this only scores whether retrieval found the right chunk at all.

Leans on

8 steps

Where to go next

Add a reranker so RAG stops burying the right chunk

when the miss is a chunk ranked too low rather than missing outright, a reranker fixes rank without touching chunk content

RAG's vector search misses exact terms that hybrid search catches

when a miss is an exact term or code vector search cannot see at all, a different failure than crowded lookalikes

Checked 15 Aug 2026

Part of the RAG (Retrieval Augmented Generation) cookbook