AI Evaluation Frameworks and LLM Benchmarking Cheat Sheet

Updated 2026-08-01

Next Topic: AI for Scientific Discovery Cheat Sheet

🧠Study flashcards on this topic113 cards · spaced repetition→

AI evaluation frameworks and LLM benchmarking turn model behavior into measurable evidence across knowledge, reasoning, coding, retrieval, safety, cost, and production reliability. Practitioners use them to compare models, catch regressions, design custom quality gates, and decide whether a system is ready for users. The key mental model is measurement triangulation: public benchmarks reveal broad capability, task-specific evals reveal product fit, and production telemetry reveals whether the system still works after real users, data drift, and prompt changes enter the loop.

What This Cheat Sheet Covers

This topic spans 13 focused tables and 113 indexed concepts, 113 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.

Table 1: Evaluation Paradigms and Benchmark RolesTable 2: General Capability BenchmarksTable 3: Coding and Agent BenchmarksTable 4: Safety and Red-Team Benchmark SuitesTable 5: RAGAS and RAG MetricsTable 6: Evaluation Frameworks and ToolsTable 7: LLM-as-a-Judge PatternsTable 8: Pairwise Ranking and Preference MethodsTable 9: Custom Metric and Rubric DesignTable 10: Evaluation Dataset ConstructionTable 11: Contamination and Overfitting RisksTable 12: Human, Blind, Automated, and Hybrid EvaluationTable 13: CI, Release Gates, and Production Monitoring

Quick IndexSubscribe to unlock

A jump-to index of every table row in this cheat sheet.

Mind MapSubscribe to unlock

An interactive map of every table and concept in this topic.

Table 1: Evaluation Paradigms and Benchmark Roles

Evaluation starts by choosing the right kind of evidence. Public benchmarks, offline test sets, human review, LLM judges, and production experiments answer different questions, so a reliable evaluation program usually combines several rather than treating one leaderboard as truth.

Paradigm	Example	Description
Benchmark Evaluation	`run HELM scenario -> compare model scores`	Standardized tasks with fixed protocols for cross-model comparison.
Offline Evaluation	`dataset + candidate_prompt -> scored report`	Repeatable pre-release testing on held-out examples.
Regression Evaluation	`old_score=0.84` `new_score=0.79 -> block`	Detects quality drops after model, prompt, tool, or retrieval changes.
Human Evaluation	`3 annotators rate helpfulness 1-5`	Expert or crowd judgments for subjective quality.
LLM-as-a-Judge	`judge("Rate faithfulness from 1 to 5")`	Uses another model to score outputs against criteria.
Online Evaluation	`variant_b wins 58 percent of live votes`	Measures behavior on real traffic, feedback, or A/B tests.
Component Evaluation	`retriever_recall@5 = 0.82`	Tests one pipeline part before end-to-end scoring.
Adversarial Evaluation	`redteam eval -> injection pass rate`	Probes safety, jailbreak, leakage, and robustness failures.

Table 2: General Capability Benchmarks

General benchmarks test broad model ability, but their signal depends on freshness, task difficulty, and whether scores are saturated. Use them to understand capability coverage, not to predict product performance by themselves.

Benchmark	Example	Description
MMLU	`57 subjects` `multiple-choice accuracy`	• Broad academic and professional knowledge benchmark • largely saturated at the frontier, which is why MMLU-Pro and HLE exist.

The rest of this section is part of the full cheat sheet

Table 3: Coding and Agent Benchmarks

Coding benchmarks range from isolated function generation to full software-engineering and tool-using agents. Repository-level, terminal, and multi-turn tool-use benchmarks now matter more for agent selection than saturated function-completion scores.

Benchmark	Example	Description
HumanEval	`164 Python tasks` `pass@1`	Function-level code generation benchmark using unit tests.

The rest of this section is part of the full cheat sheet

Table 4: Safety and Red-Team Benchmark Suites

Safety benchmarks put a number on how often a model or agent complies with harmful requests, resists jailbreaks, or breaks an explicit policy. They turn one-off red-team findings from a security exercise into a repeatable, comparable score you can track release over release.

Benchmark	Example	Description
AILuminate	`12 hazard categories -> graded safety rating`	Industry-standard general-chat safety benchmark from MLCommons across violent, sexual, and self-harm hazard categories.

The rest of this section is part of the full cheat sheet

Table 1: Evaluation Paradigms and Benchmark Roles

Paradigm	Example	Description
Benchmark Evaluation	`run HELM scenario -> compare model scores`	Standardized tasks with fixed protocols for cross-model comparison.
Offline Evaluation	`dataset + candidate_prompt -> scored report`	Repeatable pre-release testing on held-out examples.
Regression Evaluation	`old_score=0.84` `new_score=0.79 -> block`	Detects quality drops after model, prompt, tool, or retrieval changes.
Human Evaluation	`3 annotators rate helpfulness 1-5`	Expert or crowd judgments for subjective quality.
LLM-as-a-Judge	`judge("Rate faithfulness from 1 to 5")`	Uses another model to score outputs against criteria.
Online Evaluation	`variant_b wins 58 percent of live votes`	Measures behavior on real traffic, feedback, or A/B tests.
Component Evaluation	`retriever_recall@5 = 0.82`	Tests one pipeline part before end-to-end scoring.
Adversarial Evaluation	`redteam eval -> injection pass rate`	Probes safety, jailbreak, leakage, and robustness failures.

Table 2: General Capability Benchmarks

Benchmark	Example	Description
MMLU	`57 subjects` `multiple-choice accuracy`	• Broad academic and professional knowledge benchmark • largely saturated at the frontier, which is why MMLU-Pro and HLE exist.

Table 3: Coding and Agent Benchmarks

Benchmark	Example	Description
HumanEval	`164 Python tasks` `pass@1`	Function-level code generation benchmark using unit tests.

Table 4: Safety and Red-Team Benchmark Suites

Benchmark	Example	Description
AILuminate	`12 hazard categories -> graded safety rating`	Industry-standard general-chat safety benchmark from MLCommons across violent, sexual, and self-harm hazard categories.