Judge your agent's tool calls, not its final answer alone

My agent got the right answer, but it called the search tool four times when one call was enough.

Intermediate to advancedRead 7 minDoing: a couple of hours7 steps

What you'll have at the end

A per-step scorecard showing which tool calls were right and which were not

Before you start

You need

A per-call log for at least one agent run: for each tool call, the tool's name, the arguments it was sent, and the result it returned, in order.

Not covered

Whether the final answer itself was accurate; this scorecard only judges the calls that produced it.

Leans on

7 steps

Log every tool call so you can find what broke a run

go there first if you don't yet have a per-call log to score; this recipe assumes one already exists.

Write your first LLM-as-a-judge grading script

start there first if you've never written a graded LLM-judge prompt before; this recipe assumes you can build one for the necessity check.

Turn user complaints into regression tests

go there instead when the trajectory bug you're chasing came from a real user complaint rather than your own scorecard.

Checked 15 Aug 2026