Force valid JSON from a local model with a GBNF grammar

My self-hosted model has no built-in Structured Outputs API, and its JSON breaks just often enough to ruin a batch job.

Intermediate to advancedRead 7 minDoing: an afternoon8 steps

What you'll have at the end

A local-model wrapper that compiles your JSON Schema into a GBNF grammar, tested on 20 prompts with zero parse failures.

Before you start

You need

A llama.cpp-served local model you can already call directly, through llama-cpp-python or llama-server, plus a JSON Schema or Pydantic model naming the exact fields a batch job needs to pull out of raw text.

Not covered

Installing llama.cpp or downloading a GGUF checkpoint, and improving how semantically correct the extracted values are once every reply already parses cleanly.

Leans on

8 steps

Where to go next

Turn a chatty reply into one clean JSON object

if the model behind your call is a cloud API with its own structured-output mode, start there instead of hand-building a grammar.

Checked 18 Aug 2026

Part of the Structured Output Generation with LLMs cookbook