Structured Output Generation with LLMs cookbook

Every prompt asks nicely for JSON. Half the replies still bury it inside a paragraph.

The router keeps inventing categories I never gave it.

Why does the model invent a phone number that was never in the text?

I built a Pydantic model with a couple of optional fields, wired it into a schema, and the strict-mode API call keeps rejecting it without saying which field is wrong.

My parser crashed on a completely ordinary question, and it turned out the model had refused and sent back no data at all.

The model's JSON is almost right every time, one field off, and I keep fixing it by hand instead of just asking again.

Someone always says they'll follow up. By next week, nobody remembers who.

A support ticket comes in with a customer's card number pasted right into the message, and nobody flags it before the model sees it.

My structured extraction cites a phrase from the document, and the fact it pulled out still is not actually in it.

The same customer shows up five different ways in my data, and I'm not sure the LLM's merge decisions are ones I can trust.

Users stare at a blank spinner for seconds while the whole structured reply finishes, when half the fields were ready much sooner.

The schema that works fine against one provider gets rejected outright the moment I point the same request at the other.

My self-hosted model has no built-in Structured Outputs API, and its JSON breaks just often enough to ruin a batch job.