New: Cookbooks and AI ExplanationsStep-by-Step recipes to solve problems connected to Roadmaps and Cheat Sheets. Need more details? Use AI buttons for structured and simple explanations with concrete examples throughout the whole platform.Take a look
Every part of your app calls the model directly, so a rate limit, an outage, or a repeated question costs you separately each time.
What you'll have at the end
One function every part of your app calls, which serves cached answers, routes by task, and switches provider on failure
You need
At least two places in your app that already call a model provider directly, one where the same question tends to repeat and one where the provider sometimes fails, plus somewhere to store a value by a key every call site can reach.
Not covered
Deciding exactly which errors deserve a retry versus a circuit breaker, or matching two differently worded questions as the same cache entry: both slot into one part of this function rather than replacing it.
Your app breaks the moment your LLM provider does
Go there for the fuller failure classification, backoff timing, and circuit breaker this recipe's call step assumes you can already build or reuse.
Route your easy requests to a cheaper model
Use this instead when the split isn't which task is running but how confident a single tier's own answer looks, since that needs a graded set and a cutoff this recipe doesn't build.
Cache semantically similar prompts instead of paying twice
Add this once the exact-match cache here is live and you want two differently worded questions to share one entry too.
Checked 15 Aug 2026
Part of the AI Engineering cookbook