Retry a rate-limited API call with backoff instead of a crash

My script works fine in testing, then dies the moment I send more than a couple of requests in a row.

Beginner to intermediateRead 5 minDoing: about an hour, most of it spent watching the wrapper actually hit and recover from a real limit7 steps

▶ Start

What you'll have at the end

A request wrapper that retries a 429 or timeout with growing backoff, then fails cleanly after a capped number of tries.

Before you start

You need

A script that already sends real requests to an LLM API and works fine one request at a time.

Not covered

This doesn't build a circuit breaker that stops calling entirely during a sustained outage; that's a separate layer for a system that keeps failing well past what backoff alone can fix.

Leans on

7 steps

Where to go next

Test your API wrapper without spending real tokens

when you want to unit-test this same retry loop's edge cases without burning real credit triggering real rate limits every time

Checked 15 Aug 2026

Part of the LLM APIs and Integration cookbook