Make your voice agent start talking before the reply finishes

My voice bot goes dead silent for three seconds while the model finishes talking, then dumps the whole answer at once.

AdvancedRead 7 minDoing: a day of build-and-test work8 steps

What you'll have at the end

Your voice agent's first spoken word lands within a few hundred milliseconds of the model's first token, measured end to end.

Before you start

You need

a voice agent that streams the language model's reply token by token, plus a speech engine that accepts incremental text input, currently synthesizing audio only after the whole reply has been assembled

Not covered

how quickly the system decides the caller has stopped talking, and what happens when the caller interrupts mid-reply, since both are separate problems layered around this same pipeline

Leans on

8 steps

Where to go next

Hit a latency budget for your live speech captions

when the delay you're chasing is in captions reaching the screen rather than the spoken reply, since that's a separate pipeline stage with its own budget

Checked 18 Aug 2026

Part of the Multimodal AI cookbook