Hit a latency budget for your live speech captions

Users say the captions are always a beat behind, and I don't even know what number to measure to fix it.

Intermediate to advancedRead 5 minDoing: an afternoon8 steps

▶ Start

What you'll have at the end

A measured time-to-first-word and time-to-final number for your captions, both under a limit you set ahead of time.

Before you start

You need

a live captioning pipeline that streams audio to a speech-to-text service and displays whatever text it returns, partial or final, as captions on screen

Not covered

the separate latency budget a full voice agent's spoken reply needs, since that pipeline layers a language model and a speech synthesizer on top, each with its own budget math

Leans on

8 steps

Where to go next

Make your voice agent start talking before the reply finishes

captioning a voice agent's own spoken reply splits this same budget three ways, adding the language model and the speech synthesizer on top of the transcript

Checked 18 Aug 2026

Part of the Multimodal AI cookbook