How to measure latency in a voice AI application

Which latency are you measuring?

Name the start and the end of every number. “Latency” means different things to a team that measures a model call and a team that measures when a caller hears speech, and the two numbers can’t be compared.

Measurement	Starts	Ends
Caller-perceived reply delay	The caller stops speaking	The first audible reply at the caller’s ear
Turn detection	End of speech in the captured audio	The system commits the caller’s turn
Model first output	The model request starts	The first usable sentence reaches your code
TTS first byte	The speech request starts	The first audio bytes arrive where you measure
Playback start	The first usable audio arrives	The player actually starts

In a streaming pipeline these stages overlap. Adding up separately measured medians does not give you the median of the whole turn. Measure the whole path directly, then use the component timings to explain it.

What does the Mira TTS request log tell you?

Every speech request, completed or not, lands in a log you can read with GET /v2/tts/requests:

curl "https://sandbox.voice.miraiminds.co/v2/tts/requests?limit=50" \
  -H "Authorization: Bearer $MIRA_API_KEY"

Each row has first_byte_ms (how long Mirai took to send the first audio), total_ms, bytes, format, transport (http or websocket) and status: completed, failed, truncated or rejected. If those look normal and callers still hear gaps, the audio is being held up between Mirai’s server and the caller: on your link, in your event loop, or in how it is paced to the phone line.

The response format changes the first-byte number. A wav response arrives in one piece once synthesis finishes, so its first byte comes at the end. Use pcm, mulaw or alaw when you care about time to first audio. Save the X-Mira-Model header with each test so you don’t compare two model builds as if only your application had changed.

What does the Realtime API report per turn?

A Realtime session runs recognition, turn detection, the model and speech on Mirai’s side. Every response.done event carries the turn’s numbers in response.metadata, as strings:

Key	Meaning
mirai_v2v_ms	End of the caller’s speech to the first audio of the reply, on Mirai’s side
mirai_stt_ms	Speech recognition’s first result for the turn
mirai_llm_ms	The model’s first output
mirai_tts_ms	Speech synthesis’s first audio
mirai_stt_provider, mirai_llm_provider, mirai_tts_provider	mira, or the fallback that served that step
mirai_fallback	"true" when any step of the turn ran on a fallback

A time that wasn’t measured for a turn is left out. Add your own network time to mirai_v2v_ms to get what the caller hears, and split turns that used a fallback from the rest before you compare distributions. The pipecat-mirai package passes these numbers to an on_turn_metrics handler; see the Realtime guide.

How do you build a representative test?

Use one test set and one set of acceptance criteria for every candidate. Include short acknowledgements, long answers, Hindi-English switching and the phrases that matter to your product. Run it over the transport you will use in production, not only on a quiet local link.

1. Define where each timer starts and stops.
2. Record the client location, network, output format, sample rate, model build, voice and concurrency.
3. Run the whole pipeline, including the player or the receiving phone.
4. Keep failures and timeouts in the results.
5. Report the distribution and the sample size, next to task completion and intelligibility.
6. Repeat after any meaningful configuration or model change, with the same setup.

The pipecat-mirai repository publishes its end-to-end test harness, which measures time from the model’s first token to first audio at the phone. You can run it from your own location and compare against your own numbers.

Why can a fast speech endpoint still feel slow?

Much of the wait can happen outside the TTS call. Common causes:

• The application waits for the model’s whole answer before sending any text to speech, then waits for all the audio before playing it.
• Each sentence opens a new HTTPS connection. The docs put the TCP and TLS handshake at 0.4 to 1 second from India. Keep one WebSocket per call, or one keep-alive HTTP client.
• 48 kHz PCM needs 768 kbit/s per stream. Six calls at that rate need a steady 4.6 Mbit/s, and an ordinary server link can fall behind real time. A phone line carries 8 kHz anyway, so ask for ulaw_8000.
• Audio is sent to the phone provider at exactly real time, so the provider holds about 40 ms. Any stall on your server longer than that breaks the voice up. The docs recommend keeping the provider about 0.4 seconds ahead and holding about 150 ms before each sentence starts.
• After the caller interrupts, queued audio keeps playing. Cancel the reply and clear the provider’s buffer, and make sure a late network response cannot restart an interrupted turn.

Smaller text chunks can start speech earlier but may lose context the voice needs for natural phrasing. Change one setting at a time and check quality along with timing.

What should a published benchmark include?

For a worked example of a measurement write-up, read our Gemma 4 26B A4B serving measurements. Its download includes the archived request timings and a reanalysis script. Those numbers cover the model request, not the full audio path described here.

Publish the method with the number: workload, language, client and server locations where known, output format, concurrency, sample size, timing boundaries and how failures were counted. Say what the test leaves out. A best-case sample is not a guarantee, and a design diagram is not a measured comparison.

Mirai’s voice measurement journal describes the same principle for releases: compare what customers would hear after a change with what they hear today, across turns.