Turn latency is the gap your caller actually feels: the milliseconds between end of their speech and start of the agent’s reply. When it creeps past ~700 ms the call stops feeling conversational — the caller talks over the agent, repeats themselves, or hangs up. The good news is that turn latency is almost always the sum of a small number of hops, and each hop has a specific fix. Work top-down: measure where the time goes first, then attack the biggest hop. Guessing wastes time because the dominant cost is rarely the one you’d assume.
This page is about the turn budget — the round-trip from silence to first audio back. If instead the audio is delayed but steady (a fixed offset, or choppy/robotic playback), that’s a media-plane problem: see One-way / choppy audio and MOS, jitter & loss.

​
Measure before you tune

Turn latency is a chain, and every hop is attributed for you. Open the per-turn latency breakdown for a recent call and read the hop-by-hop split before changing anything. A typical cascaded turn decomposes into:
HopWhat it isWho owns it
End-of-turn detectionTrailing silence the runtime waits out before it decides the caller is doneYour turn_detection config
Transport inCaller audio → relay → runtimeNetwork + transport
ASR finalizeStreaming partials → final transcriptASR provider
LLM time-to-first-tokenPrompt sent → first token backLLM provider
TTS time-to-first-audioText → first synthesized frameTTS provider
Transport outFirst agent frame → callerNetwork + transport
A speech-to-speech (realtime) agent collapses ASR + LLM + TTS into a single provider round-trip, so its breakdown is much shorter. Identify your longest hop in the breakdown, then jump to the matching row below.

​
Symptom → cause → fix

1

Read the breakdown and find the dominant hop

Pull up the latency breakdown for a slow turn. Whichever hop is largest is your target — the rest is noise until you fix it.
2

Match it to a cause below

Use the table to map the dominant hop to a root cause and a concrete fix.
3

Change one thing, re-measure

Apply a single fix, place a fresh test call, and re-read the breakdown. Turn latency fixes compound, so verify each one before stacking the next.
Symptom (dominant hop)Likely causeFix
LLM + ASR + TTS each add a round-tripCascaded pipeline serializes three separate provider hopsMove to a speech-to-speech provider — one round-trip instead of three
One provider hop dominates (ASR or LLM or TTS)That provider’s own round-trip / region is slow, or it’s non-streamingPick a streaming client, a nearer provider region, or a faster model
First turn of every call is slow, later turns snappyCold model / cold session — the provider connection or self-hosted pod warms up on the first turnRely on session prewarm; keep self-hosted pods warm
Every hop is a bit slow, tail is spiky (p99 ≫ p50)Region / relay distance, or a TCP/WebSocket fallback legColocate the media plane near the provider; confirm you’re on QUIC, not the fallback
Agent replies feel late but the provider hops are fastEnd-of-turn detection is waiting out too much silenceLower silence_duration_ms
Small, constant overhead on phone legs onlyCodec transcode / resampling between G.711, PCM16, and OpusUsually leave it — it’s sub-frame; only matters as codec passthrough for bridges

​
Cascade vs speech-to-speech

This is the single biggest lever. A cascaded agent (ASR → LLM → TTS) pays three serialized provider round-trips per turn: the transcript must finalize before the LLM starts, and the LLM must emit text before TTS can synthesize. A speech-to-speech (realtime) agent streams audio straight into one model that emits audio back, collapsing all three into a single bidirectional stream — often the difference between a ~1.2 s and a sub-500 ms turn.

Switch to speech-to-speech

When to use a realtime model and how to wire one up. Options include OpenAI Realtime, Gemini Live, xAI Grok Voice, ElevenLabs Conversational AI, and a self-hosted realtime model.

Keep a cascade, make it fast

If you need the flexibility of separate ASR/LLM/TTS, the fixes below keep a cascade competitive.
Cascades earn their keep when you need per-stage control — a specific ASR for an accent or language, a domain-tuned LLM, a particular TTS voice. If you don’t, speech-to-speech is the faster default.

​
Provider round-trip

If one hop dominates and it’s an individual provider, the latency is coming from that provider’s own service, not from us. Three fixes, in order of impact:
1

Use streaming clients, not batch

Streaming ASR emits partials and finalizes fast; streaming TTS returns time-to-first-audio in a few frames instead of after the whole utterance. A non-streaming HTTP LLM is the worst offender — it returns nothing until the full response is generated, so the caller waits out the entire completion. Prefer a streaming or realtime model for any latency-sensitive agent.
2

Move the provider region closer

Every provider hop crosses the network twice. If your agent runs in one region and the provider’s endpoint is in another, that geography is baked into every turn. Point the provider at its nearest region to your media plane.
3

Pick a faster model

A smaller or faster-tier model at each stage trims time-to-first-token / time-to-first-audio directly. Latency and quality trade off here — tune per agent.
A non-streaming LLM also breaks clean barge-in: the runtime can silence playback the instant the caller interrupts, but the provider still finishes — and bills — the full response. See Turn detection & barge-in for why.

​
Region and relay distance

Media rides QUIC / media-over-QUIC to the relay at relay.telequick.dev, then on to your agent runtime and providers. Two distances matter, and they add up:
  • Caller → relay. Callers connect to the nearest edge; browser and app legs auto-reconnect with capped backoff if a path degrades. This hop is usually small.
  • Runtime ↔ provider. The larger, more variable cost. If your agent runtime and your ASR/LLM/TTS providers sit in different regions, every turn pays that crossing three times over on a cascade.
Check you’re actually on QUIC. QUIC / media-over-QUIC is the first-class transport; WebSocket and WebRTC are fallbacks for Safari, UDP-blocked networks, and TCP-only proxies. A fallback leg can roughly double turn latency and blow out the tail. In a WAN prototype measured at 50 ms link RTT with ~1% loss, turns ran p50 ~91 ms over QUIC vs ~193 ms over TCP+TLS, with a far cleaner tail (these are prototype-rig numbers, not a production SLA). If your tail latency is spiky, confirm the leg didn’t fall back to TCP.
For colocated agents, keep the runtime and the providers it calls in the same region so the runtime ↔ provider hop stays local.

​
Cold models

The first turn of a call can be slow while everything downstream warms up. The runtime already fights this for you:
  • Session prewarm. For realtime agents, the runtime opens and warms the provider session during the inbound INVITE / SDP exchange — before the caller even finishes connecting — so the first turn doesn’t pay connection setup.
  • Continuous prefill (self-hosted). A warm, continuously-prefilled self-hosted realtime model can return time-to-first-audio in roughly 0–15 ms once the session is live.
The failure mode is a cold self-hosted pod. If you run your own inference and the GPU pod scaled to zero, the first call after idle pays the full model cold-start — which shows up as a slow first turn, then fast turns after. Keep a warm floor of capacity for latency-sensitive traffic.

Self-hosted inference

How the on-prem inference control plane routes to your model pods, and how to keep capacity warm.

​
End-of-turn detection

Sometimes the providers are fast and the agent still feels late — because the runtime is politely waiting out silence before it decides the caller is done. That wait is silence_duration_ms (default 500). Every millisecond here is added to the front of every turn.
  • Lower it (~300 ms) for snappier replies with terse callers.
  • Raise it (~800 ms) for callers who pause mid-thought, if the agent keeps cutting them off.
This is a direct latency-vs-interruption trade, not a bug. Tune it against real calls.

Turn detection & barge-in

The full set of end-of-turn and barge-in timings, including silence_duration_ms, prefix_padding_ms, and hold-and-confirm barge-in.

​
Codec transcode

Phone legs arrive as G.711 (µ-law / A-law); the agent bus runs on narrowband PCM16; browser and app legs use Opus. The runtime transcodes and resamples between these in-process. In practice this is sub-frame and not your latency problem — chasing it is almost never worth it. The one place codec choice does matter is bridging: when you can keep the same codec end to end — for example passing Twilio Media Streams µ-law straight through to a PCMU trunk — you skip transcode entirely on that path. See Codecs and Migrate from Twilio for the passthrough fast path.

Latency breakdown

The hop-by-hop attribution to read first — where every millisecond of a turn actually goes.

Choose a runtime

Cascade vs speech-to-speech vs bring-your-own, and the latency each implies.

Barge-in not working

The other half of turn timing — when the agent won’t yield to the caller.

MOS, jitter & loss

When the problem is audio quality rather than turn speed.