Latency Breakdown

There is exactly one latency number a caller feels: how long the line stays silent after they stop talking before the agent starts talking back. Everything on this page exists to explain that number, break it into hops you can attack one at a time, and show you how to measure yours instead of trusting ours. We call it turn latency: the wall-clock gap from end-of-user-speech to first audio-out. It is the sum of five hops — capture, encode, transport, runtime, egress — plus one thing people forget to count: the endpointing window your turn detector waits before it even decides the user is done.

​

The five hops

A turn walks the same path in both directions. The user’s audio climbs capture → encode → transport → runtime; the reply comes back runtime → transport → egress. Here is where the time goes and what actually dominates each hop.

​

Capture and encode

On a browser or app leg the audio pipeline is the browser’s own — reused, not replaced. A loopback RTCPeerConnection runs echo cancellation, gain control, noise suppression, the Opus encoder, and the jitter buffer; an RTCRtpScriptTransform in a Worker taps the encoded Opus frames and hands them to the transport. See WebRTC diversion for the capture path in detail. The floor here is one Opus frame — 20 ms — plus whatever depth the browser’s jitter buffer settles to on your network. On a phone leg there is no browser pipeline: G.711 arrives in 20 ms RTP packets, the media plane decodes to 8 kHz PCM16, and noise suppression runs on ingress before the audio reaches the runtime. There is deliberately no acoustic echo cancellation or AGC in the media path — PSTN legs lean on VAD-gating instead, which is a correctness choice, not a latency one.

​

Transport: QUIC earns its place here

This is the hop where our transport choice shows up as a number. In a controlled prototype rig over a WAN link (50 ms RTT, ~1% loss), measuring the same turn — end-of-user-speech to start-of-agent-speech — over QUIC versus a TCP+TLS baseline: QUIC roughly halves the turn and, more importantly, keeps the tail flat — p99 sits right on p50 because head-of-line blocking is gone. Under concurrency (c = 20) the gap widens where it hurts most: time-to-first-token p50 93.9 ms vs 143.6 ms (−35%), and p99 149.2 ms vs 1197.9 ms — an ~8× better tail — with ~60% more throughput.

​

The browser diversion tax

Reusing the browser’s encoder and shipping the tapped frames over QUIC instead of native SRTP/UDP is not free, but it is close to it. In the localhost diversion prototype, the JS-layer round trip measured 1.20 ms over QUIC versus 0.84 ms for raw WebRTC SRTP/UDP — a +0.36 ms tax for the tap and the QUIC hop. That is noise next to a single Opus frame, and it buys you a single :443 plane, no SFU, and encoded-frame E2EE readiness.

​

Runtime: where the turn is actually spent

The runtime hop is the whole ballgame. Two shapes:

​

The hidden hop: endpointing

Before any of the five hops runs, the turn detector has to decide the user stopped. That decision has a cost, and it is real latency the caller feels:

• silence_threshold_ms — how long a pause must last before it counts as end-of-turn. Lower it and you respond faster but cut people off mid-thought; raise it and the agent feels sluggish. This is the single biggest knob you own.
• Hold-and-confirm barge-in — an onset arms a pending barge that only fires if speech sustains past a short window (~300 ms), so a backchannel (“mm-hm”) doesn’t cancel the agent. It trades a little barge-in latency for not talking over the caller.

Tune these in Turn detection; just know that a “slow agent” complaint is often an endpointing setting, not a transport or model problem.

​

Measure your own turn

Do not trust our numbers — instrument yours. Turn latency is a client-observable quantity: you have both timestamps, so you can compute it without any server-side access.

​

When the number is too high

The full runbook is in High latency.

​

Related