voice.on("...") firehose. Instead you wire a small set of purpose-built
callbacks: one for each direction of audio, one for transport session state,
and — only for human-agent softphones — a real push channel. This page is the
complete inventory, plus an honest account of where the SDK gives you a stream
and where it makes you poll.
Ground truth: the data plane rides our media-over-QUIC (MoQT) transport and is
fully callback-driven. The call control plane is request/response — there is
no server-push socket for call state. Only the human-agent control channel
(
Agent) is a live EventEmitter. Design around that split.The three surfaces at a glance
Audio frame callbacks
onUplink / onDownlink fire once per encoded 20 ms frame, in order,
with a microsecond timestamp. This is your real-time media loop.Transport session state
ConnectionState transitions (Connecting → Connected → Reconnecting →
Closed / Failed) with transparent auto-reconnect underneath.Human-agent control
Agent — the softphone control channel — is a genuine push
EventEmitter (callOffered, stateChanged, forcedLogout, …).Audio frame callbacks (data plane)
AnAudioBridge gives you one callback per direction. Both deliver encoded
frames (e.g. a 20 ms Opus packet or a PCM16 chunk) exactly as they arrive off
the media plane — the SDK does not decode, resample, or buffer them for you.
Inbound caller audio — what the caller is saying. Registered when you
attach() the bridge to a call_sid. Subscribes the voice/<sid>/uplink
track under the hood.Inbound cloud→caller audio — what is being played back to the caller.
Registered when you
attachCaller() (the browser-caller mirror of attach).
Subscribes the voice/<sid>/downlink track.timestampUs is the media timestamp in microseconds — use it to sequence,
measure inter-frame gaps, or align against your own clock. Frames are delivered
in order, one at a time; there is no batching.
- TypeScript
- Python
codec accepts opus (default), pcm16, g711_ulaw, or g711_alaw — the
same AudioCodec set in every language SDK. Voice is audio-only; there are no
video frame callbacks.
Transport session state & auto-reconnect
The media plane keeps itself alive for you. The underlying MoQT transport client reconnects transparently — on a drop it re-opens the session and re-announces every publication and re-subscribes every subscription, so youronUplink / onDownlink callbacks resume without you re-wiring anything.
You can observe those transitions through the transport client’s onState
callback, which reports a ConnectionState:
Connecting (0) · Connected (1) · Reconnecting (2) · Closed (3) ·
Failed (4). The callback signature is (state, reason?) => void.Connected
The session is established. On a re-connect (not the first), your pubs/subs
are replayed automatically.
Reconnecting
A live session dropped. The client is retrying with capped exponential
backoff — starting at 250 ms, doubling to a 5 s ceiling, reset on success.
Failed
The first connect never succeeded (bad relay host, auth, or blocked
UDP). This is terminal for that client and rejects the connect promise —
it is distinct from
Reconnecting, which only happens after you have
connected at least once.Honest gap: the high-level
AudioBridge (attach / attachCaller) opens the
transport for you but does not currently forward an onState callback — it
reconnects silently. To observe session state today, drive the MoQT transport
client directly (the primitive the bridge is built on) and layer the bridge’s
publish/subscribe calls on top, or infer a stall from your frame callbacks going
quiet. Surfacing onState through AudioBridge is a planned convenience, not a
shipped one.Reconnect leaves a gap, and the SDK tells you
A transparent reconnect is not gap-free: objects published while the session was down are lost (true gap-free replay needs a durable origin, which is not the default). Rather than swallow this, the transport surfaces it. Each subscription exposes two optional lifecycle hooks on the MoQT primitive:The track ended (publisher unpublished, or the sub was torn down).
A delivery gap. After a reconnect you receive one
DropInfo with
reason: "reconnect" (and sentinel fromGroup/fromObject/toGroup/toObject of
-1) so you can, for example, flush a jitter buffer or re-prime a decoder.AudioBridge convenience
wrapper wires only the frame callback, so if you need onEnded / onDropped,
work with the MoQT subscription directly. See
Realtime tracks for
the track/namespace model these hooks belong to.
There is no call-state event stream
This is the most important honesty point on the page. Call control is request/response over the control-plane API (tRPC to the control-plane host). The SDK does not open an event socket for call lifecycle — there is nocall.on("answered"), no onRinging, no server-push CDR event in the Voice SDK.
A Call carries a status snapshot from the moment you fetched it
(dialing | ringing | in_progress | completed | failed | no_answer). To observe
a transition, you re-fetch:
- TypeScript
- Python
Human-agent control channel (the one real event stream)
The exception to everything above is the human-agent softphone control channel. When you build a browser softphone for a live human agent, theAgent class is a genuine EventEmitter over a persistent connection (a
control-plane WebSocket at /agent/control), and it pushes:
Connection accepted; the gateway assigned a device id.
The gateway acknowledged a
setState transition.Agent presence/aux state changed (server-driven, e.g. forced to
acw).A call is being offered to this agent; confirm or reject before
ringUntilMs.The ring window elapsed with no confirm;
forcedRona is true when the missed
counter crossed the RONA threshold and the gateway forced aux:rona.The gateway accepted your confirm (a one-shot, distinct from
stateChanged).A protocol/authorization error on the channel.
The gateway evicted this device (e.g. single-device kick).
The channel closed;
clean distinguishes a graceful close from a drop.This channel runs over the control-plane WebSocket today.
Agent accepts a
preferTransport: 'wt' | 'wss' | 'auto' hint, but WebTransport carriage of
/agent/control is not yet shipped — 'auto' resolves to WSS.Callback inventory
| Surface | Callback | Fires | Push or poll |
|---|---|---|---|
AudioBridge (server) | onUplink(frame, tsUs) | per caller frame | push |
AudioBridge (browser caller) | onDownlink(frame, tsUs) | per cloud→caller frame | push |
| MoQT transport client | onState(state, reason?) | on session transition | push |
| MoQT subscription | onEnded(reason) | track ended | push |
| MoQT subscription | onDropped(info) | delivery/reconnect gap | push |
Calls | (none) | call status via calls.get | poll |
Agent (human softphone) | on("callOffered" | "stateChanged" | …) | control events | push |
Related
TypeScript SDK
Full
Voice / Calls / AudioBridge / Agents reference.Python SDK
The same callback shapes in Python (
on_uplink, publish_downlink).Sessions, calls & tracks
The
voice/<sid>/{uplink,downlink} track model behind these callbacks.Transport API
Connection states, reconnect, and the MoQT client surface in depth.