What has a ready adapter. Vapi’s PCM16 WebSocket shape is wired in the
vendor bridge. LiveKit rides the same bridge as an Opus passthrough. Twilio
Media Streams is a different path — mulaw over WebSocket maps to a
codec-passthrough SIP trunk, not this proxy. See
/modalities/voice/migration/from-twilio.
Which shape of WebSocket bridge do you have?
“Custom WebSocket audio” covers two very different edges. Pick the one that matches yours before you migrate — they map onto different TeleQuick on-ramps.Backend-edge WebSocket
A vendor or telephony provider streams audio to your AI orchestrator
over a WebSocket — typically PCM16 (Vapi-style) or mulaw (Twilio-style).
Your backend consumes and produces raw audio frames. → Map onto the PCM16
WebSocket proxy below.
Client-edge WebSocket
A browser or app streams captured audio to your backend over a
WebSocket because it can’t open a raw media socket. → Adopt the engine’s
WebSocket media fallback and let the SDK own capture. See
Browser/app clients.
The adapter that exists: a PCM16 WebSocket proxy
The vendor bridge is a WebTransport handler that sits between a customer media leg and an external audio backend reached over a secure WebSocket. For the Vapi shape, the flow is:Customer audio arrives over QUIC/WebTransport
A caller (browser, app, or a bridged phone leg) reaches the engine over
QUIC/WebTransport — or, when UDP is blocked, over the WebSocket fallback.
Audio is framed Opus on that customer leg.
The bridge transcodes to PCM16 and cleans it
On ingress, the bridge decodes Opus, resamples to 16 kHz PCM16, and runs
noise suppression so your backend receives clean, uncompressed audio —
the same reason cleanup lives at the edge and not in your service. It then
forwards raw PCM16 to your WebSocket backend as binary messages.
Your backend replies in PCM16
Your backend (the “sidecar”) speaks its vendor’s PCM16 WebSocket protocol:
it publishes the caller into your agent and streams the agent’s PCM16 audio
back over the same socket.
Map your custom source onto the proxy
Your job is to make your existing WebSocket audio backend look like the PCM16 peer the bridge already knows how to talk to. Three cases, easiest first.If your backend already speaks Vapi's PCM16 framing
Point the vendor bridge at Vapi (
VENDOR = vapi) and supply your
VAPI_API_KEY. The bridge’s PCM16 branch handles the codec transform end to
end — no engine changes. This is the turnkey path.If your backend speaks a generic PCM16 WebSocket
Conform your endpoint to the PCM16 sidecar contract: accept and emit
16 kHz mono PCM16 as binary WebSocket messages, one media leg per
connection. The bridge treats it like the Vapi PCM16 branch.
The shipped bridge branches on two codecs only — PCM16 (Vapi shape)
and Opus passthrough (LiveKit shape). A framing that is neither is
not a turnkey branch yet; conform to the 16 kHz PCM16 contract, or
treat the bridge as a reference and adapt at your sidecar rather than
expecting the engine to speak your custom framing.
Route inbound phone calls straight at your WebSocket backend
For PSTN/SIP trunks, set the trunk’s inbound AI WebSocket URL
(
inbound_ai_websocket_url) to your endpoint. Inbound calls matching that
trunk get their AI leg bridged to your WebSocket after the more specific
routes are exhausted (an explicit agent binding, then a skill/queue target,
then the QUIC bridge URL, then this WebSocket URL). Pair it with
inbound_ai_quic_url if you also expose a QUIC endpoint.Browser/app clients
If your custom WebSocket bridge is really a client transport — a browser or app shipping captured audio to a backend — you usually don’t need a bespoke socket at all. The engine serves a WebSocket media fallback (subprotocolclutch-media) that carries the identical browser ⇄ agent audio leg for clients
on networks where QUIC/UDP is blocked (Safari, strict corporate firewalls). It
uses the ws/<sid> namespace so it never collides with the WebTransport leg’s
wt/<sid>, and the wire is framed Opus both ways.
Let the SDK own capture (device AEC/AGC/noise suppression, Opus encode, the
media-over-QUIC track) rather than re-implementing it in your WebSocket client —
/modalities/voice/transport-web/browser-audio-capture.
Wire and audio contract
Frame format, sample rates, namespaces
Frame format, sample rates, namespaces
- Customer leg (WebTransport or
clutch-mediaWebSocket): an optional handshake, then length-prefixed frames[u32 BE length][Opus payload]in both directions. - Sidecar leg (PCM16 backends, e.g. Vapi): 16 kHz mono PCM16 as binary WebSocket messages. LiveKit-shape backends instead get raw Opus passthrough.
- Engine agent bus: 8 kHz PCM16 end to end; the bridge resamples between 8/16/48 kHz as needed. Phone legs are G.711 (µ-law/A-law) on the wire and transcoded internally.
- Namespaces: WebTransport legs use
wt/<sid>; the WebSocket fallback usesws/<sid>; the audio-only track model is in /modalities/voice/concepts/sessions-calls-tracks-streams.
Related
Keep your existing runtime
When the vendor bridge — not a native pipeline — is the right home for your
audio backend.
Custom runtime integration
Bring your own realtime endpoint through the provider abstraction.
From Twilio Media Streams
Mulaw-over-WebSocket → codec-passthrough SIP trunk, the other WebSocket path.
Migration overview & checklist
The full cut-over sequence for pointing trunks and agents at TeleQuick.