The four codecs
Every SDK surfaces the sameAudioCodec type. Set it when you attach an audio
bridge or bind an agent:
| Codec | Wire format | Rate | When to use it |
|---|---|---|---|
opus | Opus (48 kHz mono) | 48 kHz | Default. Browser/app legs; best quality-per-bit. |
pcm16 | Raw 16-bit PCM LE | 8 kHz¹ | Feeding a realtime AI model that wants uncompressed input. |
g711_ulaw | G.711 PCMU (PT 0) | 8 kHz | PSTN-direct, no transcode. µ-law — North America and Japan. |
g711_alaw | G.711 PCMA (PT 8) | 8 kHz | PSTN-direct, no transcode. A-law — most of the rest of the world. |
pcm16 frames are 8 kHz mono on the call bus by default; set sampleRate
to resample. See Audio shape.
- TypeScript
- Python
- Rust
opus.
Choosing a codec
Work from the consumer of the audio, not the carrier — the carrier leg is the bridge’s problem, not yours:Talking to a browser or app? Use opus.
Browser softphones capture and encode Opus natively (the encoder is borrowed
straight from the browser’s audio pipeline). Keeping the leg on
opus
end-to-end means no re-encode and the best quality for the bitrate. This is
the default for a reason.Feeding a realtime AI model? Use pcm16.
Most speech-to-speech and ASR models want uncompressed PCM on their input.
Ask for
pcm16 and the bridge hands your code decoded 16-bit samples with no
codec in the middle — the same raw format the agent runtime uses internally.Bridging straight to a PSTN trunk? Match the trunk's G.711.
If you’re relaying a call leg to a SIP/PSTN trunk and don’t need to inspect or
transform the audio, ask for the trunk’s own G.711 variant —
g711_ulaw for
North America and Japan, g711_alaw for the rest of the world. The frames
pass through untouched (see passthrough).µ-law vs A-law is a regional wire convention, not a quality choice. North
American and Japanese carriers use µ-law (PCMU, payload type 0); nearly everyone
else uses A-law (PCMA, payload type 8). Both are 8 kHz 8-bit companded G.711 with
effectively identical fidelity. Pick the one your carrier negotiates — if you get
it wrong you’ll hear distortion, and the bridge will transcode between the two
anyway when two G.711 legs of different flavours meet.
Transcoding at the bridge
Carrier legs almost always arrive as G.711 at 8 kHz (PCMU or PCMA); browser legs arrive as Opus at 48 kHz. The bridge sits between the wire and your code and converts in both directions, decoding to the runtime’s internal PCM representation and re-encoding to whatever each side needs:- The runtime bus is 8 kHz PCM16 end-to-end. A 20 ms tick is 160 samples at 8 kHz. That’s the fidelity ceiling for anything routed through the agent path — telephone-band audio, which is exactly what PSTN delivers and what speech models are trained on.
- Opus keeps a 48 kHz RTP clock even though the audio is narrowband. The Opus
rtpmap stays
opus/48000/2and each 20 ms frame advances the timestamp by 960, per the codec’s spec, while the encoder’s internal rate is driven to narrowband to match the bus. You don’t manage any of this — it’s the fix that makes browser Opus play back clean — but it’s why you’ll see48000in an SDP even for an 8 kHz call.
Passthrough fast paths (zero transcode)
Transcoding is cheap, but the fastest audio is audio you never touch. Two paths skip the codec entirely:G.711 in, same G.711 trunk out.
When you bridge a PSTN leg to another PSTN/SIP trunk of the same G.711
flavour and ask for that flavour, the bridge relays the frames verbatim — no
decode, no re-encode. This is the norm for straight call-forwarding and
carrier hand-off.
External µ-law WebSocket audio to a µ-law trunk.
Migrating from a Twilio Media Streams–style source? Its 8 kHz µ-law WebSocket
frames map straight onto a PCMU SIP trunk with zero transcode — the
payloads are already in the trunk’s format. See
Migrate from Twilio for the wiring.
Audio shape: sampleRate, channels, frameMs
Codec choice sets the format; three more knobs set the shape. All have sensible defaults, so you usually set onlycodec:
The wire format for this leg, per the table above.
Samples per second.
opus runs at 48000; G.711 is fixed at 8000. For pcm16,
set this to the rate your model wants (e.g. 16000 or 24000) and the bridge
resamples for you — the stateful resampler avoids the “chipmunk” artefacts of a
naive double-resample.Voice is mono. Recordings mix the two legs into stereo (caller left, agent
right) downstream — that’s a recording concern, not a codec one.
Milliseconds of audio per frame/object. 20 ms is the RTP and Opus standard
(160 samples at 8 kHz, 960 at 48 kHz) and what every pacer and jitter buffer on
the path assumes. Change it only if a downstream consumer demands it.
Related
Sessions, calls, tracks & streams
Where the codec’s capability tag (
voice/opus, voice/pcm16) fits the track model.SIP trunking
How carrier legs negotiate G.711 on the wire before the bridge sees them.
Browser audio capture
The Opus path: borrowing the browser’s encoder onto QUIC/MoQT.
Migrate from Twilio
The µ-law WebSocket → PCMU trunk passthrough on-ramp.