SIP gateway (B2BUA)
Terminates carrier trunks and agent phones; owns call setup, auth, and teardown.
RTP media plane
Moves the audio: decode/encode, jitter handling, DTMF, recording, and quality.
Dialplan & routing
Decides where each call goes — IVR, skill queues, and agent selection.
Agent runtime
Runs the AI agent: ASR/LLM/TTS or speech-to-speech, tools, and turn-taking.
MoQT transport
Carries browser and app audio as media-over-QUIC on a single :443 plane.
Relay mesh
Fans a published track out to subscribers across cores and across hosts.
Inference
Optional self-hosted GPU control plane for on-prem speech-to-speech models.
SIP gateway (B2BUA)
The SIP gateway is the front door for telephony. It terminates carrier trunks and registrations from agent phones, and it runs as a back-to-back user agent (B2BUA) — every call is two legs (caller and agent) that the gateway bridges, so it can transcode, transfer, and record without either side seeing the other’s signalling. It owns the full SIP lifecycle:INVITE / 200 / ACK / BYE, REFER-based
transfers, digest authentication, source-IP ACLs, and a spam filter. An internal
registrar is on by default, so software and desk phones can REGISTER
directly against your workspace’s SIP domain with no separate registrar to run.
Trunks are matched by source IP and resolve to a tenant; phone registrations land
in a short-lived location store keyed by address-of-record.
The gateway is a native SIP stack, not a fork of Asterisk or FreeSWITCH. If you
are migrating from one of those, you re-point trunks and agents rather than
bridge two systems — see
Migrate from Asterisk / FreeSWITCH.
RTP media plane
While the gateway handles signalling, the media plane handles the audio itself. It runs RTP/RTCP send and receive in the engine, on the same core the call is pinned to, so there is no hop between where the packet lands and where it is processed. Its jobs:- Codec bridging — decode carrier G.711 (PCMU/PCMA) to the runtime’s internal 8 kHz PCM16, and encode agent audio back out. Opus and G.711 legs meet here.
- Jitter handling and pacing — a per-session jitter buffer smooths arrival, and per-session pacers clock outbound frames on a 20 ms cadence.
- DTMF — RFC 2833 telephone-event digits are detected and injected.
- Recording tap — a recording sink can capture both legs (caller and agent as separate stereo channels) without disturbing the live path.
- Quality signals — RTCP feedback yields jitter and packet-loss, and per-call byte counts are attributed per tenant for the traffic dashboard.
Jitter and packet-loss are available today; a full MOS score surfaced per
call is still rolling out. Track what is live on
MOS, jitter & loss.
Dialplan & routing
The dialplan decides what happens to a call once it is answered: play an IVR prompt, send it straight to an AI agent, or queue it for a human. It was the first routing role built into the engine and it is where you express “this number, from this trunk, at this time, goes here.” For contact-center flows, a skill-queue routing brain sits on top: an inbound call is classified (IVR / VDN), matched to a skill queue, and offered to an eligible agent — AI or human — with ring strategies, glare protection, and re-offer on no-answer. The same accept signal works whether the agent is a browser softphone or a registered SIP phone. Set this up in PBX & ACD, and see Call lifecycle for the events each routing decision emits.Agent runtime
The agent runtime is what makes a call conversational. It embeds the AI agent directly in the engine, next to the media plane, so audio does not leave the box to reach the model layer. Each turn runs through a small pipeline (a DAG) that is either a staged ASR → LLM → TTS chain or a single speech-to-speech provider. You bring your own providers — cloud realtime endpoints (OpenAI Realtime, Gemini Live), separate ASR/LLM/TTS vendors (Deepgram, ElevenLabs, and others), or your own realtime endpoint — with credentials scoped per tenant. The runtime also owns turn detection and barge-in, tool calling, and session lifecycle (idle watchdogs, max-session ceilings, output caps).Runtime overview
Providers, pipelines, and how the runtime attaches to a call.
Turn detection & barge-in
When the agent listens, speaks, and yields the floor.
MoQT transport
Telephony reaches you over SIP/RTP; browsers and apps reach you over QUIC. The transport component carries their audio as media-over-QUIC (MoQT) over WebTransport on a single:443 plane — no separate media port to open.
A browser publishes its microphone as a MoQT track and subscribes to the agent’s
reply as another. The engine publishes the caller’s audio the same way, so a human
agent’s browser can subscribe to a live PSTN caller. Crucially, the SDK uses the
browser’s WebRTC stack only as an encoder tap — it extracts encoded Opus
frames and ships them over QUIC — so ICE/DTLS/SRTP are never on the wire for these
clients.
QUIC/MoQT is first-class; WebSocket (control and data) and WebRTC (media)
are supported fallbacks for browsers and networks where QUIC or WebTransport is
unavailable. See
Browser compatibility.
Relay mesh
A published track usually has more than one subscriber — the agent, a supervisor listening in, a recorder — and those subscribers are not always on the same core or even the same host. The relay mesh does that fan-out: it takes one published copy of a track and delivers it to every subscriber, across cores within a host and across hosts within a region. You do not call the relay directly; publishing and subscribing to tracks routes through it automatically. It is what lets a call scale past one core and one machine without the publisher paying a per-subscriber cost. The architecture page shows where it sits in the data path, and Sessions, calls, tracks & streams covers the track model it fans out.Inference
For most deployments the model layer is a cloud provider you configure in the runtime, and there is no separate inference component to think about. The inference component matters when you run models on your own GPUs — typically on-prem, for data-residency or cost reasons. In that mode a control plane discovers model pods, scrapes their load, and routes requests to them over QUIC, presenting a single speech-to-speech endpoint the agent runtime dials like any other provider.The self-hosted inference control plane is shipped; end-to-end
speech-to-speech on it is still being hardened, so validate latency against your
workload before you cut production traffic over. Details and current status live
in Self-hosted inference.
Next steps
Architecture
See these components wired together into the end-to-end data path.
Sessions, calls, tracks & streams
The object model the components operate on.
Deployment models
Managed cloud, on-prem, and hybrid — where each component runs.
Handoffs
How AI and human agents trade a live call without dropping audio.