A session is one run of the conversational loop: the runtime picks up a connected call, greets the caller, and then cycles through listen → think → speak — interruptible at any point — until someone hangs up or a guardrail fires. You describe the agent; the runtime drives the states. This page is the map of those states, the clocks that bound them, and the guardrails that make sure a session never runs forever or sits silent on your bill. Everything here runs inside the voice gateway, in the process that terminates the call’s media, pinned to the core that owns the call. Each session gets its own state, local to that core, so nothing is shared across calls and turn-taking never waits on a lock. Audio flows through the runtime as a single 8 kHz PCM16 stream regardless of whether the caller arrived over a phone trunk or a browser QUIC connection.

​
The state machine

        ┌──────────────┐
        │  Connecting  │  media negotiated, session allocated
        └──────┬───────┘
               ▼
        ┌──────────────┐
        │   Greeting   │  agent speaks first (optional opening line)
        └──────┬───────┘
               ▼
   ┌─▶ ┌──────────────┐
   │   │  Listening   │  VAD watches for end-of-utterance
   │   └──────┬───────┘
   │          ▼
   │   ┌──────────────┐
   │   │   Thinking   │  model runs (cascade LLM, or duplex model)
   │   └──────┬───────┘        ▲
   │          ▼                │  caller speaks over the agent
   │   ┌──────────────┐        │  → confirmed barge-in cancels
   │   │   Speaking   │────────┘     and jumps back to Listening
   │   └──────┬───────┘
   │          │ turn complete
   └──────────┘
               │  hangup, transfer, or guardrail
               ▼
        ┌──────────────┐
        │   Ending     │  cancel in-flight work, flush, emit events
        └──────────────┘
Listening, Thinking, and Speaking form the turn loop. A duplex speech-to-speech model collapses Thinking and Speaking into one streaming step, but the states around it — greeting, barge-in, guardrails, teardown — are identical.

​
Walking a session

1

Connecting

The call arrives — a phone caller on a SIP trunk, or a browser mic over QUIC — and its media is negotiated and decoded to the 8 kHz PCM16 contract. The runtime allocates a session on the call’s core, loads the agent definition, and opens the provider connection (a streaming ASR/LLM/TTS cascade, or a single duplex realtime session). If the provider connection can’t be opened within its timeout, the session ends here and the call is released.
2

Greeting

If the agent has an opening line, it speaks first — the runtime enters Speaking before it has heard anything. Leave the greeting empty and the agent starts in Listening, waiting for the caller to talk first (the usual shape for inbound support lines where the caller already knows why they called).
3

Listening

The runtime watches the inbound audio for end-of-utterance. On a cascade pipeline an on-device VAD marks the boundary; on a cloud duplex model the provider’s own end-of-turn detection can do it. Backchannels (“mhm”, “yeah”) are filtered out by a minimum-duration gate, so a caller’s acknowledgement never counts as a turn. When the caller finishes, the turn’s audio is handed to the model and the session moves to Thinking. See turn detection & barge-in for how the boundary is detected and tuned.
4

Thinking

The model runs. In a cascade this is the language-model call (fed by the transcript); in a duplex model the audio streams straight in. Each stage has its own timeout budget — if a model stalls past its budget, the runtime cancels the stage rather than leaving the caller in dead air (see Timeouts and budgets).
5

Speaking

The reply is synthesised (cascade TTS) or streamed out (duplex) as 8 kHz PCM16, paced into 20 ms frames, and encoded for the caller’s transport — G.711 back to a phone trunk, Opus back to a browser. The runtime stays ready to interrupt: the caller can barge in at any moment during Speaking.
6

Loop or end

When the turn completes the session returns to Listening for the next turn. It leaves the loop only when the caller hangs up, the call is transferred to a human, or a guardrail fires — at which point it enters Ending.

​
Barge-in cancels a turn in flight

A caller talking over the agent is not a new turn waiting its polite turn — it’s an interrupt. When a barge-in is confirmed, the runtime cancels whatever the agent is doing (Thinking or Speaking), clears the outbound audio buffer so the agent goes quiet immediately, and jumps back to Listening to catch what the caller is actually saying. “Confirmed” is the important word. The runtime uses hold-and-confirm: the onset of caller speech arms a pending barge, but it only fires if speech sustains past barge_confirm_ms (default 300 ms). A short backchannel hits its end before the window elapses and never cancels the agent. Set barge_confirm_ms: 0 for legacy immediate barge-in — the agent cuts out on the very first audio frame.
Whether a barge-in can actually cancel work already sent to a provider depends on the provider. Cloud realtime and streaming TTS support mid-flight cancel; a plain HTTP language model runs to completion server-side and the runtime can only silence the playback. The honest per-provider scorecard lives in turn detection & barge-in.

​
Timeouts and budgets

Every session runs against a set of clocks. They exist so that one stuck model, one runaway response, or one caller who wandered off never turns into an open connection billing forever.
BudgetWhat it boundsDefault
Per-stage timeoutHow long any single ASR / LLM / TTS or realtime step may run before it’s cancelledTunable (millisecond budgets)
max_output_tokensCeiling on a single model response, so one turn can’t generate without bound4096
max_sessionHard ceiling on total session wall-clock; the session is torn down when it’s hit15 minutes
max_output_tokens was previously unbounded on the realtime path and is now capped by default — a single turn cannot run away. max_session is a hard stop, not a warning: when the clock hits it the runtime tears the session down regardless of state.
A hard max_session ceiling means a genuinely long call (a detailed onboarding, a patient walkthrough) is ended at the ceiling. Raise it for agents that legitimately need longer sessions, but keep it finite — the whole point is that no session runs forever.

​
Changing budgets doesn’t disturb live calls

The per-stage timeout budgets are runtime tunables you can adjust and reload without restarting the gateway or dropping calls. Reloads are additive: a session captures its timeout budgets at the moment it’s created and keeps them for its whole life, so a reload only affects sessions that start after it. No in-flight call ever has a clock changed underneath it.

​
Idle hangup guardrails

The most common way a voice session gets stuck is silence: the caller stops talking, the agent has nothing to respond to, and without a guardrail the session would sit there until max_session finally reaps it. The idle watchdog handles this gracefully — it nudges the caller first, then hangs up. When the caller has been silent for soft_prompt_s, the runtime injects a re-engagement prompt (“Are you still there?”) and keeps listening. If silence then continues to hangup_s, it ends the session cleanly. Configure it under the agent’s turn-detection block: 
turn_detection:
  idle:
    soft_prompt_s: 20                      # nudge after this much silence
    hangup_s: 60                           # end the call after this much
    soft_prompt_text: "Are you still there?"
The idle watchdog is on by default on both pipeline shapes — the cascade path and the cloud duplex (server-VAD) path — with the defaults above (re-engage at 20 s, hang up at 60 s). Set the timings per agent, or lengthen them for flows where a caller is expected to go quiet while they look something up.
Default-on idle guardrails on the duplex realtime path are a recent addition. The behaviour ships and is enabled by default; if you’re running a duplex agent and want to be certain of the exact re-engage and hangup moments for your provider, verify them against a test call before relying on them in production.

​
Ending and teardown

A session leaves the loop for one of three reasons — the caller hangs up, the call is handed off to a human or transferred, or a guardrail (max_session or the idle watchdog) fires. Teardown is the same either way: the runtime cancels any in-flight model work, closes the provider session, flushes and stops the outbound audio pacer, releases the media leg, and frees the session’s per-core state. As it goes, the runtime emits the call’s lifecycle transitions — answer, holds and resumes, transfer or REFER, and the final end — to the events pipeline, each tagged with which handler (AI or human) owned that segment. Those events are what power your call timelines and CDRs; see call lifecycle for the telephony view and call traces for the per-session timeline you’ll actually read after the fact.

Turn detection & barge-in

How end-of-utterance and confirmed barge-in are detected and tuned.

Runtime configuration

Every knob for budgets, guardrails, and turn timing in one reference.

Tool calling

What happens to the loop when the model calls a tool mid-turn.

Call traces

Read a finished session’s timeline, segment by segment.