Phone calls (Twilio)

@fabric-harness/channels/twilio-voice connects Twilio Programmable Voice to governed voice. The bridge is runtime-neutral, so the same attachTwilioMediaStream() runs inside a Cloudflare Durable Object (recommended for Workers applications) or behind a Node ws server. Identity always comes from server state, never from Twilio stream parameters, and the model never sees Twilio metadata, DTMF or stream parameters.

The bridge needs @fabric-harness/channels 0.24 and @fabric-harness/sdk 6.5 or newer. The Durable Object host and the Workers provider socket factory (@fabric-harness/cloudflare/voice) need @fabric-harness/cloudflare 2.6. Projects generated by CLI 6.7.4 or newer already pin these ranges.

pnpm add @fabric-harness/sdk@^6.5.0 @fabric-harness/channels@^0.24.0 @fabric-harness/cloudflare@^2.6.0

Call flow

1. Twilio posts the Voice webhook. verifyTwilioVoiceWebhook() checks X-Twilio-Signature, the account and the CallSid. Your application maps the verified call to a tenant, resource and session from its own records.
2. createTwilioStreamToken() signs that binding (HMAC-SHA256, bound to the CallSid, short expiry). twilioConnectStreamTwiml() answers with <Connect><Stream> and carries the token in a <Parameter>; Twilio doesn't allow query strings on stream URLs, so route by path (/voice/twilio/stream/{CallSid}). Twilio requires each parameter's name plus value to be under 500 characters, so tokens are capped at TWILIO_STREAM_TOKEN_MAX_LENGTH (460). Minting throws when a binding would exceed it. That allows about 73 ASCII characters for each of the tenant, resource and session IDs when all three are set. Every token that mints also verifies.
3. Twilio opens the Media Streams WebSocket. Verify the signed upgrade with verifyTwilioMediaStreamUpgrade() and route it to the call's host.
4. The bridge validates the start message (μ-law 8 kHz mono only, expected CallSid and account), verifies the token, then calls your required claimStream, which must atomically record the token's nonce and return true only for the first claim. Only then does it call your resolveCall(), which returns session.voice() with server-owned instructions and tools. Nothing reaches the model until resolveCall() resolves; a rejection closes the stream.

claimStream is what makes a stream token single use. attachTwilioMediaStream() throws when it is missing, and any result other than true (including a throw or a timeout) closes the stream with unauthorized before resolveCall() runs. Implement it as one atomic write, such as an INSERT on a primary key, not as a read followed by a write on a shared store.

import {
  TWILIO_STREAM_TOKEN_PARAMETER, createTwilioStreamToken, twilioConnectStreamTwiml,
  twimlResponse, verifyTwilioVoiceWebhook,
} from '@fabric-harness/channels/twilio-voice';

export async function incoming(request: Request, env: Env): Promise<Response> {
  const call = await verifyTwilioVoiceWebhook(request, {
    authToken: env.TWILIO_AUTH_TOKEN,
    url: 'https://voice.example.com/voice/twilio/incoming',
    accountSid: env.TWILIO_ACCOUNT_SID,
  });
  if (!call) return new Response(null, { status: 403 });
  const binding = await resolveInboundBinding(call); // your tenant/case/session lookup
  const token = await createTwilioStreamToken({ secret: env.STREAM_TOKEN_SECRET, ...binding, callSid: call.callSid, expiresAt: Date.now() + 60_000 });
  return twimlResponse(twilioConnectStreamTwiml({
    streamUrl: `wss://voice.example.com/voice/twilio/stream/${call.callSid}`,
    parameters: { [TWILIO_STREAM_TOKEN_PARAMETER]: token },
    say: 'You are speaking with an automated assistant.',
  }));
}

Twilio signs the stream upgrade, but doesn't document the exact URL canonicalisation. The helper checks the exact wss:// URL and its trailing-slash variant. Treat it as defence in depth. The stream token is the mandatory binding, so a forged upgrade without a valid token for that CallSid fails closed either way.

Host on a Cloudflare Durable Object

Use one Durable Object per call. Its SQLite storage gives session.voice() the atomic appendEntries store it needs, and the Worker stays a single deployment.

import { OpenAIRealtimeVoiceProvider } from '@fabric-harness/sdk';
import { createCloudflareDurableObjectSessionStore } from '@fabric-harness/cloudflare';
import { createWorkersVoiceSocketFactory, handleTwilioMediaStream, routeTwilioMediaStream } from '@fabric-harness/cloudflare/voice';
import { attachTwilioMediaStream, twilioCallSidFromPath, verifyTwilioMediaStreamUpgrade } from '@fabric-harness/channels/twilio-voice';

export default {
  async fetch(request: Request, env: Env) {
    return (await routeTwilioMediaStream(request, {
      namespace: env.CALLS,
      pathPrefix: '/voice/twilio/stream',
      locationHint: 'enam',
      verifyUpgrade: (upgrade, callSid) => verifyTwilioMediaStreamUpgrade(upgrade, {
        authToken: env.TWILIO_AUTH_TOKEN,
        url: `wss://voice.example.com/voice/twilio/stream/${callSid}`,
      }),
    })) ?? new Response('Not found', { status: 404 });
  },
};

export class TwilioCallObject {
  store;
  constructor(private ctx: DurableObjectState, private env: Env) {
    this.store = createCloudflareDurableObjectSessionStore(ctx.storage.sql);
    ctx.storage.sql.exec('CREATE TABLE IF NOT EXISTS stream_claims (nonce TEXT PRIMARY KEY, claimed_at INTEGER NOT NULL)');
  }
  async fetch(request: Request) {
    const expectedCallSid = twilioCallSidFromPath(new URL(request.url).pathname, '/voice/twilio/stream');
    if (!expectedCallSid) return new Response(null, { status: 404 });
    return handleTwilioMediaStream(request, {
      attach: socket => attachTwilioMediaStream(socket, {
        tokenSecret: this.env.STREAM_TOKEN_SECRET,
        expectedCallSid,
        accountSid: this.env.TWILIO_ACCOUNT_SID,
        // Required: one atomic INSERT, so a replayed token cannot open a second stream.
        claimStream: binding => this.ctx.storage.sql.exec(
          'INSERT INTO stream_claims (nonce, claimed_at) VALUES (?, ?) ON CONFLICT (nonce) DO NOTHING RETURNING nonce',
          binding.nonce, Date.now(),
        ).toArray().length === 1,
        resolveCall: async ({ binding, signal, voiceOptions }) => {
          const fabric = await init({ store: this.store, tenantId: binding.tenantId, actor, runtime: 'inline', sandbox: 'virtual' });
          const session = await fabric.session(binding.sessionId!);
          return session.voice({
            provider: new OpenAIRealtimeVoiceProvider({ apiKey: this.env.OPENAI_API_KEY, socketFactory: createWorkersVoiceSocketFactory() }),
            callId: binding.callSid, resourceId: binding.resourceId,
            ...voiceOptions, // μ-law 8 kHz and the speech_stopped latency hint
            signal,
            // server-owned instructions, tools, limits, cost limits, authorize, redaction…
          });
        },
      }),
    });
  }
}

routeTwilioMediaStream() returns 404 for a malformed CallSid, 426 for a non-upgrade and 403 when verification fails, so nothing reaches a Durable Object without a verified upgrade. handleTwilioMediaStream() accepts the socket with WebSocketPair. Enable nodejs_compat and declare the class under new_sqlite_classes. The runnable examples/voice-twilio-cloudflare workspace has the complete Worker, wrangler configuration and a credential-free smoke.

Trade-offs to plan for:

• Deploys and evictions drop live calls, as a Node restart would. Deploy between call peaks or drain traffic first; the governed journal keeps the audit and prevents duplicate tool effects.
• No hibernation during a call. The outbound provider socket cannot hibernate, so the object stays resident, and billed, for the call's duration. The helpers call accept() and leave the hibernation API unused.
• Placement. Pass locationHint near your Twilio media region (for example enam for US1). The first request to a new object picks its location.
• Per-call storage. Each object's store holds only that call's journal and per-session ledger. Keep cross-call state (case records, tenant-wide perScope voice budgets) in a shared store, or route calls that must share one durable session to a shared object by a server-issued path key.

Host on Node

The bridge accepts a ws socket directly. Mount it on your server's upgrade event:

import { WebSocketServer } from 'ws';
import { attachTwilioMediaStream, twilioCallSidFromPath, verifyTwilioMediaStreamUpgrade } from '@fabric-harness/channels/twilio-voice';

const wss = new WebSocketServer({ noServer: true, maxPayload: 65_536 });
server.on('upgrade', async (req, socket, head) => {
  const url = `wss://voice.example.com${req.url}`;
  const callSid = twilioCallSidFromPath(new URL(url).pathname, '/voice/twilio/stream');
  const headers = new Headers(Object.entries(req.headers).filter((entry): entry is [string, string] => typeof entry[1] === 'string'));
  if (!callSid || !(await verifyTwilioMediaStreamUpgrade(new Request(url.replace('wss:', 'https:'), { headers }), { authToken, url }))) {
    socket.end('HTTP/1.1 403 Forbidden\r\n\r\n');
    return;
  }
  wss.handleUpgrade(req, socket, head, ws => attachTwilioMediaStream(ws, {
    tokenSecret, expectedCallSid: callSid, resolveCall,
    // Required: for example INSERT ... ON CONFLICT DO NOTHING RETURNING in Postgres.
    claimStream: binding => claimStreamNonceOnce(binding.nonce),
  }));
});

Built-in providers use the Node ws peer by default on Node, so socketFactory is optional there. Provider sockets must use wss://; see voice providers. Use a durable atomic store such as Postgres (the in-memory store loses claims on restart), and run behind TLS-terminating infrastructure that preserves the X-Twilio-Signature header.

Codec, barge-in and latency

The bridge runs μ-law 8 kHz mono end to end and rejects any other mediaFormat. resolveCall() receives voiceOptions (audioFormat: 'g711_ulaw', sampleRate: 8000, emitSpeechStopped: true); spread them into session.voice(). For pipeline mode, configure STT and TTS for μ-law 8 kHz too; PipelineVoiceProvider passes the codec to both.

• Outbound audio goes to Twilio as media messages carrying the streamSid, in 160-byte (20 ms) frames, followed by a mark per audio chunk to track playback. Every mark gets a unique name, so Twilio's echo of one mark acknowledges only the audio sent before it, even for chunks shorter than a millisecond. At most maxPendingMarks (256) marks are outstanding. Audio sent while that budget is used up still counts as unplayed, and the next echo frees a slot for one mark that covers the whole tail.
• Barge-in. When speech_started arrives while assistant audio is still playing (any byte sent to Twilio and not yet acknowledged by a mark), the bridge sends Twilio clear first, which stops playback without waiting on the model, and then calls cancelResponse({ playedMs, interruption }). Audio still arriving for the interrupted response is dropped. OpenAI Realtime truncates the assistant item to playedMs (conversation.item.truncate) so the model's context matches what the caller heard. Cancel races that server VAD already won are ignored. The provider records which reply was playing when it received speech_started, and the bridge hands that event to governed voice as interruption, so an interruption processed late (queued behind transcript redaction, or settling after asynchronous authorization) cancels and truncates only the interrupted reply, never a newer one; if the interrupted reply already finished, the cancel is a no-op.
• Metrics. onMetric reports call_first_audio_ms and response_first_audio_ms (provider end-of-speech to first reply audio; it needs the opt-in speech_stopped event that voiceOptions enables). It also reports barge_in_clear_ms and barge_in_cancel_ms. Each carries only the metric name, value and CallSid. Assert your latency objectives on these in staging.

Limits and failure behavior

Outbound audio is bounded twice. Where the socket reports bufferedAmount (Node ws), the bridge closes above maxOutboundBufferedBytes. Cloudflare Workers sockets don't report it, so the bridge also counts assistant audio sent to Twilio but not yet acknowledged by an echoed playback mark; Twilio echoes a mark once playback reaches it. When a stalled consumer lets more than maxUnplayedAudioMs pile up, the call closes on either runtime. A barge-in clear makes Twilio discard its unplayed buffer and echo the pending marks, but the bridge does not take the clear itself as proof: it sends a fence mark right after the clear and releases the cleared audio from the count only when Twilio echoes that fence (Twilio echoes a mark at once when nothing is buffered). Until then the cleared audio still counts, so repeated audio and barge-in cycles against a stalled consumer still close with backpressure, and a fence that isn't echoed within clearAckTimeoutMs closes the call too. Inbound audio is bounded by maxInboundQueueFrames, which works because built-in providers' sendAudio() waits for provider-socket capacity.

Authentication and protocol failures (missing, expired, tampered, replayed or other-call token, unexpected CallSid or account, wrong codec, media before start) close with 1008 before resolveCall() runs. Close reasons and the onClose summary are fixed codes; exception messages and tokens are never echoed to Twilio, logs or callbacks. close() is idempotent and the bridge closes the governed session on every path, including stop, abort and socket errors.

After a barge-in the bridge drops assistant audio until the provider reports response_done for the interrupted reply. That relies on the provider finishing the cancelled response before it streams the next one, which OpenAI Realtime and the built-in pipeline do. PipelineVoiceProvider holds a new reply's events until the interrupted reply's LLM cancellation and TTS teardown finish and its response_done is emitted; if that cleanup takes longer than 2 s it emits the interrupted reply's response_done (with the usage known so far) and then continues. A custom provider that interleaves a new reply's audio ahead of the cancelled reply's response_done would lose the start of that reply, so certify that ordering before using one with the bridge.

What your application still owns

• Caller verification (OTP) and caller-to-case authorization after verification.
• Consent and recording disclosure (use say or your own TwiML before <Connect>), and recording.
• PII redaction rules in redactTranscript, secure sensitive-data handoff and retention.
• Transfer to a human (end the stream and continue TwiML, or use the Twilio REST API from a governed tool).
• DTMF handling (onDtmf(digit, context, signal)): keypad input is delivered to the application only, one digit at a time in arrival order. At most maxPendingDtmf digits wait behind a running callback, each callback is bounded by dtmfTimeoutMs, signal aborts when the bridge closes, and digits still queued at close are dropped. The bridge detaches its per-digit listener from signal when each callback settles, so long calls do not accumulate listeners.
• Transcript handling (onTranscript(event, context, signal)): final transcripts, already redacted by governed voice, go through their own bounded, ordered queue, outside the loop that handles voice events. A slow or stalled callback therefore never delays a following barge-in (clear and cancellation) or the next reply's audio. One callback runs at a time in arrival order, at most maxPendingTranscripts wait behind it, each is bounded by transcriptTimeoutMs, signal aborts when the bridge closes, and transcripts still queued at close are dropped. Overflow closes the call with backpressure rather than dropping turns silently, the same fail-closed rule as DTMF; persist transcripts asynchronously (for example to a queue) if your store can be slow.
• Outbound calling. The bridge only serves calls Twilio delivers to your webhook; gate any outbound dialing separately.