Voice

Use session.voice() for application voice. It wraps OpenAI Realtime or an STT/LLM/TTS pipeline with server-owned instructions, registered tools, session capability policy, authorization checks, redaction hooks, bounded execution and durable usage/tool journals. provider.connect() remains a low-level transport without session governance. Your application supplies telephony and caller verification.

Governed server session

Create the agent/session with the authenticated tenant and actor and a shared atomic SessionStore. Use durable storage in production; the in-memory store is for tests. Resolve call and resource IDs from trusted server state so a browser can't choose another caller's case or forge a call binding.

const voice = await session.voice({
  provider,
  callId: verifiedCall.id,
  resourceId: verifiedCall.caseId,
  instructions: 'Assist with the verified case. Never request SSNs or bank credentials in speech.',
  tools: [lookupCase, requestSecureUpload],
  limits: { durationMs: 300_000, maxTurns: 30, maxToolCalls: 12 },
  costLimit: { perSession: 2, perCall: 0.25, perScope: 50, scopeKey: 'voice:2026-09-23' },
  authorize: operation => authorizeCall(verifiedCall, operation),
  prepareToolInput: (name, input) => validateAndRedactArguments(name, input),
  toolOperationKey: (name, input) => resolveLogicalActionId(verifiedCall, name, input),
  prepareToolOutput: (name, output) => modelSafeReceipt(name, output),
  redactTranscript: text => redactSensitiveText(text),
  onAudit: event => auditSink.write(event),
  signal: requestSignal,
});
try {
  for await (const event of voice.events()) {
    if (event.type === 'audio_delta') await transport.play(event.audio);
    if (event.type === 'speech_started') await transport.clearPlayback();
    if (event.type === 'transcript') await transport.caption(event.text);
    if (event.type === 'error') break;
  }
} finally {
  await voice.close();
}

You'll supply these callbacks, including identity checks and PII detection. Input preparation must validate the selected tool's schema and remove sensitive values before policy, approval or history sees them. Output preparation runs after tool envelopes are unwrapped and before history/model exposure. Use opaque references and safe receipts. Mutating tools must call governed Platform actions and use context.toolCallId as the idempotency key. Don't use dynamic state writes or durable substeps from voice tools. toolOperationKey must resolve a trusted logical action identity that stays stable across provider reconnects. Harness hashes it with the call binding and tool name. Provider-generated call IDs change on reconnect and must not be used for this callback. A genuinely new action needs a distinct key; reusing a key with changed arguments fails closed. Keys must be nonempty and at most 512 characters.

The governor doesn't emit tool calls to clients and rejects client tool results; when the provider requests an action, it rechecks authority before execution, fences replaced connections, and retains only the application's safe completed result so a reconnect can replay that receipt without invoking the external action a second time.

Audio-frame authority and ownership checks are coalesced and cached for at most one second to avoid full durable-store reads for every 20 ms frame. Tools and text inputs always check immediately. Partial provider transcript chunks are discarded without durable-store reads. For instant disconnect on revocation, abort the trusted call signal. The bridge bounds queued input to 128 messages or 1 MiB and closes on overload. A started tool without a committed result requires application reconciliation; it isn't automatically retried, and creating a new call ID would bypass the original receipt instead of resolving it.

Only complete, application-redacted transcripts are forwarded; partial text is suppressed because PII can cross chunk boundaries. Harness doesn't persist raw audio. Providers still receive audio under their own data-handling terms. Secure sensitive-data handoff must bypass speech.

Providers and runtimes

Install ws in the server host: pnpm add ws. Provider WebSockets load this optional peer for authorization headers. Loading the core SDK doesn't connect or require credentials. Never put provider keys in a browser. Supplied WebSocket providers target Node 22+; Cloudflare Workers need a compatible provider adapter or an authenticated Node voice sidecar.

import { OpenAIRealtimeVoiceProvider } from '@fabric-harness/sdk';
const provider = new OpenAIRealtimeVoiceProvider({
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-realtime',
});

The adapter uses the GA audio/session schema and response.output_audio.* events without the retired beta header. PCM16 is 24 kHz mono; G.711 μ-law/A-law uses 8 kHz. Your transport must match the codec. See OpenAI WebSockets.

For pipeline mode, compose PipelineVoiceProvider with Deepgram or Cartesia STT, a text model, and ElevenLabs or Cartesia TTS; pass it to the same session.voice() interface. See Voice Providers. ElevenLabs rejects unsupported G.711 A-law.

For budgeted pipeline barge-in, configure PipelineVoiceProvider({ interruptedLlmCostUsd, ... }) with a conservative per-request estimate for your model/context limits. When cancellation prevents the LLM from reporting usage, this explicit estimate is charged so the conversation can continue; without it, unknown usage stops the call for reconciliation. TTS estimates count submitted characters, including interrupted synthesis, rather than implying that unplayed audio was free. Reconcile these estimates with provider billing. OpenAI Realtime rejects sample rates that don't match its codec.

Cost and recovery

Every session requires a perSession ceiling, while perCall limits each individual model response. Optional perScope and scopeKey aggregate voice spend across sessions within a tenant using atomic idempotent ledgers in the session store. These are separate from text-loop cost stores. Reconnect repairs journal-to-ledger updates and retains the original call deadline and turn/tool counts.

These estimated post-response limits can overshoot while a response is in flight, especially when several calls share a budget. Use provider billing caps and application admission/concurrency limits for financial hard stops. Telephony, recording, storage and verification costs aren't included.

Realtime text/audio counts are disjoint, including modality-specific cached input. Pipeline usage charges each STT delta once, the LLM response and that response's TTS characters. Register negotiated rates with registerModelPrices. Unknown costs stop a governed session. Interrupted input without a billed response blocks same-call reconnect until application reconciliation. Disconnects cannot guarantee final provider billing, so you'll need reconciliation against provider usage/invoices.

Audit metadata and safe tool receipts are durable. onAudit delivery is awaited and failures stop processing; external audit delivery isn't exactly-once. Apply tenant access and retention controls to voice: and voice-cost: records. Store loss removes replay/budget protection. Connections don't survive process restart. Open voice in a server or activity, never deterministic workflow code.

Node endpoint and migration

WS /sessions/:id/voice is disabled unless startDevServer({ connectVoice }) is configured. The callback receives authenticated principal, tenantId, sessionId and signal; resolve the authorized application binding and return session.voice(...). The route requires voice:invoke and an existing tenant-owned session. session:read alone is insufficient.

const handle = connectFabricVoice({
  url: `wss://app.example.com/sessions/${sessionId}/voice`,
  authToken: shortLivedVoiceToken,
  tenantId,
  onEvent(event) {
    if (event.type === 'audio') speaker.write(event.audio);
    if (event.type === 'speech_started') speaker.clear();
  },
});
mic.on('frame', frame => handle.sendAudio(frame));

Move model, voice, codec and instructions into the trusted server callback. Browser configuration fields are deprecated and ignored; query parameters cannot configure the provider. Remove client submitToolResult() calls. Install ws, grant voice:invoke and configure storage, limits, identity, validators and redactors. Use short-lived query tokens with log redaction, or authenticated same-origin cookies. fh add can scaffold a telephony bridge; it does not certify its security.

Custom implementations of the public FabricSession interface must add voice(options), returning a governed session or rejecting with an unsupported-runtime error. Existing text-agent authoring and the finite/persistent builders are unchanged.

AutoRefi readiness

AutoRefi can build against the governed interface. Before a phone pilot it still needs:

• Signed telephony ingress, matching codecs and playback interruption.
• OTP identity, caller-to-case authorization, consent and recording disclosure.
• Transcript redaction, secure sensitive-data handoff and retention rules.
• Accessible captions/alternative channels, human transfer/takeover and outage recovery.
• Idempotent Platform tools and reconciliation of uncertain effects/costs.
• A credentialed staging canary covering barge-in, disconnect, denied access and exhausted budgets.

Start with the credential-free governed smoke example in examples/voice-data-collector. You'll still need a real-phone staging canary before certifying a production rollout.