Voice
Voice Call Realtime
By Give your AI assistant a phone β OpenClaw plugin for real phone calls via Twilio + OpenAI Realtime, with in-call tools, transcripts, and call screening
Install
npm install --omit=devConfiguration Example
tunnel: voice
credentials-file: /Users/you/.cloudflared/<tunnel-id>.json
ingress:
- hostname: voice.yourdomain.com
service: http://localhost:3336
- service: http_status:404README
# OpenClaw Voice Call β Realtime Edition
**Give your AI assistant a phone.**
This OpenClaw plugin lets your assistant place and receive real phone calls and hold natural, full-duplex voice conversations β powered by Twilio Programmable Voice and the OpenAI Realtime API. It books your table, checks the store's hours, navigates the phone menu, gets a human on the line, wraps up politely, hangs up on its own, and reports back with a structured outcome, an AI-written summary, and the full transcript.
```
You: "Call Luigi's and book a table for two on Friday at 7."
Assistant: *dials, talks past the IVR, negotiates with the host, confirms details*
Assistant: "Booked β table for 2, Friday 7pm, under your name.
They said parties over 15 minutes late lose the table."
```
## Why this exists
Every "AI agent" can send an email. Almost none of them can call the dry cleaner. The real world still runs on phone calls β restaurants, doctors' offices, contractors, that one store whose website hasn't been updated since 2019. This plugin closes that gap: your assistant gets a phone number, a voice, ears, a keypad, and the judgment to end the call when the job is done.
## Features
- **Full conversation mode** β Twilio Media Streams bridged to the OpenAI Realtime API (speech-to-speech, GA protocol). Sub-second turnaround, natural barge-in when the other party interrupts.
- **In-call tools** the voice AI uses autonomously:
- `press_phone_keys` β synthesized DTMF touch-tones for navigating IVR menus ("press 2 for reservations")
- `report_call_outcome` β structured result capture (status + every fact gathered: times, prices, confirmation numbers)
- `end_call` β graceful hangup: speaks a closing line, waits for it to *actually play out* on the line (Twilio mark echo), then disconnects. No clipped goodbyes, no lingering dead air.
- **Transcripts & summaries** β every call is finalized into a Markdown transcript with an AI-generated summary and the reported outcome. Retrievable after the call via the `get_transcript` tool action.
- **Call screening awareness** β handles Google Call Screen and voicemail gatekeepers by identifying itself with a configurable identity phrase.
- **Goal-directed calls** β pass `talking_points` and `call_party` (first-party vs third-party) and the AI stays on task: cover the points, collect the answers, confirm out loud, wrap up.
- **Inbound calls** (optional) β allowlist-gated, with a configurable greeting.
- **Device profiles** β per-caller policies: response length, forbidden actions, extra instructions.
- **Hardened by default** β Twilio webhook signature verification, per-call stream auth tokens, pre-auth connection throttling, SSRF-guarded provider API calls, call-duration safety caps, stale-call reaping.
- **Providers** β Twilio (recommended, full realtime conversation mode), plus Telnyx / Plivo / mock for the legacy TTS+STT pipeline.
## Use cases
- **Reservations & appointments** β restaurants, salons, dentists: "book me the earliest slot Thursday afternoon."
- **Information gathering** β opening hours, stock checks, price quotes, "is the kitchen still open?"
- **Errand triage** β call the pharmacy about a refill, the contractor about a quote, the venue about parking.
- **Notifications with a human touch** β a call that speaks a message and confirms it was heard, not just a text that might be missed.
- **Reaching *you*** β your assistant can call your own phone when something needs a real-time decision, and you can just *talk* to it.
- **Inbound assistant line** β allowlisted callers can ring your assistant directly and converse with it.
## How it works
```
Phone network Your machine OpenAI
βββββββββββββββ webhooks ββββββββββββββββββββββββ
β Twilio β ββββββββββββΊ β Plugin webhook β
β Programmableβ β server (HTTP) β
β Voice β ββββββββββββ β Β· TwiML <Connect> β
β β TwiML β <Stream> β
β β β β
β Media β WebSocket β Media stream β WebSocket ββββββββββββ
β Streams β ββββββββββββΊ β handler (ΞΌ-law 8kHz) β βββββββββββΊ β Realtime β
βββββββββββββββ audio ββββββββββββββββββββββββ audio β API β
ββββββββββββ
```
The plugin runs an HTTP server inside the OpenClaw gateway. Twilio fetches TwiML from it and opens a bidirectional audio WebSocket; the plugin bridges that audio to an OpenAI Realtime session which does STT, reasoning, tool calls, and TTS in one loop.
## Requirements
- **OpenClaw** β₯ 2026.6 (the plugin loads as a path plugin and runs inside the gateway)
- **Twilio account** with a voice-capable phone number (~$1.15/mo + ~$0.014/min)
- **OpenAI API key** with Realtime API access (`gpt-realtime-2.1` recommended)
- **A public HTTPS URL** reaching the plugin's webhook port (Cloudflare Tunnel, ngrok, or Tailscale Funnel β walkthrough below)
---
# Installation
## 1. Get the plugin
```bash
mkdir -p ~/.openclaw/plugins-src
git clone https://github.com/TristanBrotherton/openclaw-voice-call-realtime.git \
~/.openclaw/plugins-src/voice-call-realtime
cd ~/.openclaw/plugins-src/voice-call-realtime
npm install --omit=dev
```
## 2. Set up Twilio (detailed)
1. **Create an account** at [twilio.com](https://www.twilio.com/try-twilio). Trial accounts work for testing but can only call verified numbers and play a trial notice β **upgrade (add billing)** for real use.
2. **Buy a voice-capable phone number**: Console β *Phone Numbers β Manage β Buy a number*. Check the **Voice** capability box. Pick a local number; note it in E.164 form (e.g. `+15551234567`).
3. **Get your credentials**: Console home page β *Account Info* panel:
- **Account SID** β starts with `ACβ¦`
- **Auth Token** β click to reveal. Treat it like a password; it signs and verifies every webhook.
4. **Geo permissions** (only if calling outside your country): Console β *Voice β Settings β Geo permissions* β enable the destination countries.
5. **Inbound calls (optional)**: Console β your phone number β *Voice Configuration* β set **"A call comes in"** to *Webhook*, URL = your public webhook URL (below), method POST. For **outbound-only** use you can skip this β the plugin passes TwiML URLs per call via the API.
## 3. Expose the webhook publicly
Twilio must reach the plugin's webhook server (default port `3336`) over HTTPS. **Cloudflare Tunnel** (free, stable hostname) is recommended:
```bash
brew install cloudflared # or apt/pacman equivalent
cloudflared tunnel login # authorize against your Cloudflare-managed domain
cloudflared tunnel create voice
cloudflared tunnel route dns voice voice.yourdomain.com
```
Create `~/.cloudflared/config.yml`:
```yaml
tunnel: voice
credentials-file: /Users/you/.cloudflared/<tunnel-id>.json
ingress:
- hostname: voice.yourdomain.com
service: http://localhost:3336
- service: http_status:404
```
Run it: `cloudflared tunnel run voice`
**Make it survive reboots.** A dead tunnel silently breaks all calls. On macOS, install a LaunchAgent (`~/Library/LaunchAgents/com.you.cloudflared-voice.plist`):
```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key><string>com.you.cloudflared-voice</string>
<key>ProgramArguments</key>
<array>
<string>/opt/homebrew/bin/cloudflared</string>
<string>tunnel</string><string>--config</string>
<string>/Users/you/.cloudflared/config.yml</string>
<string>run</string><string>voice</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
<key>ThrottleInterval</key><integer>15</integer>
<key>StandardOutPath</key><string>/tmp/cloudflared-voice.log</string>
<key>StandardErrorPath</key><string>/tmp/cloudflared-voice.log</string>
</dict>
</plist>
```
```bash
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.you.cloudflared-voice.plist
```
On Linux, use a systemd unit with `Restart=always`. Alternatives: `ngrok http 3336` (set `tunnel.provider: "ngrok"` for auto-management) or `tailscale funnel 3336`.
## 4. Configure OpenClaw
In `~/.openclaw/openclaw.json`:
```jsonc
{
"plugins": {
"allow": ["voice-call-tristan"], // plus your other plugins
"load": { "paths": ["/Users/you/.openclaw/plugins-src"] },
"entries": {
"voice-call-tristan": {
"enabled": true,
"config": {
"enabled": true,
"provider": "twilio",
"fromNumber": "+15551234567", // your Twilio number
"toNumber": "+15557654321", // default destination (you)
"twilio": {
"accountSid": "ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"authToken": "your_twilio_auth_token"
},
"serve": { "port": 3336, "bind": "127.0.0.1", "path": "/voice/webhook" },
"publicUrl": "https://voice.yourdomain.com/voice/webhook",
"outbound": { "defaultMode": "conversation" },
"maxDurationSeconds": 3600,
"streaming": {
"enabled": true,
"sttProvider": "openai-realtime-conversation",
"realtimeModel": "gpt-realtime-2.1",
"realtimeVoice": "alloy",
"openaiApiKey": "sk-...",
"realtimePolicy": { "idleTimeoutMs": 120000, "maxSessionMs": 7200000 }
},
"callScreening": {
"enabled": true,
"callerIdentity": "Hi, this is Alex's AI assistant calling on their behalf."
}
}
}
}
}
}
```
Validate and restart:
```bash
openclaw config validate
openclaw gateway restart
```
You should see in the gateway logs:
```
[voice-call] Media streaming initialized
[voice-call] Webhook server listening on http://127.0.0.1:3336/voice/webhook
[voice
... (truncated)
voice
Comments
Sign in to leave a comment