Command Events (you send)
| Event | Payload | Description |
|---|---|---|
agent.interrupt | {"type": "agent.interrupt"} | Interrupts and clears all scheduled speaking events. |
agent.speak | {"type": "agent.speak", "audio": "<base64>"} | Adds audio to the avatar’s playback buffer. Audio must be PCM 16-bit 24 kHz encoded as Base64. Recommended chunk size: ~1 second. Max 1MB per packet. |
agent.speak_end | {"type": "agent.speak_end", "event_id": "<id>"} | Signals the end of a speaking event. |
agent.start_listening | {"type": "agent.start_listening", "event_id": "<id>"} | Transitions the avatar to a listening state. |
agent.stop_listening | {"type": "agent.stop_listening", "event_id": "<id>"} | Transitions the avatar from listening to idle. |
session.keep_alive | {"type": "session.keep_alive", "event_id": "<id>"} | Keeps the session alive beyond the 5-minute idle timeout. Send periodically to extend the session. |
Response Events (you receive)
| Event | Payload | Description |
|---|---|---|
session.state_updated | {"state": "connected" | "connecting" | "closed" | "closing"} | Tracks session state transitions. Wait for "connected" before sending events. |
agent.speak_started | {"event_id": "<id>", "task": {"id": "<id>"}} | The avatar began speaking. event_id matches the triggering agent.speak command. |
agent.speak_ended | {"event_id": "<id>", "task": {"id": "<id>"}} | The avatar finished speaking. Always follows an agent.speak_started event. |
Wait for
session.state_updated with state "connected" before sending any command events.