Command Events

With command events, developers send us events to direct avatar actions. Some events are only available depending on which mode you're in.

Full Mode Avatar Events

In Full Mode, we process events that are sent via the LiveKit Protocol. Simply send these events into the LiveKit room session and we will process all of them.

Below is the list of all events currently available and what they do.

Name	Event Data	Description
`avatar.interrupt`	N/A	Interrupts and stops any scheduled commands. This will disrupt the entire sequence of calls passed in.
`avatar.speak_text`	`{"text": string}`	Instructs the avatar to say the spoken text
`avatar.speak_response`	`{"text": string}`	Instruct the avatar to generate an LLM response to the input text
`avatar.interrupt_video`	N/A	Interrupts and stops any generated ongoing video generation. Equivalent to `avatar.interrupt` in FULL mode.
`avatar.speak_audio`	`{"audio": base64AudioString}`	Instructs the avatar to voice the input audio
`avatar.start_listening`	N/A	Instructs the avatar to switch to a listening state. Most of the time, avatars sit in an idle state when they are not speaking.
`avatar.stop_listening`	N/A	Instructs the avatar to leave the listening state and return to the idle state.

Custom Mode Events

In Custom Mode, you can send us events via websocket endpoint. As of now, we primarily support transitioning the Avatar between speaking and listening, as well as interrupting any existing events that are happening.

Name	Event Data	Description
`agent.interrupt`	`{"type": "agent.interrupt"}`	Interrupts and stops any scheduled commands. This will clear all scheduled speaking events.
`agent.speak`	`{"type": "agent.speak", "audio": <string>}`	Adds to the avatar buffer for what it will say. The event_id can be set to indicate grouping between sets of interactions. Audio should be PCM 16Bit 24KHz bytes encoded as Base64 Note - we recommend sending many small byte packets. We may close the connection if we detect too large of an audio string input (> 1MB).
`avatar.speak_end`	`{ "type": "agent.speak_end", "event_id": <str> }`	Signal the end of a speaking event.
`agent.start_listening`	`{"type":"agent.start_listening", "event_id": <str>}`	Transition the avatar to a listening state.
`agent.stop_listening`	`{"type":"agent.stop_listening", "event_id": <str>}`	Transition the avatar from a listening state to an idle state.

We intend on adding additional events depending on what our users request. Leave feedback if there are any additional commands you'd like to control.