FULL Mode Events

In Full Mode, we process events that are sent via the LiveKit Protocol. Simply send these events into the LiveKit room session and we will process all of them. All FULL mode events will be structured in the following JSON format.

{
  "event_type": str, // name of the event
  ... // event data passed passed in. 
}

To differentiate between command and server events, we track different LiveKit room topics. You will need to publish/subscribe to specific topics in-order to track this data.

Command Events

For command events, these events need to be published to this topic: agent-control.

Below is the list of all events available that allow you to better control the state of the avatar.

Name / event_type	Additional Event Data	Description
`avatar.interrupt`	N/A	Interrupts and stops any scheduled commands. This will disrupt the entire sequence of calls passed in.
`avatar.speak_text`	`{"text": string}`	Instructs the avatar to say the spoken text
`avatar.speak_response`	`{"text": string}`	Instruct the avatar to generate an LLM response to the input text
`avatar.start_listening`	N/A	Instructs the avatar to switch to a listening state. Most of the time, avatars sit in an idle state when they are not speaking.
`avatar.stop_listening`	N/A	Instructs the avatar to leave the listening state and return to the idle state.

Server Events

For server events, these events will be emitted to the topic, agent-response

Throughout the Session, we will emit these events. Developers will be able to listen in to whichever events they deem necessary to build additional experiences on the user interface.

Name / event_type	Additional Event Data	Description
`user.speak_started`	N/A	The user in the LiveKit room has started sending us audio data.
`user.speak_ended`	N/A	The user in the LiveKit room is no longer sending us audio data. Always follows a `user.speak_started` event
`avatar.speak_started`	N/A	The avatar in the LiveKit room started sending audio output
`avatar.speak_ended`	N/A	The avatar in the LiveKit room finished sending audio output. Always follows a `avatar.speak_started` event.
`user.transcription_started`	N/A	We've started processing the input audio and converting it into text.
`user.transcription_ended`	`{"text": string}`	We've finished processing the input audio. The transcribed text is included in the event data under `text`.
`avatar.transcription_started`	N/A	The avatar's text response is being formed. When calling `avatar.speak_text`, we will immediately emit this event and `avatar.transcription_ended` with the input text. When calling `avatar.speak_response, we will emit this event prior to LLM generation.
`avatar.transcription_ended`	`{"text": string}`	The avatar's text response is being formed. When calling `avatar.speak_text`, we will immediately emit this event and `avatar.transcription_ended` with the input text. When calling `avatar.speak_response`, we emit this event after LLM generation.