FULL Mode Events

In FULL Mode, we process events that are sent via the LiveKit Protocol. Simply send these events into the LiveKit room session and we will process all of them. All FULL mode events will be structured in the following JSON format.

{
  "event_type": str, // name of the event
  "session_id": str, // session id
  ... // event data passed passed in. 
}

To differentiate between command and server events, we track different LiveKit room topics. You will need to publish/subscribe to specific topics in-order to track this data.

Command Events

For command events, these events need to be published to this topic: agent-control.

Below is the list of all events available that allow you to better control the state of the avatar.

Name / event_type	Additional Event Data	Description
`avatar.interrupt`	N/A	Interrupts and stops any scheduled commands. This will disrupt the entire sequence of calls passed in.
`avatar.speak_text`	`{"text": string}`	Instructs the avatar to say the spoken text
`avatar.speak_response`	`{"text": string}`	Instruct the avatar to generate an LLM response to the input text
`avatar.start_listening`	N/A	Instructs the avatar to switch to a listening state. Most of the time, avatars sit in an idle state when they are not speaking.
`avatar.stop_listening`	N/A	Instructs the avatar to leave the listening state and return to the idle state.
`user.start_push_to_talk`	N/A	(Push-To-Talk Interactivity Only) Signal to start taking in user audio
`user.stop_push_to_talk`	N/A	(Push-To-Talk Interactivity Only) Signal to stop taking in user audio

Server Events

For server events, these events will be emitted to the topic, agent-response

Throughout the Session, we will emit these events. Developers will be able to listen in to whichever events they deem necessary to build additional experiences on the user interface.

Name / event_type	Additional Event Data	Description
`user.speak_started`	N/A	The user in the LiveKit room has started sending us audio data.
`user.speak_ended`	N/A	The user in the LiveKit room is no longer sending us audio data. Always follows a `user.speak_started` event
`avatar.speak_started`	N/A	The avatar in the LiveKit room started sending audio output
`avatar.speak_ended`	N/A	The avatar in the LiveKit room finished sending audio output. Always follows a `avatar.speak_started` event.
`user.transcription`	`{"text": string}`	We've finished processing the input audio. The transcribed text is included in the event data under `text`.
`avatar.transcription`	`{"text": string}`	The avatar's text response. When calling `avatar.speak_text`, we will immediately emit this event and `avatar.transcription_ended` with the input text. When calling `avatar.speak_response`, we emit this event after LLM generation.
`user.push_to_talk_started`	N/A	(Push-To-Talk Interactivity Only) Success on processing client push to talk start event
`user.push_to_talk_start_failed`	N/A	(Push-To-Talk Interactivity Only) Fail to process client push to talk event
`user.push_to_talk_started`	N/A	(Push-To-Talk Interactivity Only) Success on processing client push to talk stop event
`user.push_to_talk_stop_failed`	N/A	(Push-To-Talk Interactivity Only) Fail to process client push to talk stop event
`session.stopped`	`{"end_reason": string}`	When the session ends, we emit this event along with the reason the session was stopped.