Server Events

In addition, we also describe what is currently happening. By listening and processing these events, developers can build realtime interactions on top of these events.

Full Mode Events

We will emit these events from the LiveKit Room. Developers should only listen to the event architecture there.

Name	Event Data	Description
`user.speak_started`	N/A	The user in the LiveKit room has started sending us audio data.
`user.speak_ended`	N/A	The user in the LiveKit room is no longer sending us audio data. Always follows a `user.speak_started` event
`avatar.speak_started`	N/A	The avatar in the LiveKit room started sending audio output
`avatar.speak_ended`	N/A	The avatar in the LiveKit room finished sending audio output. Always follows a `avatar.speak_started` event.
`user.transcription_started`	N/A	We've started processing the input audio and converting it into text.
`user.transcription_ended`	`{"text": string}`	We've finished processing the input audio. The transcribed text is included in the event data under `text`.
`avatar.transcription_started`	N/A	The avatar's text response is being formed. When calling `avatar.speak_text`, we will immediately emit this event and `avatar.transcription_ended` with the input text. When calling `avatar.speak_response, we will emit this event prior to LLM generation.
`avatar.transcription_ended`	`{"text": string}`	The avatar's text response is being formed. When calling `avatar.speak_text`, we will immediately emit this event and `avatar.transcription_ended` with the input text. When calling `avatar.speak_response`, we emit this event after LLM generation.

Custom Mode Events.

We will emit these events from the websocket that is public when calling custom mode. This is a smaller set of events.

Name	Event Data	Description
`agent.speak_started`	`{ "type":"agent.speak_started", "event_id": <str>, "task": {"id": <str> } }`	Avatar started speaking. The event id matches the corresponding, `agent.speak` command event.
`agent.speak_ended`	`{"type":"agent.speak_started", "event_id": <str>, "task": {"id": <str> }}`	Avatar started speaking. The event id matches the corresponding, `agent.speak` command event. Note - should always follow a `agent.speak_started` event.

Name

Event Data

Description

agent.speak_started

{ "type":"agent.speak_started", "event_id": <str>, "task": {"id": <str> } }

Avatar started speaking. The event id matches the corresponding, agent.speak command event.

agent.speak_ended

{"type":"agent.speak_started", "event_id": <str>, "task": {"id": <str> }}

Avatar started speaking. The event id matches the corresponding, agent.speak command event.

Note - should always follow a agent.speak_started event.