Skip to main content
Every session in this mode follows three phases: starting, managing, and ending.

​
1. Starting a Session

  1. Generate a session token configured for this mode on your backend
  2. Start the session using that token
  3. The avatar streams into the specified WebRTC room after initialization
Your backend should manage both token generation and session start. Pass the WebRTC credentials to your frontend.
Ensure the session token is explicitly set to "LITE" mode. The token configuration determines which mode initializes.

​
2. Managing the Session

This mode provides a WebSocket connection for controlling the avatar. The typical flow:
  1. User speaks — audio is sent to the room
  2. Your agent processes — your STT/LLM/TTS pipeline handles the input
  3. Agent constructs response audio — your TTS generates the speech
  4. Agent streams audio via WebSocket — send audio chunks to LiveAvatar
  5. LiveAvatar renders video — avatar video frames are sent to the room
Your “agent” can be anything from a simple backend service to a complex multi-model pipeline.

​
Audio format

Audio sent to LiveAvatar must be PCM 16-bit 24 kHz, Base64-encoded. Recommended chunk size is ~1 second, with a maximum of 1MB per WebSocket packet.

​
Latency

LiveAvatar generates avatar video in real time as audio arrives — there is no batch processing or queuing delay on the LiveAvatar side.
  • Plugin path — end-to-end latency depends on your pipeline (STT, LLM, TTS, and network). LiveAvatar adds minimal overhead on top of your existing stack.
  • Connector path — LiveAvatar manages the full connection to your voice agent and optimizes latency on your behalf.

​
WebSocket commands

Through the WebSocket, you can:
  • Command the avatar to speak (by sending audio)
  • Interrupt avatar responses
  • Modify avatar poses (listening, idle)
  • Keep sessions alive
Audio you send is grouped into utterances. Each utterance is identified by the event_id of its first agent.speak chunk, and every server event about that utterance carries the same id as source_event_id. See Events for the full envelope and event list.

​
3. Ending the Session

When a session ends:
  1. The avatar is removed from the LiveKit room
  2. The room is torn down (if created by LiveAvatar)
  3. The WebSocket connection closes
Clean up your WebSocket resources and LiveKit data after the session ends.