Skip to main content
In the Custom LiveKit Agent guide, LiveAvatar provisions the LiveKit room and you send an agent into it. This guide covers the other case: you own the LiveKit room, and the LiveAvatar avatar joins as a participant. This guide walks through a Python starter that does exactly that.

liveavatar-starter-livekit-agent-python

Reference implementation for this guide. Demo entrypoint: src/byo_livekit_demo.py.
This guide is best suited for developers who already run a LiveKit deployment and need control beyond what the LiveKit avatar plugin provides. The plugin handles the orchestration between your LiveKit agent and the avatar participant out of the box — start there first. Reach for the BYO flow below only if your use case needs more control over room lifecycle, dispatch, recording, or worker fleet.

​
Prerequisites

Before you start, make sure you have:
  • A LiveAvatar account with an API key — sign up at app.liveavatar.com.
  • An AVATAR_ID from your account. The default in the starter repo is sandbox-compatible; swap it for one of your own avatars when going to production.
  • A LiveKit Cloud project with LIVEKIT_URL, LIVEKIT_API_KEY, and LIVEKIT_API_SECRET. Sign up at cloud.livekit.io if you do not have one.
  • Python 3.11+ and uv (or pip) to install the starter’s dependencies.
  • The LiveKit CLI (brew install livekit-cli, then lk cloud auth) for deploying the worker to LiveKit Cloud.
Unlike the Custom LiveKit Agent flow, LIVEKIT_URL is required here — you own the room, so you tell LiveAvatar where to join.

​
Quickstart

Clone the reference implementation and run it against the LiveAvatar API and your own LiveKit project. 
git clone https://github.com/heygen-com/liveavatar-starter-livekit-agent-python
cd liveavatar-starter-livekit-agent-python
cp .env.example .env.local
Set the following in .env.local: 
LIVEAVATAR_API_KEY=...   # from app.liveavatar.com
AVATAR_ID=...            # any avatar in your account; sandbox-compatible by default
LIVEKIT_URL=wss://<your-project>.livekit.cloud
LIVEKIT_API_KEY=...      # your LK Cloud project — used for the room AND inference
LIVEKIT_API_SECRET=...
IS_SANDBOX=true          # default; switch to false for production avatars / billed minutes
Install and run two terminals: 
uv sync

# Terminal 1 — register the worker against your LK project
python src/worker.py dev

# Terminal 2 — drive a session
python src/byo_livekit_demo.py
A new browser tab should pop up with the viewer connected to your LiveKit room. Try speaking to the avatar.
Both terminals load credentials from .env.local via load_dotenv. The dev worker registers against whichever LK project LIVEKIT_URL / LIVEKIT_API_KEY / LIVEKIT_API_SECRET point to — make sure those match the project you intend to dispatch into.
src/worker.py exposes three CLI subcommands inherited from LiveKit Agents:
SubcommandPurpose
devLocal development — registers the worker against your LK project, hot-reloads on file changes. Use this in Terminal 1 above.
startProduction entrypoint baked into the Dockerfile (CMD). LK Cloud runs this in the deployed worker.
download-filesDownloads VAD + turn-detector model weights. Run at Docker build time so cold-starts in LK Cloud are fast.
To verify the worker is actually registered before running the dispatcher, check lk agent status (after deploy) or look for a registered worker log line in Terminal 1. Behind the scenes, the dispatcher mints room tokens, asks LiveAvatar to join your room as a participant, and dispatches your worker into the room by name. The next sections break down the full flow.

​
Understanding the flow

  1. You mint two LiveKit tokens. One for the LiveAvatar avatar, one for the viewer.
  2. LiveAvatar joins your room. Using the minted avatar token, LiveAvatar dispatches its avatar into your room as a participant.
  3. Your agent joins your room. Your worker is dispatched into the same room by name, alongside the avatar and the viewer.
  4. The worker drives the connection. Using the WebSocket returned when the session starts, the worker controls the avatar — running STT → LLM → TTS and streaming audio for lip-sync.

​
Ownership at a glance

ConcernOwned by
LiveKit roomYou (your LK Cloud project)
Room participant-minutesYou
Avatar + viewer tokensYou (you mint both)
Agent worker fleetYou (LK Cloud or self-hosted)
Avatar minutes / creditsYou (your LiveAvatar account)
Inference billing (STT / LLM / TTS)You (your LK Cloud project)

​
Understanding the code

When we run python src/byo_livekit_demo.py the dispatcher does the following:
  1. Mint room + tokens against your LK project. Generates a fresh room name, then mints separate tokens for the avatar participant and the browser viewer.
  2. Create a LiveAvatar session pinned to your room. Calls create_session_token with livekit_config={livekit_url, livekit_room, livekit_client_token=avatar_token} so LiveAvatar’s avatar joins your room.
  3. Start the session. Calls start_session to get the avatar media-server WebSocket URL (ws_url).
  4. Dispatch the agent into the room. Calls AgentDispatchService.create_dispatch(agent_name=AGENT_NAME, room=room_name, metadata=json.dumps({ws_url, session_id})). LK Cloud routes the dispatch to a registered worker.
  5. Open the viewer. Serves viewer/index.html locally and opens a browser tab with LIVEKIT_URL + viewer token preloaded.
  6. Worker accepts the dispatch. Reads metadata, opens the LiveAvatar WebSocket, runs STT → LLM → TTS, and tees synthesized audio to the WebSocket for lip-sync.
  7. Clean up on exit. The worker registers a shutdown callback that calls stop_session on the LiveAvatar API whenever the job ends.
agent_name dispatch is not automatic dispatch. The worker must register under a name (@server.rtc_session(agent_name=AGENT_NAME) in worker.py) and the dispatcher must call create_dispatch with the same name. Workers using LiveKit’s default auto-dispatch won’t receive these jobs.
livekit_config.livekit_client_token is the avatar’s token, not the viewer’s. The avatar uses it to join your room as a participant. Mint the viewer token separately for the browser client.

​
File structure

The starter is split into a backend (the dispatcher + the worker) and a small frontend used to visualize the session locally. Backend (src/):
FileResponsibility
liveavatar_client.pyMakes API calls to LiveAvatar (create session, start session, stop session).
avatar_ws.pyMaintains the WebSocket connection to LiveAvatar and forwards audio frames for lip-sync.
agent.pyDefines the agent that handles the conversation and tees synthesized audio to the LiveAvatar WebSocket.
pipeline.pyBuilds the STT → LLM → TTS pipeline, wires up room + session observability, and exposes mute_agent_audio_on_publish (silences the agent’s own LK audio track so the viewer only hears the avatar — otherwise audio doubles).
worker.pyDefines the AgentServer worker that registers under AGENT_NAME and accepts dispatched jobs. Reads ws_url + session_id from ctx.job.metadata.
byo_livekit_demo.pyDemo dispatcher — mints tokens, calls the LiveAvatar API, dispatches the worker, and opens the local viewer.
Frontend (viewer/): We ship a minimal frontend so you can see the avatar and talk to it locally without standing up your own client. Treat it as a reference, not as production frontend code.
FileResponsibility
index.htmlVanilla LiveKit JS client. Connects to your LK room with the viewer token, publishes the user’s microphone, and renders the avatar’s audio + video.
The avatar joins the room under the participant identity "avatar" (set as AVATAR_IDENTITY in byo_livekit_demo.py). When building your own frontend, subscribe to that identity’s audio + video tracks to render the avatar.

​
Customizing the agent

The most common things you’ll want to tweak after the quickstart:
WhatWhere
AGENT_NAMEworker.py — must match the agent_name passed to create_dispatch. Rename from the default my-agent before deploying.
System prompt / agent personaagent.py — the LiveAvatarAgent definition.
LLM model, STT provider, TTS voicepipeline.py — build_session configures all three.
Audio input options (noise cancellation, etc.)pipeline.py — build_room_options.
Logging / observability hookspipeline.py — wire_room_observability, wire_session_observability.

​
Productionizing

This demo is wired up for fast local iteration. When productionizing, move dispatch logic to a backend service, deploy the worker to LiveKit Cloud, and tighten up session lifecycle.

​
Improvements beyond the demo

A few things in the starter exist purely for local convenience. We recommend the following swaps:
  • Build your own frontend. Replace the local viewer HTTP server and auto-launched browser tab with a real client that joins your LK room using the viewer token.
  • Move dispatch to a backend service. byo_livekit_demo.py mints LK tokens and calls the LiveAvatar API client-side because it’s a single-process demo. In production, that logic belongs on a backend that authenticates the user, mints the viewer token, calls create_session_token + start_session + create_dispatch, and returns only {room_url, viewer_token} to the browser. Never ship LIVEKIT_API_SECRET, LIVEAVATAR_API_KEY, or the avatar token to the browser.
  • Deploy the worker to LiveKit Cloud. python src/worker.py dev is for local iteration only. The repo ships a Dockerfile and livekit.toml for lk agent deploy (see below).
  • Rename AGENT_NAME. The default my-agent collides across LK Cloud accounts and makes lk agent logs painful. Pick something project-specific (e.g. support-agent, tutor-bot) before deploying.
  • Guard against leaked sessions. The worker registers an add_shutdown_callback that calls stop_session whenever the job ends. This is the load-bearing cleanup path in production — your backend dispatcher should fire-and-forget after create_dispatch and let the worker handle lifecycle. As a safety net for runners that crash without it (SIGKILL, OOM), set max_session_duration server-side when calling create_session_token so LiveAvatar will close abandoned sessions on its own.
  • Manual termination. To end a session early (e.g. user clicks “end call”), call stop_session(session_id) directly from your backend. The worker’s shutdown callback will fire when the job is torn down.
The Ctrl-C cleanup in byo_livekit_demo.py is belt-and-suspenders for the demo. Do not copy it into a backend dispatcher service. A backend dispatcher should exit immediately after create_dispatch and let the worker’s shutdown callback close the LiveAvatar session.

​
Deploying the worker to LiveKit Cloud

The repo ships a Dockerfile and livekit.toml for lk agent deploy. 
# First-time setup — writes subdomain + agent id back into livekit.toml
lk agent create --secrets-file .env.local

# Subsequent updates
lk agent deploy

# Inspect
lk agent status
lk agent logs
lk agent create modifies livekit.toml in place — it fills [project].subdomain and [agent].id. Commit the diff, otherwise teammates and CI won’t be able to redeploy the same agent.
The Dockerfile runs python src/worker.py download-files at build time to bake VAD + turn-detector model weights into the image. This is what makes cold-starts in LK Cloud fast. Don’t strip it.
Don’t bake LIVEKIT_* env vars into the image. LK Cloud injects LIVEKIT_URL / LIVEKIT_API_KEY / LIVEKIT_API_SECRET at runtime. Only ship application-level secrets (LIVEAVATAR_API_KEY, etc.) via --secrets-file.
After deploy, drop the python src/worker.py dev terminal entirely. Dispatches go straight to the cloud-managed worker fleet.

​
Troubleshooting

SymptomLikely causeFix
Avatar never joins the roomlivekit_config missing or wrong (bad livekit_url, room name mismatch, expired/invalid avatar token)Verify all three fields in livekit_config. Confirm the avatar token is for the same room name passed in livekit_room. Check token TTL.
Worker registered but no jobs arriveagent_name mismatch between worker and dispatcher, or worker uses default auto-dispatch@server.rtc_session(agent_name=...) on the worker must equal the agent_name in create_dispatch. Named dispatch and auto-dispatch are mutually exclusive.
start_session returned but ws_url is emptySession failed to allocate a media server (sandbox quota, region, billing)Inspect the response body. Re-check IS_SANDBOX + AVATAR_ID compatibility. Retry.
Avatar visible but silent (no lip-sync)Worker never opened the LiveAvatar WebSocket, or tts_node not teeing audio to itConfirm meta["ws_url"] parsed correctly from job metadata. Check worker logs for AvatarWebSocket connect errors.
Viewer hears doubled / echoed audioAgent audio track not muted on publishmute_agent_audio_on_publish(ctx.room) must be called in the worker entrypoint. The avatar is the audio source — the agent’s own track has to be silenced.
lk agent deploy fails on first runlk agent create was skipped — livekit.toml still has empty subdomain / idRun lk agent create --secrets-file .env.local first. Commit the resulting livekit.toml diff.
Session leaks after worker crash (SIGKILL)Shutdown callback can’t run on SIGKILLSet max_session_duration on create_session_token as a TTL safety net.