> ## Documentation Index
> Fetch the complete documentation index at: https://docs.liveavatar.com/llms.txt
> Use this file to discover all available pages before exploring further.

# BYO LiveKit Agent

> Add LiveAvatar as a participant to your LiveKit room

In the [Custom LiveKit Agent](/docs/guides/livekit/custom-livekit-agent) guide, LiveAvatar provisions the LiveKit room and you send an agent into it. This guide covers the other case: **you own the LiveKit room**, and the LiveAvatar avatar joins as a participant.

This guide walks through a Python starter that does exactly that.

<Card title="liveavatar-starter-livekit-agent-python" icon="github" href="https://github.com/heygen-com/liveavatar-starter-livekit-agent-python">
  Reference implementation for this guide. Demo entrypoint: `src/byo_livekit_demo.py`.
</Card>

<Note>
  This guide is best suited for developers who already run a LiveKit deployment **and** need control beyond what the [LiveKit avatar plugin](/docs/lite-mode/plugins/livekit) provides. The plugin handles the orchestration between your LiveKit agent and the avatar participant out of the box — start there first. Reach for the BYO flow below only if your use case needs more control over room lifecycle, dispatch, recording, or worker fleet.
</Note>

## Prerequisites

Before you start, make sure you have:

* A **LiveAvatar account** with an API key — sign up at [app.liveavatar.com](https://app.liveavatar.com).
* An `AVATAR_ID` from your account. The default in the starter repo is sandbox-compatible; swap it for one of your own avatars when going to production.
* A **LiveKit Cloud project** with `LIVEKIT_URL`, `LIVEKIT_API_KEY`, and `LIVEKIT_API_SECRET`. Sign up at [cloud.livekit.io](https://cloud.livekit.io) if you do not have one.
* **Python 3.11+** and [`uv`](https://docs.astral.sh/uv/) (or `pip`) to install the starter's dependencies.
* The **LiveKit CLI** (`brew install livekit-cli`, then `lk cloud auth`) for deploying the worker to LiveKit Cloud.

Unlike the [Custom LiveKit Agent flow](/docs/guides/livekit/custom-livekit-agent), `LIVEKIT_URL` **is** required here — you own the room, so you tell LiveAvatar where to join.

## Quickstart

Clone the reference implementation and run it against the LiveAvatar API and your own LiveKit project.

```bash theme={null}
git clone https://github.com/heygen-com/liveavatar-starter-livekit-agent-python
cd liveavatar-starter-livekit-agent-python
cp .env.example .env.local
```

Set the following in `.env.local`:

```bash theme={null}
LIVEAVATAR_API_KEY=...   # from app.liveavatar.com
AVATAR_ID=...            # any avatar in your account; sandbox-compatible by default
LIVEKIT_URL=wss://<your-project>.livekit.cloud
LIVEKIT_API_KEY=...      # your LK Cloud project — used for the room AND inference
LIVEKIT_API_SECRET=...
IS_SANDBOX=true          # default; switch to false for production avatars / billed minutes
```

Install and run two terminals:

```bash theme={null}
uv sync

# Terminal 1 — register the worker against your LK project
python src/worker.py dev

# Terminal 2 — drive a session
python src/byo_livekit_demo.py
```

A new browser tab should pop up with the viewer connected to your LiveKit room. Try speaking to the avatar.

<Note>
  Both terminals load credentials from `.env.local` via `load_dotenv`. The `dev` worker registers against whichever LK project `LIVEKIT_URL` / `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` point to — make sure those match the project you intend to dispatch into.
</Note>

`src/worker.py` exposes three CLI subcommands inherited from LiveKit Agents:

| Subcommand       | Purpose                                                                                                                      |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `dev`            | Local development — registers the worker against your LK project, hot-reloads on file changes. Use this in Terminal 1 above. |
| `start`          | Production entrypoint baked into the Dockerfile (`CMD`). LK Cloud runs this in the deployed worker.                          |
| `download-files` | Downloads VAD + turn-detector model weights. Run at Docker build time so cold-starts in LK Cloud are fast.                   |

To verify the worker is actually registered before running the dispatcher, check `lk agent status` (after deploy) or look for a `registered worker` log line in Terminal 1.

Behind the scenes, the dispatcher mints room tokens, asks LiveAvatar to join your room as a participant, and dispatches your worker into the room by name. The next sections break down the full flow.

## Understanding the flow

1. **You mint two LiveKit tokens.** One for the LiveAvatar avatar, one for the viewer.
2. **LiveAvatar joins your room.** Using the minted avatar token, LiveAvatar dispatches its avatar into your room as a participant.
3. **Your agent joins your room.** Your worker is dispatched into the same room by name, alongside the avatar and the viewer.
4. **The worker drives the connection.** Using the WebSocket returned when the session starts, the worker controls the avatar — running STT → LLM → TTS and streaming audio for lip-sync.

### Ownership at a glance

| Concern                             | Owned by                      |
| ----------------------------------- | ----------------------------- |
| LiveKit room                        | You (your LK Cloud project)   |
| Room participant-minutes            | You                           |
| Avatar + viewer tokens              | You (you mint both)           |
| Agent worker fleet                  | You (LK Cloud or self-hosted) |
| Avatar minutes / credits            | You (your LiveAvatar account) |
| Inference billing (STT / LLM / TTS) | You (your LK Cloud project)   |

```mermaid theme={null}
graph TB
    subgraph Room["Your LiveKit room"]
        Agent["Your agent"]
        User["End user"]
        Avatar["Avatar"]
    end
    Inference["Inference gateways<br/>(STT / LLM / TTS)"]
    User -->|"user audio/data"| Agent
    Agent -->|"audio (WS)"| Avatar
    Avatar -->|"realtime video"| User
    Agent <--> Inference
```

## Understanding the code

When we run `python src/byo_livekit_demo.py` the dispatcher does the following:

1. **Mint room + tokens against your LK project.** Generates a fresh room name, then mints separate tokens for the avatar participant and the browser viewer.
2. **Create a LiveAvatar session pinned to your room.** Calls `create_session_token` with `livekit_config={livekit_url, livekit_room, livekit_client_token=avatar_token}` so LiveAvatar's avatar joins **your** room.
3. **Start the session.** Calls `start_session` to get the avatar media-server WebSocket URL (`ws_url`).
4. **Dispatch the agent into the room.** Calls `AgentDispatchService.create_dispatch(agent_name=AGENT_NAME, room=room_name, metadata=json.dumps({ws_url, session_id}))`. LK Cloud routes the dispatch to a registered worker.
5. **Open the viewer.** Serves `viewer/index.html` locally and opens a browser tab with `LIVEKIT_URL` + viewer token preloaded.
6. **Worker accepts the dispatch.** Reads metadata, opens the LiveAvatar WebSocket, runs STT → LLM → TTS, and tees synthesized audio to the WebSocket for lip-sync.
7. **Clean up on exit.** The worker registers a shutdown callback that calls `stop_session` on the LiveAvatar API whenever the job ends.

<Warning>
  `agent_name` dispatch is **not** automatic dispatch. The worker must register under a name (`@server.rtc_session(agent_name=AGENT_NAME)` in `worker.py`) and the dispatcher must call `create_dispatch` with the same name. Workers using LiveKit's default auto-dispatch won't receive these jobs.
</Warning>

<Note>
  `livekit_config.livekit_client_token` is the **avatar's** token, not the viewer's. The avatar uses it to join your room as a participant. Mint the viewer token separately for the browser client.
</Note>

### File structure

The starter is split into a backend (the dispatcher + the worker) and a small frontend used to visualize the session locally.

**Backend** (`src/`):

| File                   | Responsibility                                                                                                                                                                                                                 |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `liveavatar_client.py` | Makes API calls to LiveAvatar (create session, start session, stop session).                                                                                                                                                   |
| `avatar_ws.py`         | Maintains the WebSocket connection to LiveAvatar and forwards audio frames for lip-sync.                                                                                                                                       |
| `agent.py`             | Defines the agent that handles the conversation and tees synthesized audio to the LiveAvatar WebSocket.                                                                                                                        |
| `pipeline.py`          | Builds the STT → LLM → TTS pipeline, wires up room + session observability, and exposes `mute_agent_audio_on_publish` (silences the agent's own LK audio track so the viewer only hears the avatar — otherwise audio doubles). |
| `worker.py`            | Defines the `AgentServer` worker that registers under `AGENT_NAME` and accepts dispatched jobs. Reads `ws_url` + `session_id` from `ctx.job.metadata`.                                                                         |
| `byo_livekit_demo.py`  | Demo dispatcher — mints tokens, calls the LiveAvatar API, dispatches the worker, and opens the local viewer.                                                                                                                   |

**Frontend** (`viewer/`):

We ship a minimal frontend so you can see the avatar and talk to it locally without standing up your own client. Treat it as a reference, not as production frontend code.

| File         | Responsibility                                                                                                                                      |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `index.html` | Vanilla LiveKit JS client. Connects to your LK room with the viewer token, publishes the user's microphone, and renders the avatar's audio + video. |

<Note>
  The avatar joins the room under the participant identity `"avatar"` (set as `AVATAR_IDENTITY` in `byo_livekit_demo.py`). When building your own frontend, subscribe to that identity's audio + video tracks to render the avatar.
</Note>

### Customizing the agent

The most common things you'll want to tweak after the quickstart:

| What                                           | Where                                                                                                                       |
| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `AGENT_NAME`                                   | `worker.py` — must match the `agent_name` passed to `create_dispatch`. Rename from the default `my-agent` before deploying. |
| System prompt / agent persona                  | `agent.py` — the `LiveAvatarAgent` definition.                                                                              |
| LLM model, STT provider, TTS voice             | `pipeline.py` — `build_session` configures all three.                                                                       |
| Audio input options (noise cancellation, etc.) | `pipeline.py` — `build_room_options`.                                                                                       |
| Logging / observability hooks                  | `pipeline.py` — `wire_room_observability`, `wire_session_observability`.                                                    |

## Productionizing

This demo is wired up for fast local iteration. When productionizing, move dispatch logic to a backend service, deploy the worker to LiveKit Cloud, and tighten up session lifecycle.

### Improvements beyond the demo

A few things in the starter exist purely for local convenience. We recommend the following swaps:

* **Build your own frontend.** Replace the local viewer HTTP server and auto-launched browser tab with a real client that joins your LK room using the viewer token.
* **Move dispatch to a backend service.** `byo_livekit_demo.py` mints LK tokens and calls the LiveAvatar API client-side because it's a single-process demo. In production, that logic belongs on a backend that authenticates the user, mints the viewer token, calls `create_session_token` + `start_session` + `create_dispatch`, and returns only `{room_url, viewer_token}` to the browser. Never ship `LIVEKIT_API_SECRET`, `LIVEAVATAR_API_KEY`, or the avatar token to the browser.
* **Deploy the worker to LiveKit Cloud.** `python src/worker.py dev` is for local iteration only. The repo ships a `Dockerfile` and `livekit.toml` for `lk agent deploy` (see below).
* **Rename `AGENT_NAME`.** The default `my-agent` collides across LK Cloud accounts and makes `lk agent logs` painful. Pick something project-specific (e.g. `support-agent`, `tutor-bot`) before deploying.
* **Guard against leaked sessions.** The worker registers an `add_shutdown_callback` that calls `stop_session` whenever the job ends. This is the load-bearing cleanup path in production — your backend dispatcher should fire-and-forget after `create_dispatch` and let the worker handle lifecycle. As a safety net for runners that crash without it (SIGKILL, OOM), set `max_session_duration` server-side when calling `create_session_token` so LiveAvatar will close abandoned sessions on its own.
* **Manual termination.** To end a session early (e.g. user clicks "end call"), call `stop_session(session_id)` directly from your backend. The worker's shutdown callback will fire when the job is torn down.

<Warning>
  The `Ctrl-C` cleanup in `byo_livekit_demo.py` is belt-and-suspenders for the demo. **Do not copy it into a backend dispatcher service.** A backend dispatcher should exit immediately after `create_dispatch` and let the worker's shutdown callback close the LiveAvatar session.
</Warning>

### Deploying the worker to LiveKit Cloud

The repo ships a `Dockerfile` and `livekit.toml` for `lk agent deploy`.

```bash theme={null}
# First-time setup — writes subdomain + agent id back into livekit.toml
lk agent create --secrets-file .env.local

# Subsequent updates
lk agent deploy

# Inspect
lk agent status
lk agent logs
```

<Warning>
  `lk agent create` modifies `livekit.toml` in place — it fills `[project].subdomain` and `[agent].id`. **Commit the diff**, otherwise teammates and CI won't be able to redeploy the same agent.
</Warning>

<Note>
  The `Dockerfile` runs `python src/worker.py download-files` at build time to bake VAD + turn-detector model weights into the image. This is what makes cold-starts in LK Cloud fast. Don't strip it.
</Note>

<Warning>
  **Don't bake `LIVEKIT_*` env vars into the image.** LK Cloud injects `LIVEKIT_URL` / `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET` at runtime. Only ship application-level secrets (`LIVEAVATAR_API_KEY`, etc.) via `--secrets-file`.
</Warning>

After deploy, drop the `python src/worker.py dev` terminal entirely. Dispatches go straight to the cloud-managed worker fleet.

## Troubleshooting

| Symptom                                        | Likely cause                                                                                            | Fix                                                                                                                                                            |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Avatar never joins the room                    | `livekit_config` missing or wrong (bad `livekit_url`, room name mismatch, expired/invalid avatar token) | Verify all three fields in `livekit_config`. Confirm the avatar token is for the same room name passed in `livekit_room`. Check token TTL.                     |
| Worker registered but no jobs arrive           | `agent_name` mismatch between worker and dispatcher, or worker uses default auto-dispatch               | `@server.rtc_session(agent_name=...)` on the worker must equal the `agent_name` in `create_dispatch`. Named dispatch and auto-dispatch are mutually exclusive. |
| `start_session` returned but `ws_url` is empty | Session failed to allocate a media server (sandbox quota, region, billing)                              | Inspect the response body. Re-check `IS_SANDBOX` + `AVATAR_ID` compatibility. Retry.                                                                           |
| Avatar visible but silent (no lip-sync)        | Worker never opened the LiveAvatar WebSocket, or `tts_node` not teeing audio to it                      | Confirm `meta["ws_url"]` parsed correctly from job metadata. Check worker logs for `AvatarWebSocket` connect errors.                                           |
| Viewer hears doubled / echoed audio            | Agent audio track not muted on publish                                                                  | `mute_agent_audio_on_publish(ctx.room)` must be called in the worker entrypoint. The avatar is the audio source — the agent's own track has to be silenced.    |
| `lk agent deploy` fails on first run           | `lk agent create` was skipped — `livekit.toml` still has empty `subdomain` / `id`                       | Run `lk agent create --secrets-file .env.local` first. Commit the resulting `livekit.toml` diff.                                                               |
| Session leaks after worker crash (SIGKILL)     | Shutdown callback can't run on SIGKILL                                                                  | Set `max_session_duration` on `create_session_token` as a TTL safety net.                                                                                      |