# Create Image Avatar (Alpha) Source: https://docs.liveavatar.com/api-reference/avatar/create-image-avatar-alpha /openapi.json post /v1-alpha/avatars/image **Alpha:** this endpoint may change or be removed without notice. Create an image avatar from a single JPEG or PNG upload (multipart). The image must be exactly 16:9 or 9:16 with a short side of at least 1080px, and show one clearly framed person. A failing image returns 400 with the reason for each failed check. On success the avatar is returned in `INIT`; poll `GET /v1/avatars/{id}` until it is `ACTIVE`. 1080p training requires a Business or Enterprise plan. # Delete Avatar Source: https://docs.liveavatar.com/api-reference/avatar/delete-avatar /openapi.json delete /v1/avatars/{avatar_id} Soft delete avatar # Get Avatar By Id Source: https://docs.liveavatar.com/api-reference/avatar/get-avatar-by-id /openapi.json get /v1/avatars/{avatar_id} Get avatar by id # List Public Avatars Source: https://docs.liveavatar.com/api-reference/avatar/list-public-avatars /openapi.json get /v1/avatars/public Get public avatars list # List User Avatars Source: https://docs.liveavatar.com/api-reference/avatar/list-user-avatars /openapi.json get /v1/avatars Get avatars list for auth user # Update Avatar Source: https://docs.liveavatar.com/api-reference/avatar/update-avatar /openapi.json patch /v1/avatars/{avatar_id} Update avatar # Copy Context Source: https://docs.liveavatar.com/api-reference/contexts/copy-context /openapi.json post /v1/contexts/{context_id}/copy Hard-copy a readable context into the caller's space as a new PRIVATE context. Optional `name` overrides the copy's name; otherwise it defaults to "{original name} (Copy)". # Create Context Source: https://docs.liveavatar.com/api-reference/contexts/create-context /openapi.json post /v1/contexts Create a new user context record # Delete Context Source: https://docs.liveavatar.com/api-reference/contexts/delete-context /openapi.json delete /v1/contexts/{context_id} Delete a user context # Get Context By Id Source: https://docs.liveavatar.com/api-reference/contexts/get-context-by-id /openapi.json get /v1/contexts/{context_id} Get a user context by id # List User Contexts Source: https://docs.liveavatar.com/api-reference/contexts/list-user-contexts /openapi.json get /v1/contexts List user contexts # Update Context Source: https://docs.liveavatar.com/api-reference/contexts/update-context /openapi.json patch /v1/contexts/{context_id} Update a user context # Create Embed V2 Source: https://docs.liveavatar.com/api-reference/embeddings/create-embed-v2 /openapi.json post /v2/embeddings Create an embed avatar with sandbox support. # Get Languages Source: https://docs.liveavatar.com/api-reference/languages/get-languages /openapi.json get /v1/languages Return the list of supported languages with code and display name. # Create Llm Configuration Source: https://docs.liveavatar.com/api-reference/llm-configurations/create-llm-configuration /openapi.json post /v1/llm-configurations Create a new LLM configuration for the authenticated user's space. This endpoint: - Requires API key or session authentication - Validates that the secret_id belongs to the user's space - Creates a new LLM configuration with the provided settings Args: service: Injected LLM configuration service payload: Request containing url, display_name, model_name, and secret_id Returns: Response with created LLM configuration Raises: 400 Bad Request: If user doesn't have an associated space or secret is invalid # Delete Llm Configuration Source: https://docs.liveavatar.com/api-reference/llm-configurations/delete-llm-configuration /openapi.json delete /v1/llm-configurations/{config_id} Delete an LLM configuration by ID. This endpoint: - Requires API key or session authentication - Permanently deletes the LLM configuration Args: service: Injected LLM configuration service config_id: The ID of the LLM configuration to delete Returns: Response confirming deletion Raises: 400 Bad Request: If user doesn't have an associated space 404 Not Found: If the configuration doesn't exist # Get Llm Configuration Source: https://docs.liveavatar.com/api-reference/llm-configurations/get-llm-configuration /openapi.json get /v1/llm-configurations/{config_id} Get an LLM configuration by ID. This endpoint: - Requires API key or session authentication - Returns the full LLM configuration details Args: service: Injected LLM configuration service config_id: The ID of the LLM configuration to retrieve Returns: Response with the LLM configuration Raises: 400 Bad Request: If user doesn't have an associated space 404 Not Found: If the configuration doesn't exist # List Llm Configurations Source: https://docs.liveavatar.com/api-reference/llm-configurations/list-llm-configurations /openapi.json get /v1/llm-configurations List all LLM configurations for the authenticated user's space. This endpoint: - Requires API key or session authentication - Returns all LLM configurations for the user's space - Results are ordered by creation date (newest first) Args: service: Injected LLM configuration service Returns: Response with list of LLM configurations Raises: 400 Bad Request: If user doesn't have an associated space # Update Llm Configuration Source: https://docs.liveavatar.com/api-reference/llm-configurations/update-llm-configuration /openapi.json patch /v1/llm-configurations/{config_id} Update an LLM configuration by ID. This endpoint: - Requires API key or session authentication - Validates that the new secret_id (if provided) belongs to the user's space - Updates only the provided fields Args: service: Injected LLM configuration service config_id: The ID of the LLM configuration to update payload: Request containing fields to update Returns: Response with the updated LLM configuration Raises: 400 Bad Request: If user doesn't have an associated space or secret is invalid 404 Not Found: If the configuration doesn't exist # List Llm Models Source: https://docs.liveavatar.com/api-reference/llm-models/list-llm-models /openapi.json get /v1/llm-models List the managed LLM models selectable per provider. These are the `llm_settings` choices accepted on a `liveavatar_agent`'s configuration and on a session's `avatar_persona` — a static catalog, not space-scoped data. Custom LLM endpoints (`/v1/llm-configurations`) are a separate concept and are not listed here. # Create Memory Source: https://docs.liveavatar.com/api-reference/memory/create-memory /openapi.json post /v1/memory # Delete Memory Source: https://docs.liveavatar.com/api-reference/memory/delete-memory /openapi.json delete /v1/memory/{memory_id} # Get Memory Source: https://docs.liveavatar.com/api-reference/memory/get-memory /openapi.json get /v1/memory/{memory_id} # List Memories Source: https://docs.liveavatar.com/api-reference/memory/list-memories /openapi.json get /v1/memory # Create Secret Source: https://docs.liveavatar.com/api-reference/secrets/create-secret /openapi.json post /v1/secrets Create a new encrypted secret for the authenticated user's space. This endpoint: - Requires API key authentication - Encrypts the secret value using AWS KMS - Stores the encrypted value in the database - Returns metadata about the created secret (including id for future reference) Args: service: Injected secrets service payload: Request containing secret_name and secret_value Returns: Response with created secret metadata including id Raises: 400 Bad Request: If user doesn't have an associated space 500 Internal Server Error: If encryption or storage fails # Delete Secret Source: https://docs.liveavatar.com/api-reference/secrets/delete-secret /openapi.json delete /v1/secrets/{secret_id} Delete a secret by id. This endpoint: - Requires API key authentication - Permanently deletes the encrypted secret from the database Args: service: Injected secrets service secret_id: The id of the secret to delete Returns: Response confirming deletion Raises: 400 Bad Request: If user doesn't have an associated space 404 Not Found: If the secret doesn't exist # List Secrets Source: https://docs.liveavatar.com/api-reference/secrets/list-secrets /openapi.json get /v1/secrets List all secrets for the authenticated user's space. This endpoint: - Requires API key authentication - Returns metadata only (id, name, type) - never returns decrypted values - Results are ordered by creation date (newest first) Args: service: Injected secrets service Returns: Response with list of secret metadata Raises: 400 Bad Request: If user doesn't have an associated space # Create Session Token Source: https://docs.liveavatar.com/api-reference/sessions/create-session-token /openapi.json post /v1/sessions/token Create session token for an API-key-authenticated user. Minting a session token is only doable via an API key (``APIUserDep``) — a session JWT (``SessionUser``) cannot mint a token for another participant. HeyGen gateway identities mint through ``/heygen_session_token``. This route only authenticates native API keys. # Delete Session Events Source: https://docs.liveavatar.com/api-reference/sessions/delete-session-events /openapi.json delete /v1/sessions/{session_id}/events Hard delete all session events for a given session # Get Session Source: https://docs.liveavatar.com/api-reference/sessions/get-session /openapi.json get /v1/sessions/{session_id} Get a specific session by ID # Get Session Transcript Source: https://docs.liveavatar.com/api-reference/sessions/get-session-transcript /openapi.json get /v1/sessions/{session_id}/transcript Get the transcript for a session # Keep Session Alive Source: https://docs.liveavatar.com/api-reference/sessions/keep-session-alive /openapi.json post /v1/sessions/keep-alive Keep the session alive # List Sessions Source: https://docs.liveavatar.com/api-reference/sessions/list-sessions /openapi.json get /v1/sessions List active sessions for the authenticated user # Start Session Source: https://docs.liveavatar.com/api-reference/sessions/start-session /openapi.json post /v1/sessions/start Start a new session # Stop Session Source: https://docs.liveavatar.com/api-reference/sessions/stop-session /openapi.json post /v1/sessions/stop Stop a session # Get User Credits Source: https://docs.liveavatar.com/api-reference/users/get-user-credits /openapi.json get /v1/users/credits Get the current user's credit balance. # Delete Voice Agent Source: https://docs.liveavatar.com/api-reference/voice-agents/delete-voice-agent /openapi.json delete /v1/voice_agents/{agent_id} Delete a voice agent. Fails with `409` while any embed still references the agent; the referencing embed ids are returned in the response payload. Repoint or delete those embeds first, then retry. # Get Voice Agent Source: https://docs.liveavatar.com/api-reference/voice-agents/get-voice-agent /openapi.json get /v1/voice_agents/{agent_id} Retrieve a voice agent, including its full `agent_configuration`. # List Voice Agents Source: https://docs.liveavatar.com/api-reference/voice-agents/list-voice-agents /openapi.json get /v1/voice_agents List voice agents. `scope` selects which agents are returned: * `private` (default) — only the agents your space owns. * `public` — preset agents owned by no space. Read-only: they cannot be updated or deleted (PATCH/DELETE return `404`). Items omit `agent_configuration` — retrieve a single agent to see its full configuration. # Bind Third Party Voice Source: https://docs.liveavatar.com/api-reference/voices/bind-third-party-voice /openapi.json post /v1/voices/third_party Bind a third-party voice (e.g. ElevenLabs) using a stored secret. Validates the secret can be used to bind a third-party voice and creates a voice record. Returns a voice_id usable in sessions. # Delete Voice Source: https://docs.liveavatar.com/api-reference/voices/delete-voice /openapi.json delete /v1/voices/{voice_id} Delete voice by id # Get Voice By Id Source: https://docs.liveavatar.com/api-reference/voices/get-voice-by-id /openapi.json get /v1/voices/{voice_id} Get voice by id, with the TTS engines it can be voiced by. # Get Voice Preview By Id Source: https://docs.liveavatar.com/api-reference/voices/get-voice-preview-by-id /openapi.json get /v1/voices/{voice_id}/preview Get voice preview by id # List Voices Source: https://docs.liveavatar.com/api-reference/voices/list-voices /openapi.json get /v1/voices Get voices list # Update Voice Source: https://docs.liveavatar.com/api-reference/voices/update-voice /openapi.json patch /v1/voices/{voice_id} Update voice # Changelog Source: https://docs.liveavatar.com/changelog Product updates and announcements for LiveAvatar API **October 23, 2026** New Endpoint ## Create image avatars (alpha) * **`POST /v1-alpha/avatars/image`** creates an avatar from a single JPEG or PNG image, sent as `multipart/form-data` with a **`file`** field. The image must be exactly 16:9 or 9:16, have a short side of at least 1080px, and show one clearly framed person. Images that fail validation return 400 with the reason for each failed check. The endpoint is in alpha and may change or be removed without notice. * **`resolution`** sets the training resolution: **`720`** (default) or **`1080`**. 1080p training requires a Business or Enterprise plan. * New avatars are returned in **`INIT`** status. Poll **`GET /v1/avatars/{avatar_id}`** until the status is **`ACTIVE`**. * **Session `source`** on **`GET /v1/sessions`** and **`GET /v1/sessions/{session_id}`** adds the **`HEYGEN_API`** value for sessions started through the HeyGen API. **October 7, 2026** API Changes ## GPT-Live and OpenAI voice agents, new TTS models, and voice engine details * **GPT-Live in OpenAI Realtime mode.** **`openai_realtime_config.model`** is now an enum: **`gpt-realtime`** (default), **`gpt-realtime-mini`**, **`gpt-realtime-2.1`**, **`gpt-realtime-2.1-mini`**, and **`gpt-live-1`**. GPT-Live models run full-duplex and use their own voice set (**`aster`**, **`beacon`**, **`cinder`**, **`marin`**, **`stone`**, **`vesper`**; default **`marin`**). **`temperature`** applies to Realtime models only. * **`openai_agent` voice agent type.** **`agent_type`** adds **`openai_agent`**, which runs OpenAI Realtime or GPT-Live on your own OpenAI API key. Its **`agent_configuration`** takes **`model`**, **`voice`**, **`temperature`**, **`secret_id`** (required), and **`context_id`**, and is returned by **`GET /v1/voice_agents/{agent_id}`**. * **New TTS models.** * Cartesia: **`sonic-3.6`**, now the default. * ElevenLabs: **`eleven_v4_turbo`**. Like the v3 models, it honors only **`stability`** from the voice settings. * Fish Audio: **`s2-pro`** and **`s2.1-pro`**. **`s2.1-pro`** is the new default. **`latency_mode`** adds **`low`** for the lowest latency. * **`GET /v1/voices/{voice_id}`** now returns **`voice_engines`**, the TTS providers that can voice the voice (`elevenLabs`, `fish`, `cartesia`). * **`PATCH /v1/avatars/{avatar_id}`** accepts **`is_deactivated`** to deactivate or reactivate an avatar. **`name`** is limited to 64 characters. **September 23, 2026** Bug Fixes ## ElevenLabs EU data residency * Fixed an issue where ElevenLabs API keys from EU data residency workspaces were rejected by **`POST /v1/secrets`**. **September 17, 2026** API Changes ## ElevenLabs v3 models, new default LLM, and new avatar status values * **ElevenLabs v3 voice models.** **`voice_settings.model`** for the **`elevenlabs`** provider now accepts **`eleven_v3`** and **`eleven_v3_conversational`** alongside **`eleven_flash_v2_5`** (default) and **`eleven_multilingual_v2`**. The v3 models are the most expressive but honor only **`stability`** from the voice settings; `similarity_boost`, `style`, and `use_speaker_boost` are ignored. * **Default LLM is now `gpt-5.4-nano`.** **`llm_settings.model`** defaults to **`gpt-5.4-nano`** instead of `gpt-4o-mini`. **`llm_settings`** is no longer nullable on **`agent_configuration`**: agents stored without it resolve to the default model on read, so every agent now reports an explicit model. * **Avatar status** adds **`DEACTIVATED`** and **`DELETED`** values. **`is_expired`** is now read-only and always present. **September 16, 2026** SDK Update ## JavaScript SDK v0.0.19: automatic keep-alive and voice chat config * **[`@heygen/liveavatar-web-sdk` v0.0.19](https://github.com/heygen-com/liveavatar-web-sdk)** adds an **`autoKeepAlive`** session option. When `true`, the SDK sends a keep-alive request every 60 seconds while the session is connected and stops on disconnect. Defaults to `false`; without it, call **`keepAlive()`** yourself. * **`voiceChat`** now takes a **`VoiceChatConfig`** object only. The default `{}` starts the microphone unmuted; pass `{ defaultMuted: true }` for a text-only session. Passing `true` or `false` is deprecated, logs a console warning, and is mapped to the closest object form. The boolean shim is removed in 0.0.20. **September 8, 2026** API Changes ## Avatar Only (LITE Mode) WebSocket events The Avatar Only WebSocket protocol now uses the same event envelope as FULL Mode. See the updated [Events](/docs/lite-mode/events) reference. * **Event correlation.** Every server event now carries **`event_id`** and **`source_event_id`**. Set **`event_id`** on the first **`agent.speak`** chunk of an utterance and all events for that utterance (`agent.speak_started`, `agent.speak_ended`, `agent.speak_interrupted`, and buffer acks) return it as **`source_event_id`**. The unused **`task.id`** field has been removed. * **New server events.** **`agent.state_updated`** reports avatar state transitions (`idle`, `listening`, `talking`). Added **`agent.audio_buffer_appended`**, **`agent.audio_buffer_committed`**, and **`agent.audio_buffer_cleared`** acks, plus **`error`** and **`warning`** events. * **`session.state_updated`** states are now **`new`**, **`connected`**, **`disconnected`**. * **`agent.speak_end`** accepts an optional trailing **`audio`** chunk. **September 1, 2026** New Features ## Managed LLM selection, public voice agents, and context copying * **LLM model selection.** New **`llm_settings`** field on **`avatar_persona`** and on a **`liveavatar_agent`**'s **`agent_configuration`** selects the managed model (`provider` + `model`). Currently supports OpenAI **`gpt-4o-mini`** (default), **`gpt-4o`**, **`gpt-5-nano`**, and **`gpt-5.4-nano`**; omit to use the default. See [Configuring LLM Settings](/docs/full-mode/llm-settings). * New endpoint **`GET /v1/llm-models`** lists the managed models available per provider. * **Public voice agents.** All accounts now have access to a set of read-only preset voice agents. List them with **`GET /v1/voice_agents?scope=public`**; the default `scope=private` still returns only your own agents. Responses now include **`is_public`**. * **Copy contexts.** New endpoint **`POST /v1/contexts/{context_id}/copy`** hard-copies a readable context into your space as a new private context. Optional **`name`** in the body overrides the copy's name; default is `"{original name} (Copy)"`. * **Rate limits.** Public endpoints now enforce per-caller limits (10 requests/second per API key on most endpoints; 5 requests/second per session token on **`POST /v1/sessions/start`**, **`stop`**, and **`keep-alive`**). Over-limit requests return **`429`** with **`Retry-After`** and **`X-RateLimit-*`** headers. See [Rate Limits](/docs/core-concepts/rate-limits). **September 1, 2026** Bug Fixes ## Connector fixes * **ElevenLabs Agent connector.** Fixed an issue where the avatar kept speaking after the user interrupted. The connector now stops playback and flushes buffered audio as soon as ElevenLabs reports an interruption. * **Bring-your-own LiveKit.** Fixed an issue where sessions joining your own LiveKit room occasionally produced no avatar audio. The avatar now waits for its audio and video tracks to be fully bound before streaming, and retries the connection once if binding does not complete. **August 5, 2026** New Features ## Voice agent endpoints and Cartesia agents * **Voice agent endpoints.** You can now list, retrieve, and delete your stored voice agents: **`GET /v1/voice_agents`**, **`GET /v1/voice_agents/{agent_id}`**, and **`DELETE /v1/voice_agents/{agent_id}`**. * **Cartesia voice agents.** A voice agent can now be backed by a Cartesia-hosted agent, using the new **`cartesia_agent`** type. In LITE Mode, pass **`cartesia_agent_config`** on **`POST /v1/sessions/token`** with your **`agent_id`** and the **`secret_id`** of a stored **`CARTESIA_API_KEY`**. **July 22, 2026** New Features ## Cartesia TTS support * **Cartesia** is now available as a voice provider. The `voice_settings` object on **`avatar_persona`** accepts a new **`CartesiaVoiceSettings`** variant (discriminated by **`provider: "cartesia"`**), with **`speed`** (0.6–1.5, default `1.0`) and **`model`**. * New **`CartesiaModelEnum`**: **`sonic-3.5`** (default), **`sonic-3`**, **`sonic-2`**, **`sonic-turbo`**. * **`SecretTypeEnum`** — Added **`CARTESIA_API_KEY`** and **`FISH_API_KEY`** for bring-your-own-key configuration via **`POST /v1/secrets`**. * **`GET /v1/sessions`** — Session list entries (**`ListSessionEntrySchema`**, **`ListHistoricEntrySchema`**) now include **`api_key_id`**, identifying the API key that created the session. **July 22, 2026** Deprecation ## VP8 video encoding deprecated * **`VideoEncodingEnum`** — The **`VP8`** value is deprecated and will be removed in a future release. Use **`H264`** (the default). Sessions that still request `VP8` continue to work for now, but should migrate to `H264`. Affects the `encoding` field of `video_settings` on session token requests. **June 22, 2026** New Features ## Voice agents **Voice agents** are a new primitive for defining how your avatar converses. A voice agent stores the conversational configuration (voice, context, and language) as a single resource you reference by **`id`**, instead of passing each one on every request. See [Voice Agents](/docs/core-concepts/voice-agents) for the full concept. * **Two agent types.** A `liveavatar_agent` runs LiveAvatar's managed **FULL Mode** pipeline (composed from a [voice](/docs/core-concepts/voices) and a [context](/docs/core-concepts/contexts)). An `elevenlabs_agent` is backed by an ElevenLabs-hosted agent and produces a **LITE Mode** session — even when referenced from a FULL Mode token request. * **`POST /v1/sessions/token`** (FULL Mode) — New optional **`voice_agent`** object referencing a stored voice agent by **`id`**. Mutually exclusive with **`avatar_persona`**. * **Per-session overrides** for a `liveavatar_agent`: **`language`** and **`dynamic_variables`**, layered over the agent's stored config for that session only. Both are rejected with a `400` for an `elevenlabs_agent`. **`dynamic_variables`** is mutually exclusive with the top-level field, capped at 50 entries (keys ≤ 64 chars, string values ≤ 1000 chars). * **`POST /v2/embeddings`** — New optional **`voice_agent_id`** to resolve voice and context from a stored voice agent. Mutually exclusive with **`voice_id`** / **`context_id`**. * New schema: **`VoiceAgentRef`**. **June 1, 2026** New Features ## Attach memory when creating a session token * **`POST /v1/sessions/token`** — FULL Mode tokens now accept an optional **`memory`** object. Provide **`prev_session_id`** to seed or reuse memory from a prior session, **`session_memory_id`** to attach an existing memory, or both (they must resolve to the same memory). Sessions started without it are unaffected. **June 1, 2026** New Endpoint ## Session memory endpoints * **`POST /v1/memory`** — Create a memory from a completed session. Accepts **`memory_type`** (`session`) and **`session_id`**. * **`GET /v1/memory`** — List memories with **`type`**, **`page`**, and **`page_size`** query parameters. Each item returns a **`memory_content_preview`**. * **`GET /v1/memory/{memory_id}`** — Fetch a single memory, including its rolled-up **`memory_content`** and the per-session **`session_episodes`** that compose it. * **`DELETE /v1/memory/{memory_id}`** — Delete a memory and its episodes. * New schemas: **`MemoryResponse`**, **`MemoryWithEpisodesResponse`**, **`MemoryListItemResponse`**, **`MemoryEpisodeSchema`** (each episode carries a **`session_id`**, **`sequence_num`**, and **`summary`**), and **`MemoryTypeEnum`** (currently `session`). **May 20, 2026** API Changes ## ElevenLabs Japanese text normalization * **`ElevenLabsVoiceSettings`** — New optional **`apply_language_text_normalization`** boolean (default `false`). Enables ElevenLabs language-specific text normalization to improve pronunciation. Currently only Japanese (`ja`) is supported; valid only when the session language is set to `ja`. Adds latency, so it is opt-in. **May 13, 2026** New Endpoint ## Delete session events endpoint * **`DELETE /v1/sessions/{session_id}/events`** — New endpoint to hard-delete all events for a given session. Accepts `session_id` path parameter; authenticated via API key or bearer token. **May 7, 2026** SDK Update ## JavaScript SDK v0.0.18: ElevenLabs Agent improvements * **[`@heygen/liveavatar-web-sdk` v0.0.18](https://github.com/heygen-com/liveavatar-web-sdk)** — Includes a set of ElevenLabs Agent connector improvements, notably bidirectional passthrough of raw ElevenLabs Conversational AI events. Frontends can now both receive every ElevenLabs server event (via [`elevenlabs_agent_event`](/docs/lite-mode/connectors/elevenlabs-agent#listening-for-elevenlabs-events)) and send raw ElevenLabs client events (via [`elevenlabs_agent_command`](/docs/lite-mode/connectors/elevenlabs-agent#sending-elevenlabs-commands)) without per-feature plumbing. **May 4, 2026** API Changes ## 1080p avatars, ElevenLabs voice override, and more Gemini voices * **1080p avatars** — Added support for 1080p avatars on the new payment plan. **`AvatarSchema`** exposes an **`is_1080p`** boolean to identify them. * **Smaller session tokens** — The session token JWT no longer includes sensitive fields, which also reduces its size. Sandbox Mode tokens keep these fields for testing. * **ElevenLabs voice override** — **`ElevenLabsAgentConfigSchema`** accepts an optional **`voice_id`** that is forwarded as `conversation_config_override.tts.voice_id` to the ElevenLabs Agent WebSocket, overriding the agent's default voice for the session. * **Expanded Gemini Realtime voices** — **`GeminiVoiceEnum`** adds Achernar, Achird, Algenib, Algieba, Alnilam, Autonoe, Callirrhoe, Despina, Enceladus, Erinome, Gacrux, Iapetus, Laomedeia, Pulcherrima, Rasalgethi, Sadachbia, Sadaltager, Schedar, Sulafat, Umbriel, Vindemiatrix, and Zubenelgenubi. **April 22, 2026** New Features ## Gemini Realtime, vertical embeds, and dynamic variables * **Gemini Realtime agent** — LITE sessions can now be configured with **`gemini_realtime_config`**, running on Gemini Live API with configurable model, voice, and temperature. Supported voices include **Aoede**, **Charon**, **Fenrir**, **Kore**, **Leda**, **Orus**, **Puck**, and **Zephyr**. Requires a new **`GEMINI_API_KEY`** secret type. * **Embed orientation** — `/v2/embeddings` now accepts an **`orientation`** field (`horizontal` for 16:9 or `vertical` for 9:16). The response also returns the new **`embed_id`** field. * **Session dynamic variables** — Session creation now accepts a **`dynamic_variables`** object for filling `${var}` placeholders in the context's opening text and prompt (max 50 entries). * **New session filters** — `GET /v1/sessions` now supports **`embed_id`** and **`context_id`** query parameters. * **New `AGENT_HANG_UP` session end reason** — Added to **`SessionEndReasonEnum`** to indicate the agent itself terminated the call. Currently only emitted by the ElevenLabs agent. **April 9, 2026** New Features ## ElevenLabs events, embed transcripts, and SDK update * **ElevenLabs agent events** — underlying ElevenLabs server events are now forwarded via the [`elevenlabs_agent_event`](/docs/lite-mode/connectors/elevenlabs-agent#events) event type, giving you access to raw ElevenLabs Conversational AI events * **Transcript on embed** — Transcripts are now available directly on embed sessions * **JavaScript SDK updated** — Updated the [`@heygen/liveavatar-web-sdk`](https://github.com/heygen-com/liveavatar-web-sdk) package with new event types and improvements **April 6, 2026** API Changes ## Authentication docs and session filtering * **Authentication** — Each endpoint now documents which authentication it accepts (API key or session token) * **New `avatar_id` filter on `/v1/sessions`** — List sessions can now be filtered by an optional **avatar\_id** query parameter **April 2, 2026** New Features ## Embeds v2 and image avatars * **`/v2/embeddings`** — New endpoint for creating embeds, with Sandbox Mode support * **Image avatars** — Create an animated avatar from a single photo using the new validate and submit endpoints ### Improvements * [**Update avatar endpoint**](/api-reference/avatar/update-avatar) — You can now change an avatar's default voice * **Independent avatar and voice deletion** — Deleting a voice no longer affects its associated avatar, and vice versa * **ElevenLabs agent dynamic variables** — Configure dynamic variables for ElevenLabs-powered agents that get replaced at session start * **Source events fixed in FULL Mode** — `avatar.say` and `avatar.repeat` events now correctly return `source_event_id` **March 26, 2026** New Features ## ElevenLabs Scribe and embed settings * Added support for **ElevenLabs Scribe**, expanding ASR capabilities * **Embed Improvements** * Default language selection, letting users set the language for their embed experience * Time limits, giving creators control over session duration **March 18, 2026** API Changes ## Voice settings and Fish Audio * Added additional voice settings for more granular control over avatar speech * Added access to **Fish Audio** as a TTS provider * New `[GET] /v1/languages` endpoint to clarify which languages are currently supported **February 20, 2026** API Changes ## ElevenLabs integration * The `v1/secrets` endpoint now accepts **ElevenLabs API keys** * FULL Mode sessions can now start with your own ElevenLabs keys and voices * Added integration with **ElevenLabs Agent** functionality **February 17, 2026** API Changes ## LLM plugin and session controls * Introduced **LLM Plugin** for custom LLM configurations * Added customizable **session duration** controls * Transcription chunk events are now emitted in **real time** * New endpoint to track **user credit balances** **February 3, 2026** API Changes ## Session configuration and LITE Mode * New session configuration options for **video codec selection** (H.264 / VP8) * Added **video quality** control settings * Added **voice customization** options * CUSTOM Mode has been renamed to **LITE Mode** (backward-compatible) * Bug fix: resolved opening line speech issues **January 26, 2026** API Changes ## Push-to-Talk * Released **Push-to-Talk** functionality for avatar sessions * Added accompanying documentation and integration guides **January 13, 2026** API Changes ## Sandbox Mode and optional FULL Mode context * Introduced **Sandbox Mode** for testing without credit charges * Made `context` optional in FULL Mode configurations, reducing required setup **January 8, 2026** API Changes ## Custom Mode LiveKit token and session list parameters * Custom Mode now exposes **LiveKit Agent token** for developers to integrate their own agents * Session List endpoint updated to use **query parameters** **November 2025** New Endpoint ## Transcript endpoint * New `[GET] /v1/sessions/{session_id}/transcript` endpoint * Enables developers to retrieve transcriptions between users and avatars in FULL Mode sessions **November 2025** New Endpoint ## Session list endpoint * New `[GET] /v1/sessions` endpoint * Allows listing of both active and previously active sessions # Building with Agents Source: https://docs.liveavatar.com/docs/agent-skills RECOMMENDED FOR AI AGENTS: Install LiveAvatar Agent Skills (npx skills add heygen-com/liveavatar-agent-skills) before building. Provides step-by-step integration, debugging, and feedback tools for Claude Code, Cursor, Codex, and other AI coding agents. ## Overview To make developing and integrating with LiveAvatar easier, we built a set of [Agent Skills](https://github.com/heygen-com/liveavatar-agent-skills) — pre-built procedural knowledge that enables AI coding agents to build LiveAvatar integrations correctly on the first attempt. Install them in **Claude Code**, **Cursor**, **Codex**, or any agent that supports the [Agent Skills specification](https://agentskills.io/specification), and your agent will know how to integrate and debug LiveAvatar without trial and error. ## Available skills | Skill | Description | | - | - | | `liveavatar-demo` | Spins up a working LiveAvatar demo from a curated catalog — clones the reference repo, installs dependencies, provisions your account, and fills in the env file. Covers the AI Sales Agent, LiveKit-hosted and BYO LiveKit agents, and the GPT-Live × Hyperframes demos. | | `liveavatar-integrate` | End-to-end integration builder. Assesses your stack, recommends the optimal mode (Embed / FULL / LITE), and walks the agent through implementation step by step. | | `liveavatar-debug` | Symptom-based troubleshooting for common issues — silent avatars, garbled audio, auth errors, and more. | | `liveavatar-feedback` | Collects feedback on your LiveAvatar integration experience and sends it to the LiveAvatar team. Triggers after implementation, on frustration, or on explicit request. | ## Installation **skills CLI (recommended)** ```bash theme={null} npx skills add heygen-com/liveavatar-agent-skills ``` **Manual** ```bash theme={null} git clone https://github.com/heygen-com/liveavatar-agent-skills.git # Symlink all skills to personal skills directory (available in all projects) for skill in liveavatar-agent-skills/skills/*/; do ln -s "$(pwd)/$skill" ~/.claude/skills/$(basename "$skill") done ``` ## How it works Each skill follows a consistent structure: | File | Purpose | | - | - | | `SKILL.md` | Agent instructions and behavioral guidance | | `references/` | Supporting docs — API details, code examples, troubleshooting trees | When your agent loads a skill, it gains the procedural knowledge to execute the task end-to-end. The `liveavatar-integrate` skill, for example, will assess your existing stack, recommend FULL vs LITE vs Embed Mode, and generate working code — backend and frontend — with correct auth patterns. > [!TIP] > > Skills are designed around a "pit of success" philosophy: the correct > implementation is the easiest path. They identify failure points upfront and > clearly separate backend (API key) from frontend (session token) code. # Avatars Source: https://docs.liveavatar.com/docs/core-concepts/avatars The visual identity of your LiveAvatar An avatar is the visual representation of your AI agent — the face your users see and interact with. LiveAvatar renders a real-time, lip-synced video stream of a human-like character that speaks, reacts, and maintains natural eye contact. Facial expressions, head movement, and overall appearance stay consistent across every interaction, so users always recognize the same character. You can browse and preview all available avatars on the [LiveAvatar dashboard](https://app.liveavatar.com). ## Stock avatars LiveAvatar comes with a library of stock avatars available to all users. They cover a range of appearances, styles, and demographics — perfect for any project or use case. ### Green-screen studio avatars A subset of the stock library consists of **studio avatars** built for background replacement: their video renders against a solid green background so you can remove it client-side and place the avatar over your own UI, a branded backdrop, or ambient footage. See [Change the avatar background](/docs/guides/change-background) for the chroma-key technique. You can recognize them in the catalog by their preview images — the `preview_url` on these avatars is a transparent (alpha-channel) cutout of the person rather than a full scene. > [!NOTE] > > There is no dedicated field on the avatar object that marks an avatar as a > studio avatar. To detect one programmatically, check whether its > `preview_url` image contains an alpha channel. Each studio look is available in landscape and portrait, with a separate `avatar_id` for each orientation: | Look | Landscape `avatar_id` | Portrait `avatar_id` | | - | - | - | | Alessandra in Black Suit | `9c59a215-4c9f-478f-9d95-edca74c7b0d0` | `de82b2cc-7314-40a1-b41f-cc456c66601c` | | Alessandra in Grey Sweater | `ab0765ad-69de-41fb-9f8a-bd01c3c52d6f` | `63014563-22f7-4401-95db-034f3c992ec3` | | Amina in Black Suit | `42700a53-38ab-4485-b46f-26be6e0953dc` | `06bcaebf-feeb-4e7a-8ab8-16b9f9537a19` | | Amina in Blue Suit | `bfed3e3e-7d44-4fdb-b2be-ce9a9fd0b9b5` | `334c4101-4c2d-4aa4-9c4e-342867842608` | | Anastasia in Black Suit | `ebdfdc7e-7e2c-4d2c-8407-a78883e5000a` | `649e4023-7e82-4a01-af90-4476d75df75d` | | Anastasia in Grey Shirt | `b4fc2d60-3b82-4694-b243-93e9d2bb0242` | `65ee9a5b-00ae-4c96-acf2-3326d9566467` | | Anthony in Black Suit | `38ad67ed-98f0-407c-a2d2-4f0998b306fc` | `3466af9c-45f5-40f2-8392-c51f8385422c` | | Anthony in White Suit | `509609b9-cda3-4f74-b1b2-97b4d98834fd` | `b26c00a5-8861-4fd3-90cc-144791d2d81d` | | Graham in Black Shirt | `bb1f6ebc-b388-4a39-9e2b-8df618e0377c` | `f0a9978d-634a-4063-8164-42ba6670542f` | | Graham in Black Suit | `03f8332d-9046-42a1-bff3-3b2309f77b58` | `fc4125a5-83fa-45e2-8574-bf657ac19998` | | Katya in Black Suit | `26393b8e-e944-4367-98ef-e2bc75c4b792` | `260d706a-bf31-48fd-96d0-511553a9060e` | | Katya in Pink Suit | `09919247-f4b2-45d8-a75e-86fc2fceaebf` | `69cf601f-b35b-4d1b-a701-c854d223b5a5` | | Marianne in Black Suit | `f86e8b45-3389-424a-b3d7-7f6e8729e36d` | `f0953388-a391-4518-945b-f2db3b602e77` | | Marianne in Red Suit | `8532b602-89e8-44fa-a9e2-5a4259a058cc` | `9d4d6cdd-38a1-448e-a336-1ac682ba6a27` | | Pedro in Black Suit | `200eba85-74c0-4210-8670-81ceab4efd0d` | `90a94dae-8df2-4b12-99a8-5bbc34c9551e` | | Pedro in Blue Shirt | `7001c332-8101-4e5a-b695-eac2a72d9568` | `9a5a4cb2-a32e-415f-8689-aa9d61acee1b` | | Rika in Black Suit | `075abc67-2fae-4548-8ca9-b815fcbd34c7` | `de16689a-234a-4334-b504-3b379d30daa1` | | Rika in Blue Suit | `0aae6046-0ab9-44fe-a08d-c5ac3f406d34` | `5c6e2257-bb91-48f4-a75d-d4c9ad212af9` | | Thaddeus in Black Shirt | `16141106-96b5-4dd9-9846-593728c5d0ed` | `37521110-f8cf-4c34-b37e-bbb4f670711e` | | Thaddeus in Black Suit | `246e8d9d-5826-4f49-b8a0-07cb73ff7556` | `6a1906e3-6caa-4aba-995f-c49088661560` | The remaining stock avatars (for example *Ann Therapist*, *Judy Doctor Sitting*, *Santa Fireplace Front*) render a complete scene with a real background and are meant to be shown as-is. ## Custom avatars If you need a specific visual identity — a branded spokesperson, a recognizable instructor, or a character unique to your product — you can create a custom avatar. There are two ways to create one: from video footage or from a single image. Custom avatar creation is available as a paid subscription add-on and requires an available **avatar slot** on your plan (see [Avatar slots](#avatar-slots) below). ### Video avatars Video avatars are created from recorded footage. The process requires: 1. **Two minutes of input footage** — LiveAvatar trains on this video to learn the subject's likeness, expressions, and mannerisms. 2. **A consent recording** — a short video documenting the subject's permission to create an avatar using their likeness. Because the training footage includes speech, LiveAvatar automatically generates a **voice clone** from the video. This gives your avatar a voice that naturally matches its appearance — no extra setup needed. You can still override this voice with any other available voice at session time. #### Tips for getting a high-quality video avatar The quality of your input footage directly affects the quality of the resulting avatar. Record a 2-minute video following this structure: 1. **15 seconds of listening** — Smile and nod to the camera. Act as if someone is speaking to you. 2. **90 seconds of speaking** — Talk about anything to the camera. Have a natural conversation, smile, and be yourself. 3. **15 seconds of idling** — Look at the camera, breathe and smile, with no obvious movement. A few additional guidelines: * Record in a well-lit, quiet environment. * Pause between each sentence with your mouth closed. * Keep your body position stable with minimal movement or gestures. ### Image avatars Image avatars let you create a custom avatar from a **photo** instead of video footage. Upload an image of a person and LiveAvatar will generate an avatar based on their likeness. This is a faster, lighter alternative to video avatars — ideal when you don't have access to recorded footage or need to spin up a custom avatar quickly. #### Key differences from video avatars | | Video avatar | Image avatar | | - | - | - | | **Input** | 2-minute video recording | Single image | | **Consent recording** | Required | Not required | | **Voice clone** | Automatically generated from footage | Not generated — you must specify a voice when starting a session | | **Expressions & mannerisms** | Learned from footage | Generated from the image | > [!WARNING] > > Image avatars do not include a voice. You must supply a voice when starting a > session — either by selecting one from the voice library, using a custom TTS > integration, or bringing your own audio pipeline in Avatar Only (LITE) mode. See > [Voices](/docs/core-concepts/voices) for available options. #### Tips for getting a high-quality image avatar * Use a **high-resolution, well-lit photo** with the subject facing the camera. * Ensure the face is clearly visible and not obscured by accessories or shadows. * Use a neutral or plain background for best results. ## Avatar slots Custom avatars each require an avatar slot. You can purchase slots as add-ons to your subscription plan. When all your slots are in use, you'll need to either remove an existing custom avatar to free up a slot or purchase additional ones. > [!NOTE] > > Stock avatars do not require avatar slots. Slots only apply to custom avatars > (both video and image). # Contexts Source: https://docs.liveavatar.com/docs/core-concepts/contexts The knowledge and personality behind your LiveAvatar A **context** defines what your avatar knows and how it behaves — its personality, knowledge, guardrails, and conversational style. Together with a [voice](/docs/core-concepts/voices), it is one of the building blocks of a [voice agent](/docs/core-concepts/voice-agents). > [!NOTE] > > Contexts are used in **FULL Mode** only — outside it, you bring your own AI stack, so personality and knowledge are managed on your side. ## What goes into a context ### Prompt The system-level instructions that drive behavior. This is where you define the avatar's role, tone, knowledge boundaries, and rules. Think of it like a system prompt for an LLM — it tells the avatar who it is, what it should talk about, and how it should respond. ### Opening text An optional greeting the avatar speaks when a session starts. Useful for cases where you want the avatar to introduce itself or set expectations upfront — for example, a support agent opening with "Hi there, I can help you with billing, account setup, or troubleshooting. What do you need?" ### Links Optional URLs with FAQ-style content the avatar can reference during conversations. These let you ground responses in specific documentation or knowledge base articles without stuffing everything into the prompt. Each link includes a URL and a short description of the content found there. ## Why contexts matter > [!WARNING] > > If no context is supplied when creating a session, the avatar enters **restricted mode**. It will not respond to user input — transcripts are still emitted, but the avatar can only repeat pre-set phrases. The context is what makes the avatar conversational. Without a context, your avatar is essentially a talking head with nothing to say. The context is the piece that turns a visual and audio experience into an interactive conversation. ## Reusable and composable Contexts are independent resources. They are not tied to a specific avatar or voice — you can reuse the same context across different avatar and voice combinations. Build one great support persona and pair it with different faces and voices for different audiences or markets. Swap out the visual layer for localization without rewriting a single line of your prompt. ### Dynamic variables If you want to make a shared context more personalizable, you can inject per-user information at session start using dynamic variables — rather than authoring a new context for every user. Reference a variable in your prompt or opening text using `${variable_name}` syntax, then supply the values in the `dynamic_variables` object when creating a session. ```json theme={null} { "dynamic_variables": { "user_name": "Jordan", "plan": "Pro" } } ``` The variable names the context expects are surfaced on the context resource as `required_dynamic_variables`. Extra keys are ignored; up to 50 entries, with keys ≤ 64 chars and values ≤ 1000 chars. ## Tips for writing good prompts The prompt field is the single most important factor in shaping your avatar's behavior. A few guidelines: * **Be specific about the role.** "You are a billing support agent for Acme Corp" is far better than "You are a helpful assistant." Give the avatar a clear identity. * **Set boundaries.** Define what topics the avatar should cover and what it should decline to answer. Explicit boundaries prevent off-topic drift. * **Define the tone.** Specify whether responses should be formal, casual, technical, or friendly. The avatar will follow your lead. * **Include guardrails.** Add explicit constraints like "never share personal medical advice" or "always recommend the user contact support for account-level changes." These protect both you and your users. # Memory Source: https://docs.liveavatar.com/docs/core-concepts/memory Continuity across your sessions **Memory** lets you carry over previous conversations into new ones. Without it, every session starts from a blank slate — the avatar has no idea it has spoken with this person before. Memory is the layer that connects sessions so the avatar can pick up where it left off. > [!NOTE] > > Memory is used in **FULL Mode** only — outside it, you bring your own AI stack, so conversation history and recall are managed on your side. ## Why memory exists By default, sessions are isolated. Each one runs independently, knows nothing about the sessions before it, and forgets everything when it ends. That works for one-off interactions, but it breaks down the moment a conversation is meant to continue over time: * A coaching or therapy avatar that should recall what was discussed last week. * A tutor that remembers which lesson a student is on and what they struggled with. * A support agent that knows the customer's history instead of asking them to repeat it. Memory closes that gap. It captures the key information from past sessions and adds it to the context of your current one. ## How memory works Memory follows a simple lifecycle across the life of a session: **1. Session transcripts are recorded** As part of FULL Mode, both user and avatar responses are recorded throughout the session. **2. Memory is created** Using the recorded transcript, the key takeaways from the conversation are distilled into a memory. There are two ways to create one: * **Bootstrapped from a prior session** — pass `prev_session_id` when creating a session token. If that session has no memory yet, one is created from its transcript as the new session starts. The prior session must be a completed FULL Mode session with a transcript — bootstrapping from an empty or silent session returns an error. * **Created directly** — call `POST /v1/memory` with a `session_id` ahead of time, then attach it to a later session with `session_memory_id`. > [!WARNING] > > Bootstrapping creates the memory on the spot, so the first session that triggers it sees added latency at startup while the transcript is distilled. Creating the memory ahead of time with `POST /v1/memory` avoids this. **3. Injected at the start of the next session** When a new session starts, the memory is injected as additional context. **4. Updated with each new session** When a session tied to a memory ends, that session's transcript is folded back into the memory automatically. It grows richer with each successive conversation, with no extra calls on your part. Once a memory exists, starting a session with it doesn't slow things down — the avatar reads the prepared summary instead of re-processing past conversations. ## Linking sessions together Memory is opt-in. A standalone session never creates or uses memory, so existing integrations are unaffected. To give sessions continuity, you link them to a shared memory when creating a session token. The first linked session establishes the memory; each session after that adds to it and reads the accumulated summary. Conceptually, a memory is an ongoing thread, and the sessions tied to it are the conversations that make up that thread. > [!NOTE] > > Memory is scoped to the sessions you explicitly link, not inferred automatically. You decide which conversations belong to the same thread. ## How to use it Memory is driven by two API surfaces: * **[Create a session token](/api-reference/sessions/create-session-token)** — pass a `memory` object with `prev_session_id` or `session_memory_id` to seed or attach memory for the session. * **[Memory endpoints](/api-reference/memory/create-memory)** — create, list, fetch, and delete memories directly. ## What memory is not Memory in its current form is intentionally focused: * It injects a single rolled-up summary, not a searchable archive of every past detail. * It is readable, so you can inspect what an avatar remembers and surface it in your own UI. * It is not editable — you can read and remove memory, but not hand-author its contents. These boundaries keep the behavior predictable while leaving room to grow. # Rate Limits Source: https://docs.liveavatar.com/docs/core-concepts/rate-limits Per-caller request limits on the LiveAvatar API The LiveAvatar API enforces per-caller rate limits on every public endpoint. A caller that exceeds a limit receives a `429 Too Many Requests` response and should retry after the interval given in the `Retry-After` header. ## How limits are counted Limits are counted per **caller** and per **endpoint**, over a fixed one-second window. * **Per caller.** The caller is identified by the credential on the request: the `X-API-KEY` header for backend calls, or the `Bearer` session token for session-scoped calls. Unauthenticated requests are counted by client IP. * **Per endpoint.** Each endpoint has its own counter. Requests to `GET /v1/avatars` do not consume budget for `GET /v1/voices`. Path parameters are ignored, so `GET /v1/sessions/{session_id}` shares one counter across all session IDs. * **Fixed window.** The counter resets at the start of each one-second window. The `X-RateLimit-Reset` header reports the seconds remaining in the current window. Because the identity is the API key rather than the account, two API keys in the same space have independent budgets. ## Limits by endpoint | Endpoints | Limit | Counted per | | - | - | - | | `POST /v1/sessions/token` | 10 requests / second | API key | | `POST /v1/sessions/start`, `POST /v1/sessions/stop`, `POST /v1/sessions/keep-alive` | 5 requests / second | Session token | | All other public endpoints | 10 requests / second | API key | Session lifecycle calls are counted per session token, so each active session has its own budget. A backend that manages many concurrent sessions is not throttled by their combined keep-alive traffic. ## Response headers Every rate-limited endpoint returns these headers on both successful and rejected responses: | Header | Description | | - | - | | `X-RateLimit-Limit` | Maximum requests allowed in the current window | | `X-RateLimit-Remaining` | Requests remaining in the current window | | `X-RateLimit-Reset` | Seconds until the current window resets | A rejected request also includes: | Header | Description | | - | - | | `Retry-After` | Seconds to wait before retrying. Equal to `X-RateLimit-Reset`. | ## Rejected requests An over-limit request returns HTTP `429` with the standard error envelope: ```json theme={null} { "code": 4290, "data": null, "message": "Rate limit exceeded" } ``` The request is rejected before any processing, so a rejected call has no side effects. ## Handling limits * **Honor `Retry-After`.** Windows are one second, so the wait is never longer than that. Retry after the given interval rather than immediately. * **Watch `X-RateLimit-Remaining`.** When it reaches zero, pause until the window resets instead of sending a request that will be rejected. * **Spread bursts.** Limits cap requests per second, not per minute. Ten calls in one second are rejected; ten calls spread across two seconds are not. * **Reuse session tokens.** Minting a new token for every operation consumes the `POST /v1/sessions/token` budget. Mint one token per session and reuse it for lifecycle calls. ## Availability Rate limiting is designed to fail open. If the limiter itself is unavailable, requests are allowed through rather than rejected, and no `5xx` is returned. Rate limit headers are still present on the response. # Secrets & Integrations Source: https://docs.liveavatar.com/docs/core-concepts/secrets Connect third-party providers and securely manage API keys LiveAvatar is designed to work with the providers you already use. Whether you want to bring your own language model, use a custom voice from an external TTS provider, or connect a real-time conversational AI platform, integrations let you extend LiveAvatar beyond its built-in capabilities. To make this work securely, LiveAvatar uses **secrets** — encrypted stores for your third-party API keys. You store a key once and reference it by ID wherever it is needed. All secret values are encrypted with AWS KMS, and the API never exposes decrypted values. ## What you can integrate LiveAvatar currently supports integrations with: * **LLM providers** — Bring your own language model by connecting any OpenAI-compatible endpoint. Use your preferred model for reasoning and response generation instead of the platform default. See [Custom LLM Integration](/docs/full-mode/custom-llm) for details. * **TTS providers** — Import voices from third-party text-to-speech providers and use them in your sessions. See [Custom TTS Integration](/docs/full-mode/custom-tts) for details. * **Conversational AI platforms** — Connect platforms like ElevenLabs Agents, Cartesia Agents, or OpenAI Realtime to power sessions with your own AI stack. As LiveAvatar adds support for more providers, additional integration types will follow the same pattern. ## How secrets work When you store a secret, LiveAvatar encrypts it using enterprise-grade AWS KMS encryption. The API never returns the actual key value — only metadata like the secret's name, type, and creation date. When a LiveAvatar session starts, the platform decrypts the relevant secrets and uses them to call your third-party providers on your behalf. Your credentials are never exposed to end users or returned through the API. ```mermaid theme={null} sequenceDiagram participant Dev as Developer participant LA as LiveAvatar participant KMS as AWS KMS participant Provider as Third-Party Provider Dev->>LA: Store API key as a secret LA->>KMS: Encrypt secret KMS-->>LA: Encrypted secret stored Dev->>LA: Start session (with secret reference) LA->>KMS: Decrypt secret KMS-->>LA: Decrypted credentials LA->>Provider: Call provider on your behalf Provider-->>LA: Response ``` A single secret can be referenced by multiple resources. For example, one API key could back several different LLM configurations, each with its own model and system prompt. ## When you need secrets > [!TIP] > > You only need secrets if you are connecting third-party providers. If you are using LiveAvatar's built-in voices and default LLM, you do not need to create any secrets. ## Key rotation > [!WARNING] > > If you delete a secret, any resources that depend on it — imported voices, LLM configurations, Avatar Only integrations — will stop working. To rotate a key safely, create a new secret with the updated credentials, update your resources to reference the new secret ID, and only then delete the old one. # Voice Agents Source: https://docs.liveavatar.com/docs/core-concepts/voice-agents The reusable primitive that defines your avatar's persona A **voice agent** is a stored, reusable definition of your avatar's persona — how it sounds and how it thinks. It bundles that persona into a single resource you reference by `id` when starting a session — instead of wiring up voice, context, and language on every request. Define your avatar's persona once, then reuse it across sessions and embeds. The same agent can be reused with different per-session values each time. Build and manage your voice agents in the dashboard at [app.liveavatar.com/voice-agent](https://app.liveavatar.com/voice-agent). Each agent has an `id` you reference when starting a session. ## Agent types A voice agent has a stored `agent_type` that determines how the session is built and which configuration applies. The server resolves the type from the agent `id` — you do not pass it at session time. ### `liveavatar_agent` Backed by LiveAvatar's managed conversational pipeline (STT → LLM → TTS). This is a **FULL Mode** agent: LiveAvatar runs the entire real-time stack. A `liveavatar_agent` builds its persona from the two building blocks documented in this section: * A **[voice](/docs/core-concepts/voices)** — how the avatar sounds. * A **[context](/docs/core-concepts/contexts)** — what the avatar knows and how it behaves. See [FULL Mode Overview](/docs/full-mode/overview) for the pipeline details. ### `elevenlabs_agent` Backed by an ElevenLabs-hosted agent, which handles the conversational AI while LiveAvatar renders the avatar video. An `elevenlabs_agent` produces an **Avatar Only (LITE mode)** session even when referenced from a FULL Mode token request. See the [ElevenLabs Agent Connector](/docs/lite-mode/connectors/elevenlabs-agent) for setup. ### `cartesia_agent` Backed by a Cartesia-hosted agent, which handles the conversational AI while LiveAvatar renders the avatar video. Like an `elevenlabs_agent`, it produces an **Avatar Only (LITE mode)** session even when referenced from a FULL Mode token request. A Cartesia agent owns its voice and prompt on Cartesia's side, so its stored configuration is just two fields: the `agent_id` of your Cartesia agent, and the `secret_id` of a stored [`CARTESIA_API_KEY` secret](/docs/core-concepts/secrets). See the [Cartesia Agent Connector](/docs/lite-mode/connectors/cartesia-agent) for setup. ## Referencing a voice agent Pass the stored agent `id` where the request accepts one: * **Session tokens** — supply a `voice_agent` object on a FULL Mode `POST /v1/sessions/token` request. Mutually exclusive with `avatar_persona`. * **Embeds** — supply `voice_agent_id` on `POST /v2/embeddings`. Mutually exclusive with `voice_id` / `context_id`. ```json theme={null} { "mode": "FULL", "avatar_id": "", "voice_agent": { "id": "" } } ``` The server resolves the agent's type and builds the session accordingly. ## Per-session overrides A voice agent stores defaults, but you can layer per-session values over them at creation time. Overrides apply to that session only — they do not modify the stored agent. Which overrides are accepted depends on the resolved `agent_type`; an override that does not apply is **rejected with a `400`** rather than silently ignored. | Override | `liveavatar_agent` | `elevenlabs_agent` | `cartesia_agent` | | - | - | - | - | | `language` | Layered over the agent's stored language | Rejected (`400`) | Rejected (`400`) | | `dynamic_variables` | Substituted into the context's `${var}` placeholders | Rejected (`400`) | Rejected (`400`) | A provider-hosted agent (`elevenlabs_agent`, `cartesia_agent`) is used as-is — configure its language and prompt variables on the provider side. ```json theme={null} { "mode": "FULL", "avatar_id": "", "voice_agent": { "id": "", "language": "en", "dynamic_variables": { "user_name": "Jordan", "plan": "Pro" } } } ``` > [!NOTE] > > `dynamic_variables` here is mutually exclusive with the top-level `dynamic_variables` (`400` if both are set). Values must be strings. At most 50 entries, with keys ≤ 64 chars and values ≤ 1000 chars. # Voices Source: https://docs.liveavatar.com/docs/core-concepts/voices The audio identity of your LiveAvatar A voice defines how your avatar sounds. LiveAvatar pairs real-time lip-synced video with natural-sounding speech, and the voice you choose shapes the entire audio experience — tone, accent, pacing, and personality. Together with a [context](/docs/core-concepts/contexts), it is one of the building blocks of a [voice agent](/docs/core-concepts/voice-agents). > [!NOTE] > > Voices are used in **FULL Mode** only — outside it, you bring your own audio pipeline, so voice selection and configuration are handled on your side. ## What voices are available ### Voice library LiveAvatar includes a library of voices spanning different languages, genders, and speaking styles. You can preview any voice before committing to it, making it easy to audition options and find the right fit. ### Avatar-generated voices When you create a custom avatar from video footage, LiveAvatar also generates a voice from that footage. This gives your avatar a voice that naturally matches its appearance — no extra setup needed. ### Bring your own voice In addition to the above, you can import voices from third-party TTS providers. This opens the door to custom-cloned voices, branded voices, or any voice available in a provider's catalog. See [Custom TTS Integration](/docs/full-mode/custom-tts) for a step-by-step guide on adding your own voice. ## How to use a voice in a LiveAvatar session LiveAvatar gives you control over how your avatar sounds when starting a session. Every avatar comes with a default voice, but you can override it with any available voice. Once you have selected a voice, you can fine-tune how it sounds at session time. The available controls depend on the underlying provider — for example, you can adjust speed, stability, style, and more. These settings are optional, and sensible defaults are applied if you skip them. For a full breakdown of available controls by provider, see the [Voice Settings](/docs/full-mode/voice-settings) page. # API Key Configuration Source: https://docs.liveavatar.com/docs/faq/api-key Where to find your LiveAvatar API key Your API key can be found at [app.liveavatar.com/developers](https://app.liveavatar.com/developers). # Credits and Subscriptions Source: https://docs.liveavatar.com/docs/faq/credits How LiveAvatar billing and credit deductions work ## Credit rates | Mode | Cost | | - | - | | Embed / FULL Mode | 2 credits per minute | | Avatar Only (LITE mode) | 1 credit per minute | ### How credits are deducted Sessions cannot start without sufficient credits for at least one minute. LiveAvatar charges per minute of session time. The system checks all concurrent active sessions every minute and deducts credits based on the mode and elapsed duration. > [!NOTE] > > Unlike HeyGen's previous approach, LiveAvatar starts the session before enabling client token generation, so you're not metered during frontend setup. ### Overage billing If you need more credits, toggle on overage in your settings. You are charged for each additional credit consumed. Each plan has a different overage rate. Without overage, sessions automatically end when your credit balance is exhausted and you cannot start new ones. With overage enabled, sessions do not end automatically and you can continue starting new sessions. For higher-volume needs, contact [support@liveavatar.com](mailto:support@liveavatar.com). ## Subscription tiers | Tier | Price | Credits / month | Overage | | - | - | - | - | | Free | \$0 | 10 | Not available | | Essential | \$99 | 1,100 +10 | \$0.095/credit | | Business | \$475 | 6,000 +10 | \$0.09/credit | # Firewall Configuration Source: https://docs.liveavatar.com/docs/faq/firewall Network requirements for LiveAvatar WebRTC connections LiveAvatar uses LiveKit for WebRTC. Connections require WebSocket (WSS) and WebRTC (TLS/DTLS) protocols over TCP and UDP with encrypted connections. ## Minimum required | Host | Port | Purpose | | - | - | - | | `*.livekit.cloud` | TCP 443 | Secure WebSocket signaling | | `*.turn.livekit.cloud` | TCP 443 | TURN over TLS fallback | | `*.host.livekit.cloud` | UDP 3478 | TURN/UDP for peer-to-peer | | `api.liveavatar.com` | TCP 443 | Avatar API and signaling | ## Recommended for best performance * **All hosts**: UDP 50000–60000 for WebRTC media * **All hosts**: TCP 7881 for WebRTC TCP fallback ## Tips * Enable UDP hole-punching where supported * Avoid symmetric NAT configurations when possible * UDP is strongly recommended for low-latency performance; TCP fallback degrades quality ## Wildcard alternatives If your firewall doesn't support wildcards like `*.livekit.cloud`, consult [LiveKit's firewall documentation](https://docs.livekit.io/home/cloud/firewall/) using subdomain `heygen-feapbkvq`. ## Troubleshooting ### Verify browser compatibility Run the [WebRTC browser test](https://livekit.io/webrtc/browser-test) on the client's machine. It checks that the browser supports the APIs LiveAvatar needs and that nothing is blocking them. Confirm the following are not blocked: * **Camera and microphone permissions** — a session requires the browser to grant media device access. Blocked or denied permissions prevent media from flowing. * **WebRTC** — some privacy extensions, enterprise policies, or browser flags disable WebRTC entirely. * **Insecure context** — getUserMedia and WebRTC require HTTPS (or `localhost`). Pages served over plain HTTP cannot start a session. ### Verify end-to-end connection If the firewall and browser checks pass but sessions still fail, walk the full connection path manually. Each step isolates where the failure occurs. **1. Create a session token** Use your API key to request a session token. See [Create Session Token](https://docs.liveavatar.com/api-reference/sessions/create-session-token). ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/sessions/token \ -H "X-API-KEY: " \ -H "accept: application/json" \ -H "content-type: application/json" \ -d '{ "mode": "FULL", "avatar_id": "", "avatar_persona": { "voice_id": "", "context_id": "", "language": "en" } }' ``` The response returns a `session_id` and `session_token`. A failure here points to an API key or request issue, not a network one. **2. Start a session** Use the session token to start the session. See [Start Session](https://docs.liveavatar.com/api-reference/sessions/start-session). ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/sessions/start \ -H "accept: application/json" \ -H "authorization: Bearer " ``` On success, the response returns `livekit_url` and `livekit_client_token` for the created WebRTC room. **3. Test the WebRTC room connection** Pass the `livekit_url` and `livekit_client_token` from the previous step to the [connection tester](https://livekit.io/connection-test) to confirm the client can reach the WebRTC room. If the connection fails or the test shows degraded results, the firewall is likely blocking the required hosts, ports, or UDP media. Re-check the [Minimum required](#minimum-required) hosts and ports above. # Master FAQ Source: https://docs.liveavatar.com/docs/faq/master-faq Comprehensive LiveAvatar FAQ maintained in Notion. # Configuration Source: https://docs.liveavatar.com/docs/full-mode/configuration Customize your LiveAvatar's appearance, voice, behavior, and interactivity FULL Mode provides three configurable layers for your LiveAvatar session. ## Avatar (Visual Layer) Defines what the avatar looks like. Choose from a wide selection of avatars, each with unique styles, appearances, and expressions. Each avatar has a unique `avatar_id`. Browse available avatars through the [List Public Avatars](/api-reference/avatar/list-public-avatars) or [List User Avatars](/api-reference/avatar/list-user-avatars) endpoints. ### Video Settings Video settings tune the rendered output. They apply to FULL Mode and to Avatar Only (LITE mode). #### Quality The `quality` parameter controls output resolution: | Value | Resolution | | - | - | | `very_high` | 1080p | | `high` (default) | 720p | | `medium` | 480p | | `low` | 360p | > [!NOTE] > > Higher resolution increases streaming latency. Use `high` or `medium` for most real-time applications. #### Encoding The `encoding` parameter controls the video codec. Use `H264` (default). > [!WARNING] > > `VP8` is deprecated and will be removed in a future release. Use `H264`. ## Voice Agent (Persona Layer) Defines how the avatar converses — both how it sounds and how it thinks. This is the [voice agent](/docs/core-concepts/voice-agents) layer: a voice paired with a context. You can configure it two ways: * **Reference a stored voice agent** with `voice_agent` (by `id`). The voice and context are resolved from the stored agent. See [Voice Agents](/docs/core-concepts/voice-agents). * **Configure inline** with `avatar_persona`, supplying `voice_id`, `context_id`, and `language` directly. `voice_agent` and `avatar_persona` are mutually exclusive. ### Voice Defines what the avatar sounds like. Voices vary by gender, age, tone, and accent. The `voice_settings` parameter enables fine-grained audio control for speed, style, and stability. See [Configuring Voice Settings](/docs/full-mode/voice-settings) for provider-specific options. ### Context Defines how the avatar thinks and responds. The context layer controls: * Available information and knowledge * Response constraints and guardrails * Personality traits * Opening text (spoken at session start) * Instructions for response generation > [!WARNING] > > If no context is supplied, the avatar operates in **restricted mode** — it will not respond to user input. User transcripts are still emitted, but the avatar can only repeat pre-set phrases. ### LLM Model Defines which language model generates responses. Set `llm_settings` (a `provider` and `model` pair) on `avatar_persona` or on a stored `liveavatar_agent`. Omit it to use the managed default model. See [Configuring LLM Settings](/docs/full-mode/llm-settings) for supported models. To use your own model endpoint and key instead of a managed model, see [Custom LLM Integration](/docs/full-mode/custom-llm). ## Interactivity Type (Conversational Layer) Controls how user input is registered and when the avatar responds. ### Conversational (default) The system manages conversation flow automatically based on speech pauses and user interruptions. ### Push-to-Talk You control exactly when user input is registered by signaling start/stop events. VAD and ASR are disabled until you signal. See [Push-to-Talk](/docs/full-mode/push-to-talk) for details. # Custom LLM Integration Source: https://docs.liveavatar.com/docs/full-mode/custom-llm Use your own language model with LiveAvatar FULL Mode Integrate your own LLM API keys and endpoints with LiveAvatar instead of using the platform's default. ## When to use this * You have an existing streamable LLM setup you want to maintain * You need privacy protections that preclude using LiveAvatar's built-in LLM * You need access to language models not currently supported by the platform ## Setup ### Step 1: Register your API key Store your API credentials securely. LiveAvatar encrypts secrets using Amazon KMS. ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/secrets \ -H "X-API-KEY: " \ -H "content-type: application/json" \ -d '{ "secret_type": "LLM_API_KEY", "secret_value": "", "secret_name": "My LLM Key" }' ``` The response returns a `secret_id` for future reference. ### Step 2: Configure your LLM endpoint Create an LLM configuration with your endpoint details: ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/llm_configurations \ -H "X-API-KEY: " \ -H "content-type: application/json" \ -d '{ "display_name": "My Custom LLM", "model_name": "gpt-4o-mini", "secret_id": "", "base_url": "https://api.openai.com" }' ``` > [!NOTE] > > The `base_url` is optional and defaults to OpenAI. Your endpoint must comply with the OpenAI API specification and use the `/chat/completions` route. The response returns an `llm_configuration_id`. ### Step 3: Start a session with your LLM Set `llm_configuration_id` in your session token: ```json theme={null} { "mode": "FULL", "avatar_id": "", "llm_configuration_id": "", "avatar_persona": { "voice_id": "", "context_id": "" } } ``` ## Supported systems Any **OpenAI-compatible endpoint** using the `/chat/completions` protocol. This includes OpenAI directly, Azure OpenAI, and custom endpoints that wrap other providers. # Custom TTS Integration Source: https://docs.liveavatar.com/docs/full-mode/custom-tts Bring your own ElevenLabs, Fish Audio, or Cartesia voice into LiveAvatar FULL Mode Integrate your own third-party TTS API keys and voice configurations with LiveAvatar FULL Mode sessions. Three providers are supported: | Provider | `secret_type` | | - | - | | ElevenLabs | `ELEVENLABS_API_KEY` | | Fish Audio | `FISH_API_KEY` | | Cartesia | `CARTESIA_API_KEY` | > [!WARNING] > > Custom ElevenLabs TTS is not available on ElevenLabs free-tier API keys. ElevenLabs blocks third-party usage on free plans. You must have a paid ElevenLabs subscription to use ElevenLabs voices. ## When to use this * You have existing voice configurations with one of these providers * You need enhanced control over voice selection and settings ## Setup ### Step 1: Register your provider API key Store the API key for your provider. Set `secret_type` to the value for that provider from the table above. ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/secrets \ -H "X-API-KEY: " \ -H "content-type: application/json" \ -d '{ "secret_type": "CARTESIA_API_KEY", "secret_value": "", "secret_name": "My Cartesia Key" }' ``` Returns a `secret_id`. ### Step 2: Import your voice Bind a provider voice to your stored secret. `provider_voice_id` is the voice ID from the provider; `name` is optional. ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/voices/third_party \ -H "X-API-KEY: " \ -H "content-type: application/json" \ -d '{ "secret_id": "", "provider_voice_id": "", "name": "My Custom Voice" }' ``` Find your voice IDs through the provider's dashboard or API. The imported voice will appear in your LiveAvatar voice listing. ### Step 3: Start a session with your voice Use the `voice_id` from the import when creating your session token: ```json theme={null} { "mode": "FULL", "avatar_id": "", "avatar_persona": { "voice_id": "", "context_id": "" } } ``` LiveAvatar uses your provider credentials for audio generation during the session. > [!WARNING] > > The voice becomes invalid if the underlying provider voice is deleted or the associated secret is removed. # Events Source: https://docs.liveavatar.com/docs/full-mode/events Events for managing FULL Mode sessions LiveAvatar processes events via the LiveKit Protocol. All events follow this JSON structure: ```json theme={null} { "event_id": "string", "event_type": "string", "session_id": "string", "source_event_id": "string | null" } ``` Events are differentiated by LiveKit room topic: * **`agent-control`** — commands you send to the avatar * **`agent-response`** — events the server sends to you ## Command Events Send these to the `agent-control` topic. | Event | Payload | Description | | - | - | - | | `avatar.interrupt` | — | Interrupts and stops any scheduled commands. Disrupts the entire sequence of queued calls. | | `avatar.speak_text` | `{"text": "..."}` | Instructs the avatar to speak the provided text. | | `avatar.speak_response` | `{"text": "..."}` | Instructs the avatar to generate an LLM response to the input text, then speak it. | | `avatar.start_listening` | — | Switches the avatar to a listening state (from idle). | | `avatar.stop_listening` | — | Returns the avatar from listening to idle state. | | `user.start_push_to_talk` | — | *(PTT only)* Signal to start capturing user audio. | | `user.stop_push_to_talk` | — | *(PTT only)* Signal to stop capturing user audio. | ## Response Events Listen for these on the `agent-response` topic. | Event | Payload | Description | | - | - | - | | `user.speak_started` | — | The user has started sending audio. | | `user.speak_ended` | — | The user has stopped sending audio. | | `avatar.speak_started` | — | The avatar has started speaking. | | `avatar.speak_ended` | — | The avatar has finished speaking. | | `user.transcription` | `{"text": "..."}` | Transcription of the user's speech. | | `avatar.transcription` | `{"text": "..."}` | The avatar's text response. Emitted after LLM generation or immediately with `avatar.speak_text`. | | `user.push_to_talk_started` | — | *(PTT only)* PTT successfully started. | | `user.push_to_talk_start_failed` | — | *(PTT only)* PTT failed to start. | | `user.push_to_talk_stopped` | — | *(PTT only)* PTT successfully stopped. | | `user.push_to_talk_stop_failed` | — | *(PTT only)* PTT failed to stop. | | `session.stopped` | `{"end_reason": "..."}` | The session has ended. Includes the reason for termination (see below). | ### `session.stopped` end reasons The `end_reason` field on `session.stopped` is one of: | Value | Meaning | | - | - | | `USER_DISCONNECTED` | The user's client disconnected from the session. | | `USER_CLOSED` | The user explicitly closed the session. | | `IDLE_TIMEOUT` | The session was idle for too long. | | `MAX_DURATION_REACHED` | The session reached its configured maximum duration. | | `AGENT_HANG_UP` | The agent itself terminated the call (e.g. decided the conversation was complete). Currently only emitted by the ElevenLabs agent. | | `AVATAR_DELETED` | The avatar used by the session was deleted. | | `NO_CREDITS` | The account ran out of credits. | | `SERVER_ERROR` | The session ended due to a server-side error. | | `ZOMBIE_SESSION_REAP` | The session was cleaned up after going unresponsive. | | `UNKNOWN` | The session ended for an unspecified reason. | # Lifecycle Source: https://docs.liveavatar.com/docs/full-mode/lifecycle The three phases of a FULL Mode LiveAvatar session ## 1. Starting a Session 1. Generate a session token configured for FULL Mode on your backend 2. Pass the token to your frontend application 3. Call the start session endpoint 4. Establish a WebRTC connection to the LiveAvatar-managed room ```mermaid theme={null} sequenceDiagram participant User as End User participant Frontend as Developer Frontend participant Backend as Developer API participant API as LiveAvatar API participant Room as LiveAvatar Managed Room User->>Frontend: User visits site Frontend->>Backend: Start session flow Backend->>API: POST /v1/sessions/token API-->>Backend: Session-scoped token Backend-->>Frontend: Send token back User->>Frontend: User starts the session Frontend->>API: POST /v1/sessions/start (with token) API->>Room: Start room with avatar API-->>Frontend: Room client token + session details Frontend->>Room: Establish room connection with client token Room-->>Frontend: Conversation starts Frontend-->>User: User sees avatar conversation ``` > [!TIP] > > Make sure the session token has `mode` set to `"FULL"`. Token configuration determines which mode initializes. ## 2. Managing the Session ### Session participants A FULL Mode session is a LiveKit room with three participants: the end user, the LiveAvatar agent, and the avatar. The agent and avatar are both managed by LiveAvatar — they join the room on your behalf. ```mermaid theme={null} graph LR User["End User"] subgraph LA["LiveAvatar Managed"] Room["LiveKit Room"] Agent["Agent"] Avatar["Avatar"] end User <-->|"audio, video, events"| Room Room <-->|"events"| Agent Room <-->|"avatar stream"| Avatar Agent -->|"drives"| Avatar ``` ### Session initialization A session is fully initialized once all three conditions are met: 1. The room is created 2. The avatar joins the room 3. The LiveAvatar agent joins the room The agent's responsibility is to manage the room and process events. It consumes everything happening inside the LiveKit room and drives the avatar to reflect it. ### Event-driven control After connecting to the room, session management is entirely **event-driven**. You control the session by sending and receiving events through LiveKit data channels. Events let you: * Track speaker state (user speaking, avatar speaking) * Capture transcript data * Control avatar behaviors (interrupt, idle, listening) * Direct avatar responses and audio input processing ### Recommended architecture We recommend making your **frontend the primary controller** for emitting events from the LiveKit room — this minimizes latency between user interactions and avatar responses. Event data can be relayed to your backend for downstream processing, logging, and analytics. ```mermaid theme={null} graph LR Frontend["Developer Frontend"] <-->|"send/receive events"| Room["LiveAvatar Managed Room"] Frontend <-->|"avatar video stream"| Room Frontend -->|"relay event data"| Backend["Developer API"] Agent["LiveAvatar Agent"] <--> Room ``` ## 3. Ending the Session When the session ends: 1. The LiveKit room closes 2. The avatar leaves the room 3. The user is removed if still present 4. External resources are cleaned up You can end a session by calling the [Stop Session](/api-reference/sessions/stop-session) endpoint or by the session timing out. ## Next steps * [Configuration](/docs/full-mode/configuration) — customize avatar, voice, context, and interactivity * [Events](/docs/full-mode/events) — full reference of command and response events # Configuring LLM Settings Source: https://docs.liveavatar.com/docs/full-mode/llm-settings Choose the language model behind a FULL Mode conversation LLM settings select which managed language model generates the avatar's responses in FULL Mode. The setting is a `provider` and `model` pair, configured either on a stored voice agent or inline on `avatar_persona` when creating a session token. If you omit `llm_settings`, the session uses the managed default model. ## Supported models LiveAvatar currently supports the following OpenAI models. Support for additional vendors and models is in progress. | Provider | Model | Notes | | - | - | - | | `openai` | `gpt-4o-mini` | Default | | `openai` | `gpt-4o` | | | `openai` | `gpt-5-nano` | | | `openai` | `gpt-5.4-nano` | | Retrieve the current catalog programmatically with [List LLM Models](/api-reference/llm-models/list-llm-models). The response maps each provider to the models it accepts: ```bash theme={null} curl https://api.liveavatar.com/v1/llm-models \ -H "X-API-KEY: " ``` ```json theme={null} { "code": 100, "data": { "openai": ["gpt-4o", "gpt-4o-mini", "gpt-5-nano", "gpt-5.4-nano"] }, "message": "LLM models retrieved successfully" } ``` A model not in this list is rejected with a `422` when you write it to a voice agent or a session token. ## Schema | Field | Type | Default | Description | | - | - | - | - | | `provider` | string | `openai` | LLM vendor. Currently only `openai`. | | `model` | string | `gpt-4o-mini` | Model identifier from the provider's list in [List LLM Models](/api-reference/llm-models/list-llm-models). | ## On a session token Set `llm_settings` under `avatar_persona` when creating a session token. The choice applies to that session only. ```json theme={null} { "mode": "FULL", "avatar_id": "", "avatar_persona": { "voice_id": "", "context_id": "", "language": "en", "llm_settings": { "provider": "openai", "model": "gpt-4o" } } } ``` ## On a voice agent A stored `liveavatar_agent` saves its own `llm_settings` as part of its `agent_configuration`. Every session that references the agent with `voice_agent` uses the stored model, so the choice is made once rather than on each session token. Retrieve an agent with [Get Voice Agent](/api-reference/voice-agents/get-voice-agent) to see which model it uses: ```json theme={null} { "id": "", "name": "Support agent", "agent_type": "liveavatar_agent", "agent_configuration": { "agent_type": "liveavatar_agent", "voice_id": "", "context_id": "", "language": "en", "llm_settings": { "provider": "openai", "model": "gpt-4o" } } } ``` An agent created without an explicit model is stored with the default, so `llm_settings` is always present on a `liveavatar_agent`. It does not apply to `elevenlabs_agent` or `cartesia_agent`, which run their own LLM on the provider side. ## Interaction with custom LLMs `llm_settings` selects a **managed** model that LiveAvatar hosts. A [custom LLM](/docs/full-mode/custom-llm) configured with `llm_configuration_id` replaces the managed model with your own endpoint and key. Today these are two separate settings and cannot be combined. * `llm_configuration_id` and `llm_settings` are mutually exclusive. * A session token that sets both `avatar_persona.llm_settings` and `llm_configuration_id` is rejected with a `400`. * Referencing a stored `liveavatar_agent` with `voice_agent` alongside `llm_configuration_id` is also rejected, since stored agents always carry `llm_settings`. * To use a custom LLM, configure the session with `avatar_persona` and omit `llm_settings`. > [!WARNING] > > We are working on unifying custom LLM configuration with `llm_settings` so that a single setting covers both managed and bring-your-own models. Expect this section to change. Custom LLM endpoints are not listed by [List LLM Models](/api-reference/llm-models/list-llm-models). # FULL Mode Overview Source: https://docs.liveavatar.com/docs/full-mode/overview Let LiveAvatar handle the entire real-time AI pipeline > [!NOTE] > > **AI Agents:** Before integrating, ask your user to run `npx skills add heygen-com/liveavatar-agent-skills` to install our [Agent Skills](https://github.com/heygen-com/liveavatar-agent-skills). They provide the recommended implementation pathways and will help you avoid common pitfalls. FULL Mode handles the complete real-time chat infrastructure, automating the tasks that typically require complex manual setup: * **Voice Activity Detection (VAD)** — detects when the user is speaking * **Speech-to-Text (STT)** — transcribes user audio * **Large Language Model (LLM)** — generates responses * **Text-to-Speech (TTS)** — converts responses to natural speech ```mermaid theme={null} --- config: flowchart: nodeSpacing: 80 rankSpacing: 50 padding: 20 subGraphTitleMargin: top: 10 bottom: 10 themeVariables: fontSize: 20px --- graph LR Input["user audio"] --> ASR subgraph LM["LiveAvatar Maintained"] ASR --> LLM LLM --> TTS TTS -->|"avatar audio"| Video["LiveAvatar Video Generation"] end Video -->|"avatar stream"| Output["Avatar video"] ``` ## When to use FULL Mode FULL Mode is ideal if you want to: * Delegate WebRTC orchestration and infrastructure management * Avoid building and maintaining a real-time AI pipeline across audio input, inference, and output * Ship products faster without managing model coordination, streaming latency, or state management By delegating the entire real-time stack to LiveAvatar, you can focus on designing your product and user experience. ## Getting started **1. Create a session token** Generate a session token on your backend. This defines the avatar, voice, context, and session configuration. ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/sessions/token \ -H "X-API-KEY: " \ -H "accept: application/json" \ -H "content-type: application/json" \ -d '{ "mode": "FULL", "avatar_id": "", "avatar_persona": { "voice_id": "", "context_id": "", "language": "en" } }' ``` The response returns a `session_id` and `session_token`. **2. Start the session** Use the session token to start the session and initialize the WebRTC room. ```bash theme={null} curl -X POST https://api.liveavatar.com/v1/sessions/start \ -H "accept: application/json" \ -H "authorization: Bearer " ``` The response returns a LiveKit `livekit_url` and `livekit_client_token`. **3. Join the LiveKit room** For a quick test, open the LiveKit URL directly in your browser: ```text theme={null} https://meet.livekit.io/custom?liveKitUrl=&token= ``` For production, connect from your frontend using the [Web SDK](https://github.com/heygen-com/liveavatar-web-sdk): ```bash theme={null} npm install @heygen/liveavatar-web-sdk ``` > [!TIP] > > Create session tokens and start sessions from your backend. Pass the LiveKit credentials to your frontend for the user-facing experience. ## Learn more > **[Lifecycle](/docs/full-mode/lifecycle)** > > Understand the three phases of a FULL Mode session. > **[Configuration](/docs/full-mode/configuration)** > > Customize avatar, voice, context, and interactivity. > **[Voice Settings](/docs/full-mode/voice-settings)** > > Fine-tune TTS provider settings. > **[Events](/docs/full-mode/events)** > > Command and response events reference. # Push-to-Talk Source: https://docs.liveavatar.com/docs/full-mode/push-to-talk Explicit control over when user speech starts and ends Push-to-Talk (PTT) gives you deterministic control over speech boundaries instead of relying on automatic voice activity detection. ## When to use PTT * Users with non-linear or thoughtful conversational patterns * Scenarios where short pauses shouldn't signal speech completion * Applications requiring precise control over speech boundaries * Custom UI/UX (press-and-hold buttons, toggles, keyboard shortcuts) ## How it works **1. Start PTT** Your application sends `user.start_push_to_talk`. The system begins buffering audio. **2. User speaks** The user talks freely — pauses, corrections, and hesitations are all captured. No response is generated yet. **3. End PTT** Your application sends `user.stop_push_to_talk`. Audio capture stops. **4. Processing & Response** The avatar processes the captured audio and delivers a response. Audio outside PTT windows is ignored. ## Setup Set `interactivity_type` to `PUSH_TO_TALK` when creating the session token: ```json theme={null} { "mode": "FULL", "interactivity_type": "PUSH_TO_TALK", "avatar_id": "", "avatar_persona": { "voice_id": "", "context_id": "" } } ``` ## Events ### Commands (you send) | Event | Description | | - | - | | `user.start_push_to_talk` | Signal that the user is beginning to speak. | | `user.stop_push_to_talk` | Signal that the user has finished speaking. | ### Responses (you receive) | Event | Description | | - | - | | `user.push_to_talk_started` | PTT successfully started. | | `user.push_to_talk_start_failed` | PTT failed to start. | | `user.push_to_talk_stopped` | PTT successfully stopped. | | `user.push_to_talk_stop_failed` | PTT failed to stop. | ## UI recommendations Map PTT controls to familiar patterns: * **Press-and-hold** button (walkie-talkie style) * **Toggle switch** (tap to start, tap to stop) * **Keyboard shortcut** (spacebar hold) * **Touch or controller input** for mobile/VR # Configuring Voice Settings Source: https://docs.liveavatar.com/docs/full-mode/voice-settings Fine-tune text-to-speech settings for ElevenLabs, Fish Audio, and Cartesia Voice settings are configured under `avatar_persona.voice_settings` when creating a session token. LiveAvatar supports three TTS providers: ElevenLabs, Fish Audio, and Cartesia. ## Shared Settings These apply to all providers: | Parameter | Type | Range | Default | Description | | - | - | - | - | - | | `speed` | number | 0.8–1.2 | 1 | Speaking speed. Below 1 slows down, above 1 speeds up. Cartesia accepts a wider range (0.6–1.5). | ## ElevenLabs | Parameter | Type | Range | Default | Description | | - | - | - | - | - | | `stability` | number | 0–1 | 0.75 | Voice consistency. Higher = more predictable. | | `similarity_boost` | number | 0–1 | 0.75 | Likeness to original voice. | | `style` | number | 0–1 | 0 | Style intensity exaggeration. | | `use_speaker_boost` | boolean | — | true | Voice clarity enhancement. | | `model` | string | — | — | `eleven_flash_v2_5` (low latency) or `eleven_multilingual_v2`. | ## Fish Audio > [!NOTE] > > Fish Audio is currently in beta. Supports English, Chinese, and Japanese only. | Parameter | Options | Default | Description | | - | - | - | - | | `latency_mode` | `balanced`, `normal` | `balanced` | Speed vs. quality tradeoff. | | `model` | `s1`, `s2` | `s2` | Speech generation model. | ## Cartesia | Parameter | Options | Default | Description | | - | - | - | - | | `model` | `sonic-3.5`, `sonic-3`, `sonic-2`, `sonic-turbo` | `sonic-3.5` | Speech generation model. | | `speed` | 0.6–1.5 | 1.0 | Speaking speed. Wider range than the shared default. | ## Example ```json theme={null} { "avatar_persona": { "voice_id": "", "voice_settings": { "speed": 1.0, "stability": 0.8, "similarity_boost": 0.75, "model": "eleven_flash_v2_5" } } } ``` # Change the avatar background Source: https://docs.liveavatar.com/docs/guides/change-background Swap the solid-color backdrop behind your LiveAvatar for transparency, an image, or a video — entirely in the browser. ## What you'll build Some of LiveAvatar's avatars are rendered with solid green backgrounds — see the [green-screen studio avatars](/docs/core-concepts/avatars#green-screen-studio-avatars) list for the stock ones. By detecting and removing those green pixels on the client, you can: * Make the background **transparent** so the avatar sits on top of your own UI. * Replace the background with an **image** (branded backdrop, product shot, office scene). * Replace the background with a **video** (looping environment, ambient footage). This guide shows how to do all three in real time, with no server-side processing and no additional credits consumed. It works in both [FULL Mode](/docs/full-mode/overview) and [Avatar Only (LITE mode)](/docs/lite-mode/overview). > [!TIP] > > Want to skip ahead and try it live? Jump to [Run the reference demo](#run-the-reference-demo). ## How it works The avatar video element exposes a normal `MediaStream` with a green background. You render that stream to a canvas, and for every frame: 1. Convert each pixel to HSV and check whether its hue falls inside a green range. 2. Replace qualifying pixels — either with full transparency, or with a pixel from your custom background. 3. Draw the result to the visible canvas. This runs in any modern browser with no external libraries. > [!NOTE] > > If you're using the default [embed iframe](/docs/guides/embed-avatar), you won't have direct access to the underlying video element because it's in a cross-origin frame. Use the [Web SDK](https://github.com/heygen-com/liveavatar-web-sdk) or an [Avatar Only integration](/docs/lite-mode/integration-paths) (LiveKit, Agora, Pipecat) instead — those give you the raw `MediaStream`. > [!NOTE] > > Training a [custom avatar](/docs/core-concepts/avatars)? Record against a specific solid background color (green is standard, but any saturated, uniform color works) and use the same chroma key technique below to swap it out — adjust `minHue` / `maxHue` to match the color you trained on. ## Run the reference demo > **[liveavatar-web-sdk · apps/bg-removal-demo](https://github.com/heygen-com/liveavatar-web-sdk/tree/master/apps/bg-removal-demo)** > > Complete Next.js reference app — chroma key toggle, solid color, preset images, custom image/video URL, voice chat. Demonstrates: * Toggling the chroma key on/off * Swapping to a solid color * Preset image backgrounds * A custom image or video URL pasted at runtime * Voice chat with the avatar while the backdrop changes ### Initialize Clone repo, install deps, configure env: ```bash theme={null} git clone https://github.com/heygen-com/liveavatar-web-sdk.git cd liveavatar-web-sdk pnpm install cp apps/bg-removal-demo/.env.example apps/bg-removal-demo/.env.local # fill in API_KEY and defaults in apps/bg-removal-demo/.env.local ``` ### Start the dev server Run from monorepo root: ```bash theme={null} pnpm demo:bg-removal ``` Open [http://localhost:3002](http://localhost:3002), click **Start session**, grant microphone permission, then use the **Background** panel on the right to swap backdrops live. ### Environment variables | Variable | Description | | - | - | | `API_KEY` | LiveAvatar API key (server-side only) | | `API_URL` | LiveAvatar API base URL | | `NEXT_PUBLIC_API_URL` | Same as `API_URL` (read by the SDK in the browser) | | `DEFAULT_AVATAR_ID` | Default avatar ID for the start form | | `DEFAULT_VOICE_ID` | Default voice ID | | `DEFAULT_CONTEXT_ID` | Default context ID | | `DEFAULT_LANGUAGE` | Default language code (e.g. `en`) | See [`apps/bg-removal-demo/README.md`](https://github.com/heygen-com/liveavatar-web-sdk/blob/master/apps/bg-removal-demo/README.md) for full setup details, project structure, and the `lib/chromaKey.ts` source the production code is derived from. ## Understanding the details Here's how the reference demo does it. Three steps, each with a clear job: 1. **Mount the DOM surfaces** — a `