Changelog

​

GPT-Live and OpenAI voice agents, new TTS models, and voice engine details

• GPT-Live in OpenAI Realtime mode. openai_realtime_config.model is now an enum: gpt-realtime (default), gpt-realtime-mini, gpt-realtime-2.1, gpt-realtime-2.1-mini, and gpt-live-1. GPT-Live models run full-duplex and use their own voice set (aster, beacon, cinder, marin, stone, vesper; default marin). temperature applies to Realtime models only.
• openai_agent voice agent type. agent_type adds openai_agent, which runs OpenAI Realtime or GPT-Live on your own OpenAI API key. Its agent_configuration takes model, voice, temperature, secret_id (required), and context_id, and is returned by GET /v1/voice_agents/{agent_id}.
• New TTS models.
  • Cartesia: sonic-3.6, now the default.
  • ElevenLabs: eleven_v4_turbo. Like the v3 models, it honors only stability from the voice settings.
  • Fish Audio: s2-pro and s2.1-pro. s2.1-pro is the new default. latency_mode adds low for the lowest latency.
• GET /v1/voices/{voice_id} now returns voice_engines, the TTS providers that can voice the voice (elevenLabs, fish, cartesia).
• PATCH /v1/avatars/{avatar_id} accepts is_deactivated to deactivate or reactivate an avatar. name is limited to 64 characters.

​

ElevenLabs EU data residency

• Fixed an issue where ElevenLabs API keys from EU data residency workspaces were rejected by POST /v1/secrets.

​

ElevenLabs v3 models, new default LLM, and new avatar status values

• ElevenLabs v3 voice models. voice_settings.model for the elevenlabs provider now accepts eleven_v3 and eleven_v3_conversational alongside eleven_flash_v2_5 (default) and eleven_multilingual_v2. The v3 models are the most expressive but honor only stability from the voice settings; similarity_boost, style, and use_speaker_boost are ignored.
• Default LLM is now gpt-5.4-nano. llm_settings.model defaults to gpt-5.4-nano instead of gpt-4o-mini. llm_settings is no longer nullable on agent_configuration: agents stored without it resolve to the default model on read, so every agent now reports an explicit model.
• Avatar status adds DEACTIVATED and DELETED values. is_expired is now read-only and always present.

​

JavaScript SDK v0.0.19: automatic keep-alive and voice chat config

• @heygen/liveavatar-web-sdk v0.0.19 adds an autoKeepAlive session option. When true, the SDK sends a keep-alive request every 60 seconds while the session is connected and stops on disconnect. Defaults to false; without it, call keepAlive() yourself.
• voiceChat now takes a VoiceChatConfig object only. The default {} starts the microphone unmuted; pass { defaultMuted: true } for a text-only session. Passing true or false is deprecated, logs a console warning, and is mapped to the closest object form. The boolean shim is removed in 0.0.20.

​

Avatar Only (LITE Mode) WebSocket events

• Event correlation. Every server event now carries event_id and source_event_id. Set event_id on the first agent.speak chunk of an utterance and all events for that utterance (agent.speak_started, agent.speak_ended, agent.speak_interrupted, and buffer acks) return it as source_event_id. The unused task.id field has been removed.
• New server events. agent.state_updated reports avatar state transitions (idle, listening, talking). Added agent.audio_buffer_appended, agent.audio_buffer_committed, and agent.audio_buffer_cleared acks, plus error and warning events.
• session.state_updated states are now new, connected, disconnected.
• agent.speak_end accepts an optional trailing audio chunk.

​

Managed LLM selection, public voice agents, and context copying

• LLM model selection. New llm_settings field on avatar_persona and on a liveavatar_agent’s agent_configuration selects the managed model (provider + model). Currently supports OpenAI gpt-4o-mini (default), gpt-4o, gpt-5-nano, and gpt-5.4-nano; omit to use the default. See Configuring LLM Settings.
• New endpoint GET /v1/llm-models lists the managed models available per provider.
• Public voice agents. All accounts now have access to a set of read-only preset voice agents. List them with GET /v1/voice_agents?scope=public; the default scope=private still returns only your own agents. Responses now include is_public.
• Copy contexts. New endpoint POST /v1/contexts/{context_id}/copy hard-copies a readable context into your space as a new private context. Optional name in the body overrides the copy’s name; default is "{original name} (Copy)".
• Rate limits. Public endpoints now enforce per-caller limits (10 requests/second per API key on most endpoints; 5 requests/second per session token on POST /v1/sessions/start, stop, and keep-alive). Over-limit requests return 429 with Retry-After and X-RateLimit-* headers. See Rate Limits.

​

Connector fixes

• ElevenLabs Agent connector. Fixed an issue where the avatar kept speaking after the user interrupted. The connector now stops playback and flushes buffered audio as soon as ElevenLabs reports an interruption.
• Bring-your-own LiveKit. Fixed an issue where sessions joining your own LiveKit room occasionally produced no avatar audio. The avatar now waits for its audio and video tracks to be fully bound before streaming, and retries the connection once if binding does not complete.

​

Voice agent endpoints and Cartesia agents

• Voice agent endpoints. You can now list, retrieve, and delete your stored voice agents: GET /v1/voice_agents, GET /v1/voice_agents/{agent_id}, and DELETE /v1/voice_agents/{agent_id}.
• Cartesia voice agents. A voice agent can now be backed by a Cartesia-hosted agent, using the new cartesia_agent type. In LITE Mode, pass cartesia_agent_config on POST /v1/sessions/token with your agent_id and the secret_id of a stored CARTESIA_API_KEY.

​

Cartesia TTS support

• Cartesia is now available as a voice provider. The voice_settings object on avatar_persona accepts a new CartesiaVoiceSettings variant (discriminated by provider: "cartesia"), with speed (0.6–1.5, default 1.0) and model.
• New CartesiaModelEnum: sonic-3.5 (default), sonic-3, sonic-2, sonic-turbo.
• SecretTypeEnum — Added CARTESIA_API_KEY and FISH_API_KEY for bring-your-own-key configuration via POST /v1/secrets.
• GET /v1/sessions — Session list entries (ListSessionEntrySchema, ListHistoricEntrySchema) now include api_key_id, identifying the API key that created the session.

​

VP8 video encoding deprecated

• VideoEncodingEnum — The VP8 value is deprecated and will be removed in a future release. Use H264 (the default). Sessions that still request VP8 continue to work for now, but should migrate to H264. Affects the encoding field of video_settings on session token requests.

​

Voice agents

• Two agent types. A liveavatar_agent runs LiveAvatar’s managed FULL Mode pipeline (composed from a voice and a context). An elevenlabs_agent is backed by an ElevenLabs-hosted agent and produces a LITE Mode session — even when referenced from a FULL Mode token request.
• POST /v1/sessions/token (FULL Mode) — New optional voice_agent object referencing a stored voice agent by id. Mutually exclusive with avatar_persona.
• Per-session overrides for a liveavatar_agent: language and dynamic_variables, layered over the agent’s stored config for that session only. Both are rejected with a 400 for an elevenlabs_agent. dynamic_variables is mutually exclusive with the top-level field, capped at 50 entries (keys ≤ 64 chars, string values ≤ 1000 chars).
• POST /v2/embeddings — New optional voice_agent_id to resolve voice and context from a stored voice agent. Mutually exclusive with voice_id / context_id.
• New schema: VoiceAgentRef.

​

Attach memory when creating a session token

• POST /v1/sessions/token — FULL Mode tokens now accept an optional memory object. Provide prev_session_id to seed or reuse memory from a prior session, session_memory_id to attach an existing memory, or both (they must resolve to the same memory). Sessions started without it are unaffected.

​

Session memory endpoints

• POST /v1/memory — Create a memory from a completed session. Accepts memory_type (session) and session_id.
• GET /v1/memory — List memories with type, page, and page_size query parameters. Each item returns a memory_content_preview.
• GET /v1/memory/{memory_id} — Fetch a single memory, including its rolled-up memory_content and the per-session session_episodes that compose it.
• DELETE /v1/memory/{memory_id} — Delete a memory and its episodes.
• New schemas: MemoryResponse, MemoryWithEpisodesResponse, MemoryListItemResponse, MemoryEpisodeSchema (each episode carries a session_id, sequence_num, and summary), and MemoryTypeEnum (currently session).

​

ElevenLabs Japanese text normalization

• ElevenLabsVoiceSettings — New optional apply_language_text_normalization boolean (default false). Enables ElevenLabs language-specific text normalization to improve pronunciation. Currently only Japanese (ja) is supported; valid only when the session language is set to ja. Adds latency, so it is opt-in.

​

Delete session events endpoint

• DELETE /v1/sessions/{session_id}/events — New endpoint to hard-delete all events for a given session. Accepts session_id path parameter; authenticated via API key or bearer token.

​

JavaScript SDK v0.0.18: ElevenLabs Agent improvements

• @heygen/liveavatar-web-sdk v0.0.18 — Includes a set of ElevenLabs Agent connector improvements, notably bidirectional passthrough of raw ElevenLabs Conversational AI events. Frontends can now both receive every ElevenLabs server event (via elevenlabs_agent_event) and send raw ElevenLabs client events (via elevenlabs_agent_command) without per-feature plumbing.

​

1080p avatars, ElevenLabs voice override, and more Gemini voices

• 1080p avatars — Added support for 1080p avatars on the new payment plan. AvatarSchema exposes an is_1080p boolean to identify them.
• Smaller session tokens — The session token JWT no longer includes sensitive fields, which also reduces its size. Sandbox Mode tokens keep these fields for testing.
• ElevenLabs voice override — ElevenLabsAgentConfigSchema accepts an optional voice_id that is forwarded as conversation_config_override.tts.voice_id to the ElevenLabs Agent WebSocket, overriding the agent’s default voice for the session.
• Expanded Gemini Realtime voices — GeminiVoiceEnum adds Achernar, Achird, Algenib, Algieba, Alnilam, Autonoe, Callirrhoe, Despina, Enceladus, Erinome, Gacrux, Iapetus, Laomedeia, Pulcherrima, Rasalgethi, Sadachbia, Sadaltager, Schedar, Sulafat, Umbriel, Vindemiatrix, and Zubenelgenubi.

​

Gemini Realtime, vertical embeds, and dynamic variables

• Gemini Realtime agent — LITE sessions can now be configured with gemini_realtime_config, running on Gemini Live API with configurable model, voice, and temperature. Supported voices include Aoede, Charon, Fenrir, Kore, Leda, Orus, Puck, and Zephyr. Requires a new GEMINI_API_KEY secret type.
• Embed orientation — /v2/embeddings now accepts an orientation field (horizontal for 16:9 or vertical for 9:16). The response also returns the new embed_id field.
• Session dynamic variables — Session creation now accepts a dynamic_variables object for filling ${var} placeholders in the context’s opening text and prompt (max 50 entries).
• New session filters — GET /v1/sessions now supports embed_id and context_id query parameters.
• New AGENT_HANG_UP session end reason — Added to SessionEndReasonEnum to indicate the agent itself terminated the call. Currently only emitted by the ElevenLabs agent.

​

ElevenLabs events, embed transcripts, and SDK update

• ElevenLabs agent events — underlying ElevenLabs server events are now forwarded via the elevenlabs_agent_event event type, giving you access to raw ElevenLabs Conversational AI events
• Transcript on embed — Transcripts are now available directly on embed sessions
• JavaScript SDK updated — Updated the @heygen/liveavatar-web-sdk package with new event types and improvements

​

Authentication docs and session filtering

• Authentication — Each endpoint now documents which authentication it accepts (API key or session token)
• New avatar_id filter on /v1/sessions — List sessions can now be filtered by an optional avatar_id query parameter

​

Embeds v2 and image avatars

• /v2/embeddings — New endpoint for creating embeds, with Sandbox Mode support
• Image avatars — Create an animated avatar from a single photo using the new validate and submit endpoints

​

Improvements

• Update avatar endpoint — You can now change an avatar’s default voice
• Independent avatar and voice deletion — Deleting a voice no longer affects its associated avatar, and vice versa
• ElevenLabs agent dynamic variables — Configure dynamic variables for ElevenLabs-powered agents that get replaced at session start
• Source events fixed in FULL Mode — avatar.say and avatar.repeat events now correctly return source_event_id

​

ElevenLabs Scribe and embed settings

• Added support for ElevenLabs Scribe, expanding ASR capabilities
• Embed Improvements
  • Default language selection, letting users set the language for their embed experience
  • Time limits, giving creators control over session duration

​

Voice settings and Fish Audio

• Added additional voice settings for more granular control over avatar speech
• Added access to Fish Audio as a TTS provider
• New [GET] /v1/languages endpoint to clarify which languages are currently supported

​

ElevenLabs integration

• The v1/secrets endpoint now accepts ElevenLabs API keys
• FULL Mode sessions can now start with your own ElevenLabs keys and voices
• Added integration with ElevenLabs Agent functionality

​

LLM plugin and session controls

• Introduced LLM Plugin for custom LLM configurations
• Added customizable session duration controls
• Transcription chunk events are now emitted in real time
• New endpoint to track user credit balances

​

Session configuration and LITE Mode

• New session configuration options for video codec selection (H.264 / VP8)
• Added video quality control settings
• Added voice customization options
• CUSTOM Mode has been renamed to LITE Mode (backward-compatible)
• Bug fix: resolved opening line speech issues

​

Push-to-Talk

• Released Push-to-Talk functionality for avatar sessions
• Added accompanying documentation and integration guides

​

Sandbox Mode and optional FULL Mode context

• Introduced Sandbox Mode for testing without credit charges
• Made context optional in FULL Mode configurations, reducing required setup

​

Custom Mode LiveKit token and session list parameters

• Custom Mode now exposes LiveKit Agent token for developers to integrate their own agents
• Session List endpoint updated to use query parameters

​

Transcript endpoint

• New [GET] /v1/sessions/{session_id}/transcript endpoint
• Enables developers to retrieve transcriptions between users and avatars in FULL Mode sessions

​

Session list endpoint

• New [GET] /v1/sessions endpoint
• Allows listing of both active and previously active sessions