Discussions

Hi there, I have been trying for several days now to get the avatar to repeat a test I send over. the avatar connects, greets me, syncs lips during the greeting; but it won't talk after I pass some text. I mirror the TS demo exactly in JS, but some reason it does not work (FYI - I did not manage to get the TS demo working at my end, it reported many install issues with the rollup).

Here are the critical parts of my code:

// JSON Payload send over LiveKit data channel
async function sendAvatarJson(obj) {
  console.log("[Avatar][sendAvatarJson] called with:", obj);
  debugRoomState("[Avatar][sendAvatarJson] BEFORE checks");

  if (!AvatarState.room) {
    console.warn(
      "[Avatar][sendAvatarJson] No room. AvatarState.room is null/undefined"
    );
    logStatus("Room is not ready");
    return;
  }

  if (!AvatarState.room.localParticipant) {
    console.warn("[Avatar][sendAvatarJson] No localParticipant on room");
    logStatus("Local participant is not ready");
    return;
  }

  console.log(
    "[Avatar][sendAvatarJson] room.state:",
    AvatarState.room.state
  );
  console.log(
    "[Avatar][sendAvatarJson] localParticipant.identity:",
    AvatarState.room.localParticipant.identity
  );
  console.log(
    "[Avatar][sendAvatarJson] localParticipant.permissions:",
    AvatarState.room.localParticipant.permissions
  );

  const json = JSON.stringify(obj);
  console.log("[Avatar][sendAvatarJson] JSON to send:", json);

  const bytes = new TextEncoder().encode(json);
  console.log(
    "[Avatar][sendAvatarJson] Encoded bytes length:",
    bytes.byteLength
  );

  try {
    console.log(
      "[Avatar][sendAvatarJson] Calling publishData with options:",
      { reliable: true, topic: LIVEKIT_COMMAND_CHANNEL_TOPIC }
    );
    await AvatarState.room.localParticipant.publishData(bytes, {
      reliable: true,
      topic: LIVEKIT_COMMAND_CHANNEL_TOPIC,
    });
    console.log(
      "[Avatar][sendAvatarJson] publishData completed successfully"
    );
  } catch (e) {
    console.error("[Avatar][sendAvatarJson] publishData error:", e);
  }
}

// Avatar Speak helper
async function avatarSpeakText_Plain(text) {
  console.log("[Avatar][avatarSpeakText_Plain] called with text:", text);

  const cleanText = String(text || "").trim();
  console.log("[Avatar][avatarSpeakText_Plain] cleanText:", cleanText);

  if (!cleanText) {
    console.warn(
      "[Avatar][avatarSpeakText_Plain] cleanText is empty, not sending"
    );
    return;
  }

  // Behavior A: avatar repeats exactly what you send (no LLM)
  const payload = {
    event_type: "avatar.speak_text",
    text: cleanText,
  };

  // Behavior B (optional): ask persona/LLM to generate a response
  // const payload = {
  //   event_type: "avatar.speak_response",
  //   text: cleanText,
  // };

  console.log("[Avatar][avatarSpeakText_Plain] payload:", payload);
  return sendAvatarJson(payload);
}

The code is called correctly. It's jut not triggering the repeat via the websocket.

Could you assist? FYI - I have about 20 days left to complete the (forced) migration from heygen to liveavatar.

Also, as feedback: It will be good if the documentation shows how to construct the events dictionary. Is unclear whenever I should use "type", "event" or "event_type" in the events payload. Also, the documentation makes no mention of the required fixed topic variable "LIVEKIT_COMMAND_CHANNEL_TOPIC". I found that one after looking into the TS code.