Real voice, one key, one build.

The recipe

# 1. In your Lovable project, add the single required secret (Settings -> Secrets):
FISH_AUDIO_API_KEY=...           # https://fish.audio/go-api

# 2. Copy a mega-prompt from this repo into Lovable. One paste:
#    - scaffolds the React + TanStack Start app
#    - writes a server function that calls Fish Audio (TTS / ASR / voice design)
#    - wires the client surface (<audio>, MediaRecorder, caption strip) for the chosen kernel
#    - includes the hackathon credit in the footer and in JSDoc on the server fn

# 3. Hit play. Your demo is speaking with a real Fish Audio voice.

1. The TTS server function

Every prompt in the library follows the same shape: keep the API key on the server with a TanStack createServerFn, POST the text to Fish Audio, and return the raw audio bytes to the client as base64.

// src/lib/tts.functions.ts — TanStack server function that calls Fish Audio TTS
// Built during the Creative AI & Quantum Hackathon — StreetKode Fam · Indian Krump Festival 14
import { createServerFn } from "@tanstack/react-start";
import { z } from "zod";

export const speak = createServerFn({ method: "POST" })
  .inputValidator((d) => z.object({ text: z.string().min(1).max(4000) }).parse(d))
  .handler(async ({ data }) => {
    const r = await fetch("https://api.fish.audio/v1/tts", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.FISH_AUDIO_API_KEY!}`,
        "Content-Type": "application/json",
        model: "s2.1-pro", // backend is picked with a header, not a body field
      },
      body: JSON.stringify({ text: data.text, format: "mp3", mp3_bitrate: 128 }),
    });
    if (!r.ok) throw new Error(`TTS failed: ${r.status}`);
    const buf = await r.arrayBuffer(); // raw audio bytes, not JSON
    return { audio: Buffer.from(buf).toString("base64") };
  });

2. The spoken loop

Fish Audio has no hosted agent runtime, and it doesn't need one: chain/v1/asr → Lovable AI → /v1/tts inside a single server function and the browser only has to record and play.

// src/lib/agent.functions.ts — one spoken turn: ASR -> Lovable AI -> TTS
import { createServerFn } from "@tanstack/react-start";
import { generateText } from "ai";
import { z } from "zod";
import { gateway } from "./ai-gateway.server";

export const converse = createServerFn({ method: "POST" })
  .inputValidator((d) => z.object({ audioBase64: z.string() }).parse(d))
  .handler(async ({ data }) => {
    const key = process.env.FISH_AUDIO_API_KEY!;
    const form = new FormData();
    form.append("audio", new Blob([Buffer.from(data.audioBase64, "base64")]), "turn.webm");

    // ears
    const asr = await fetch("https://api.fish.audio/v1/asr", {
      method: "POST",
      headers: { Authorization: `Bearer ${key}` },
      body: form,
    });
    const { text: heard } = await asr.json();

    // brain
    const { text: reply } = await generateText({
      model: gateway()("google/gemini-3-flash-preview"),
      prompt: heard,
    });

    // mouth
    const tts = await fetch("https://api.fish.audio/v1/tts", {
      method: "POST",
      headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json", model: "s2.1-pro" },
      body: JSON.stringify({ text: reply, format: "mp3" }),
    });
    return { heard, reply, audio: Buffer.from(await tts.arrayBuffer()).toString("base64") };
  });

3. Voice design on demand

// src/lib/voice.functions.ts — invent a voice from a written brief
import { createServerFn } from "@tanstack/react-start";
import { z } from "zod";

export const designVoice = createServerFn({ method: "POST" })
  .inputValidator((d) => z.object({ instruction: z.string() }).parse(d))
  .handler(async ({ data }) => {
    const r = await fetch("https://api.fish.audio/v1/voice-design", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.FISH_AUDIO_API_KEY!}`,
        "Content-Type": "application/json",
        model: "voice-design-1",
      },
      body: JSON.stringify({
        instruction: data.instruction, // "warm elder storyteller, slow, Lagos accent"
        reference_text: "This is how the voice sounds in your project.",
        language: "en",
        n: 2,
      }),
    });
    return await r.json(); // candidate voices, each with base64 audio
  });