Drape And Dictate

Required key.

Add this in your Lovable project under Settings → Secrets before pasting the prompt below.

Paste into a fresh Lovable project. Make sure the key above is set first. read the build strategy →

Build "Drape And Dictate" as a ONE-SHOT Lovable build. The participant has only
5 credits — this single message must produce a working demo with no follow-ups.
Single-page TanStack Start app. Cut scope ruthlessly.

CONCEPT
Lovable AI structures messy fabric notes so you can drape freely; Fish Audio captures your hands-free dictation.
Discipline: Fashion & Textile Design (moulage and draping).
Recipe: Lovable AI brain + Speech-to-Text with Timestamps (live transcription) as the voice surface.
Why this voice surface: Hands are occupied with pins and shears, making realtime voice capture essential for logging adjustments.

LOVABLE BUDGET (HARD CAP: ONE-SHOT, ~5 CREDITS TOTAL):
The participant has FIVE Lovable credits for the whole build. This prompt MUST
ship a working demo on the FIRST message with zero follow-ups. Engineer for that.
- ONE TanStack Start app, ONE route (`src/routes/index.tsx`). No extra pages, no auth, no nav.
- TWO TanStack server functions max: one for the Lovable AI call (the brain),
  one for the Fish Audio call (the voice). Fold them into one if the voice
  primitive does not need server-side text generation.
- ONE client surface (a button, a mic, or a prompt box) wired to those server fns.
- NO database, NO Lovable Cloud, NO auth, NO file uploads, NO extra integrations.
- NO tests, NO docs pages, NO settings screens, NO theming toggles.
- Libraries: template defaults + `ai` + `@ai-sdk/openai-compatible` + `zod`.
  Fish Audio is plain `fetch` — do NOT install an audio SDK.
- Keep the diff small enough to land in one build pass. If a feature is not on
  screen in the user flow below, do not build it. Cut scope before adding scope.

STACK
- TanStack Start app, the index route only.
- Lovable AI Gateway (the brain) + Fish Audio (the voice). All calls live
  inside `createServerFn` handlers so both keys stay on the server.
- Client surface fits the kernel: a prompt box for narration and voice design,
  a mic button for the voice loop, a live caption strip for transcription.
  Render markdown from the brain with `react-markdown` if you show the text.
- Tailwind + shadcn. Editorial look: gold accent on a dark or warm-cream
  background, generous type, one strong headline, one primary action.
- Footer renders: "Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14".

BRAIN — Lovable AI Gateway (free for participants, no key prompt needed):
```ts
// src/lib/ai-gateway.server.ts
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
export function gateway() {
  return createOpenAICompatible({
    name: "lovable",
    baseURL: "https://ai.gateway.lovable.dev/v1",
    headers: {
      "Lovable-API-Key": process.env.LOVABLE_API_KEY!,
      "X-Lovable-AIG-SDK": "vercel-ai-sdk",
    },
  });
}
```
Default model: `google/gemini-3-flash-preview`. Use `generateText` (or
`streamText` for long output) from `ai`. Keep prompts and model calls inside
the server function — never call the gateway from client code.

FISH AUDIO NOTES (read before writing the fetch call):
- Base URL `https://api.fish.audio`. Auth is `Authorization: Bearer <key>`.
- The TTS backend is chosen with a REQUEST HEADER, not a body field:
  `model: s2.1-pro`. (`s2.1-pro-free` is the same model at $0 for prototyping; `s1` is legacy.)
- `POST /v1/tts` takes JSON and returns RAW AUDIO BYTES, not JSON. Read it with
  `await res.arrayBuffer()` and base64 it with `Buffer.from(buf).toString("base64")`.
- Body fields: `text` (required), `format` ("mp3" | "wav" | "pcm" | "opus"),
  `mp3_bitrate` (64 | 128 | 192), `sample_rate`, `prosody: { speed, volume }`,
  and `reference_id` to speak in a specific voice model. Omit `reference_id`
  to use the default voice — that is the right call for a one-shot demo.
- `POST /v1/asr` takes `multipart/form-data` (`audio` file, optional `language`,
  `ignore_timestamps=false` for segment times) and returns JSON
  `{ text, duration, segments: [{ text, start, end }] }`.
- Errors come back as `{ "message": "...", "status": 401 }`. Always check
  `res.ok` and surface `res.status` plus a short slice of the body.

SERVER FUNCTION (src/lib/transcription.functions.ts) — trantranscription with timestamps:
```ts
import { createServerFn } from "@tanstack/react-start";
import { z } from "zod";

type Segment = { text: string; start: number; end: number };

/** Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14 */
export const trantranscription = createServerFn({ method: "POST" })
  .inputValidator((d) => z.object({
    audioBase64: z.string().min(1),
    language: z.string().optional(),   // omit to auto-detect
  }).parse(d))
  .handler(async ({ data }) => {
    const form = new FormData();
    form.append(
      "audio",
      new Blob([Buffer.from(data.audioBase64, "base64")], { type: "audio/webm" }),
      "clip.webm",
    );
    if (data.language) form.append("language", data.language);
    form.append("ignore_timestamps", "false");   // we want segment times

    const r = await fetch("https://api.fish.audio/v1/asr", {
      method: "POST",
      headers: { Authorization: `Bearer ${process.env.FISH_AUDIO_API_KEY!}` },
      body: form,
    });
    if (!r.ok) throw new Error(`ASR failed: ${r.status} ${(await r.text()).slice(0, 120)}`);
    return (await r.json()) as { text: string; duration: number; segments: Segment[] };
  });
```

OPTIONAL BRAIN PASS (src/lib/refine.functions.ts) — Lovable AI shapes the
raw transcript into something useful for moulage and draping:
```ts
import { createServerFn } from "@tanstack/react-start";
import { generateText } from "ai";
import { z } from "zod";
import { gateway } from "./ai-gateway.server";

/** Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14 */
export const refine = createServerFn({ method: "POST" })
  .inputValidator((d) => z.object({ transcript: z.string().min(1) }).parse(d))
  .handler(async ({ data }) => {
    const { text } = await generateText({
      model: gateway()("google/gemini-3-flash-preview"),
      system: `Turn the user's spoken moulage and draping notes into a clean, actionable result.`,
      prompt: data.transcript,
    });
    return { text };
  });
```

CLIENT: record with `MediaRecorder`, and call `rec.start(4000)` so a chunk is
emitted every four seconds. Send each chunk to `trantranscription()` as it arrives and
append the returned `segments` to a live caption strip (render `start`/`end` as
mm:ss). When the user stops, join the text and call `refine()` once.

TRANSLATION — skip:
This kernel is non-linguistic (audio in, or voice timbre out), so do not add a
language selector or a translation pass. Keep the brain-to-voice path direct.

USER FLOW (the entire app — nothing else exists)
1. Land on the page; the headline previews what the demo does for moulage and draping.
2. The primary action (a caption strip that fills in as the user speaks, with timestamped lines saved on every pause) is one tap away; the rest of the layout supports it.
3. Lovable AI does the thinking, Fish Audio handles the voice surface, and the
   result (audio + any text) stays on screen so the user can retry or share.

KEYS — both already provided to participants for free:
1. `LOVABLE_API_KEY` (the AI brain). Auto-injected in every Lovable project.
   Read it only on the server via `process.env.LOVABLE_API_KEY`. Never prefix
   with `VITE_` and never expose to the client.
2. `FISH_AUDIO_API_KEY` (the voice). Ask Lovable to store it as a project
   secret (Project Settings -> Secrets); grab the key from
   https://fish.audio/go-api. Read it only on the server and send it as
   `Authorization: Bearer ${process.env.FISH_AUDIO_API_KEY}`. Never prefix with
   `VITE_`, never call api.fish.audio from the browser.

CREDIT (must appear in UI footer AND as JSDoc on the server function):
Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14

Market sizing.

Indicative figures for hackathon pitches — refine with your own research before raising.