Hydra — live-coded video from chains of functions

When someone asks for live-coded visuals, Hydra, a video synth, VJ visuals for a set or a party, algorithmic or generative video, feedback tunnels and kaleidoscopes, visuals that move with the music, or a page where a model rewrites the picture on request, this is the recipe. Hydra is a video synth that runs on the GPU and is played as text: osc(8, 0.1, 1.2).kaleid(5).rotate(() => time * 0.1).out() is a whole moving picture, and a sketch can be typed by a person, written by knobs, saved as a document, remixed, or produced by callAI from a sentence. It is the visual side of the live-coding scene that Strudel and Tidal serve for music, and the two play together (the third example below).

Hydra is the right call when the picture is a chain of video functions (sources, geometry, colour, blending, modulation, feedback). A 3D scene with meshes and a camera is the three-js skill's; a sketch that draws shapes frame by frame with a pen is p5's.

The worked examples are jchris/hydra-pad (an editor, example chips, a prompt box that rewrites the sketch, saved sketches in a shared gallery), jchris/video-synth (knobs that write the sketch, and the camera as a source) and jchris/beat-visuals (a Strudel pattern whose beat drives a Hydra sketch).

The engine arrives after first paint

Import the engine with its full pinned esm.sh URL inside a memoized module-level loader: await import("https://esm.sh/[email protected]"), called from the stage's mount effect, so the page paints first and a skeleton holds the stage until the first frame (measured 2026-11-11: about 1.2 s for the engine in Chromium, then frames at once). Visuals need no press, unlike sound; the microphone and the camera each wait for theirs.

Construct it with makeGlobal: false, autoLoop: false and detectAudio: false. The defaults put osc, src, o0, time and fifty more names on window (where a Strudel engine on the same page defines its own sine, saw and friends), run a frame loop nobody can stop, and ask for the microphone the moment the page opens. With all three off, the names stay on hydra.synth, the app runs its own requestAnimationFrame loop calling hydra.tick(dt) (stopped when the stage unmounts), and the microphone waits for a "Listen" press.

Hydra is AGPL-licensed. The app links to the engine at its own address, the way a browser links to any page, so the two programs stay separate: credit Hydra visibly (a footer line with a link to hydra.ojack.xyz is the house shape), and keep its code at its URL rather than copied into the app.

Run a sketch inside with blocks over the synth, never with hydra.eval(). Hydra's own eval runs code at global scope, so with makeGlobal: false every name in the sketch is undefined. new Function("synth", "extras", "with (synth) { with (extras) {" + code + "} }") lets a sketch name osc, src, o0 and time bare, sees the app's own values (a.fft) first, and lets speed = 2 or bpm = 120 reach the synth (measured: both land on hydra.synth). Arrow functions in the sketch keep the scope, so () => time * 0.1 reads the live clock every frame.

A sketch that does not run throws, and the last good one keeps playing. A syntax error throws from new Function ("missing ) after argument list"), an unknown function throws when the chain runs ("osc(...).wobble is not a function"); run() returns { ok: false, message } for both and the app shows the message beside the editor. A custom GLSL function that fails to compile is logged to the console by the engine and the picture holds.

Project shape

App.jsx           the interface: the stage, an editor or knobs, examples, the credit line
lib/hydra.js      the loader, attach(), run(), hush(), listen(), startCamera(), hit(), EXAMPLES
lib/sketch-prompt.js   the text-to-sketch system prompt for callAI
access.js         who may publish a sketch to the shared gallery

Relative imports include the extension (./lib/hydra.js). Everything that touches the engine lives in the module, so App.jsx reads as an interface.

Complete starter — a sketch pad

The engine module. attach(el) puts the one canvas in the stage and starts the loop; run(code) shows a sketch; audio.fft is what sketches read as a.fft[0..3], filled by the microphone (listen()) or by the app (hit()); startCamera() feeds the camera into s0.

lib/hydra.js

js// lib/hydra.js — the video synth: one engine, one canvas, a loop the app owns.
const ENGINE_URL = "https://esm.sh/[email protected]";
const WIDTH = 1280;
const HEIGHT = 720;

let enginePromise;
let raf = 0;
let owner = 0; // which attach() holds the canvas now; an older one's cleanup leaves it alone
let micStop = null;
let micStarting; // a listen() still waiting for permission, so a second press joins it

/** What sketches read as `a.fft[0..3]`: four bands from 0 to about 1, filled by listen() or by the app. */
export const audio = { fft: [0, 0, 0, 0] };

/** One download and one canvas however many times it is asked for. Never throws: null when it failed. */
export function loadHydra() {
  enginePromise ??= (async () => {
    const { default: Hydra } = await import(ENGINE_URL);
    const canvas = document.createElement("canvas");
    canvas.width = WIDTH;
    canvas.height = HEIGHT;
    canvas.style.cssText = "display:block;width:100%;height:100%;object-fit:cover";
    const hydra = new Hydra({
      canvas,
      width: WIDTH,
      height: HEIGHT,
      makeGlobal: false, // osc, src, o0… stay on hydra.synth, not on window
      autoLoop: false, // the loop below is ours, so it stops when the stage leaves
      detectAudio: false, // no microphone prompt on load; listen() asks on a press
      enableStreamCapture: false,
    });
    return { hydra, canvas };
  })().catch((error) => {
    enginePromise = undefined; // a later attempt may fetch again
    console.warn("hydra failed to load", error);
    return null;
  });
  return enginePromise;
}

/**
 * Put the canvas in `el` and start the frame loop; the returned function takes both away.
 * Resolves null when the engine did not load, so the stage can say so instead of going live.
 */
export async function attach(el) {
  const engine = await loadHydra();
  if (!engine || !el) return null;
  const { hydra, canvas } = engine;
  const mine = ++owner;
  el.appendChild(canvas);
  cancelAnimationFrame(raf);
  let last = performance.now();
  const frame = (now) => {
    if (!micStop) for (let i = 0; i < 4; i++) audio.fft[i] *= 0.9; // hits fade unless the mic is filling the bands
    hydra.tick(Math.min(now - last, 100));
    last = now;
    raf = requestAnimationFrame(frame);
  };
  raf = requestAnimationFrame(frame);
  return () => {
    if (owner !== mine) return; // a newer attach owns the canvas and the loop
    cancelAnimationFrame(raf);
    canvas.remove();
  };
}

/**
 * Run a sketch. Hydra code is JavaScript that names its functions bare (osc, src, o0, time),
 * so it runs inside `with` blocks over the synth and the app's own values; `speed = 2` and
 * `bpm = 120` reach the synth. Returns { ok: true } or { ok: false, message } — the last good
 * sketch keeps playing either way.
 */
export async function run(code, extras = {}) {
  const engine = await loadHydra();
  if (!engine) return { ok: false, message: "the video engine did not load — check the connection and run again" };
  try {
    const sketch = new Function("synth", "extras", `with (synth) { with (extras) {\n${code}\n} }`);
    sketch(engine.hydra.synth, { a: audio, ...extras });
    return { ok: true };
  } catch (error) {
    return { ok: false, message: String(error?.message || error) };
  }
}

/** Blank every output and forget update(); the canvas stays, black. */
export async function hush() {
  const engine = await enginePromise;
  engine?.hydra.hush();
}

/** Kick one band of a.fft (0 bass … 3 treble) to `amount`; it fades on its own over a few frames. */
export function hit(band, amount = 1) {
  audio.fft[band] = Math.max(audio.fft[band], amount);
}

/**
 * Feed the camera into a source slot (s0 by default) from a press. Throws when the person says no,
 * so the app can say so; returns a function that turns the camera off.
 */
export async function startCamera(slot = "s0") {
  const engine = await loadHydra();
  if (!engine) throw new Error("the video engine did not load");
  const stream = await navigator.mediaDevices.getUserMedia({ video: { facingMode: "user" }, audio: false });
  const video = document.createElement("video");
  video.muted = true;
  video.playsInline = true; // iOS plays inline only with this set
  video.srcObject = stream;
  await video.play();
  engine.hydra.synth[slot].init({ src: video, dynamic: true });
  return () => {
    stream.getTracks().forEach((t) => t.stop());
    engine.hydra.synth[slot].clear();
  };
}

/** Fill audio.fft from the microphone. Call from a press; returns a stop function. */
export async function listen() {
  if (micStop) return micStop;
  micStarting ??= openMic().finally(() => {
    micStarting = undefined;
  });
  return micStarting;
}

async function openMic() {
  const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
  const ctx = new AudioContext();
  await ctx.resume();
  const analyser = ctx.createAnalyser();
  analyser.fftSize = 256;
  analyser.smoothingTimeConstant = 0.6;
  ctx.createMediaStreamSource(stream).connect(analyser);
  const bins = new Uint8Array(analyser.frequencyBinCount);
  const bands = [[1, 4], [4, 12], [12, 40], [40, 100]];
  let id = 0;
  const read = () => {
    analyser.getByteFrequencyData(bins);
    bands.forEach(([from, to], i) => {
      let sum = 0;
      for (let k = from; k < to; k++) sum += bins[k];
      audio.fft[i] = sum / (to - from) / 255;
    });
    id = requestAnimationFrame(read);
  };
  read();
  micStop = () => {
    cancelAnimationFrame(id);
    stream.getTracks().forEach((t) => t.stop());
    ctx.close();
    audio.fft.fill(0);
    micStop = null;
  };
  return micStop;
}

export const EXAMPLES = [
  { name: "kaleido bloom", code: `osc(8, 0.1, 1.2)\n  .kaleid(5)\n  .rotate(() => time * 0.1)\n  .modulate(noise(2), 0.2)\n  .out()` },
  { name: "feedback tunnel", code: `src(o0)\n  .scale(1.01)\n  .rotate(0.01)\n  .blend(shape(4, 0.4, 0.01).color(1, 0.4, 0.8), 0.1)\n  .out()` },
  { name: "voronoi tide", code: `voronoi(5, 0.3, 0.3)\n  .color(0.2, 0.6, 1)\n  .modulateScrollX(osc(3), 0.2)\n  .out()` },
  { name: "pixel rain", code: `noise(4, 0.2)\n  .thresh(0.5)\n  .color(0.2, 1, 0.6)\n  .pixelate(40, 30)\n  .scrollY(0, -0.1)\n  .out()` },
  { name: "sound bloom", code: `shape(6, () => 0.25 + a.fft[0] * 0.5, 0.01)\n  .repeat(3, 3)\n  .modulate(osc(6), () => a.fft[1] * 0.3)\n  .color(1, () => 0.4 + a.fft[2], 0.9)\n  .out()` },
];

The prompt the rewrite box sends. It is the verbatim system prompt for turning a sentence into a sketch (its own section below); the module exports it so any app can hand it to callAI.

lib/sketch-prompt.js

js// lib/sketch-prompt.js — the system prompt for turning a sentence into a Hydra sketch.
// Hand it to callAI with the person's request and the current sketch; read back { name, code }.
export const SKETCH_PROMPT = `You write Hydra sketches. Hydra is a live-coding video synth: a sketch is a chain of functions that builds a moving image, ending in .out().

SOURCES start a chain: osc(frequency, sync, offset) stripes, noise(scale, offset), voronoi(scale, speed, blending), shape(sides, radius, smoothing), gradient(speed), solid(r, g, b, a), src(o0) reads an output back (feedback).
GEOMETRY: .rotate(angle, speed), .scale(amount, xMult, yMult), .kaleid(sides), .pixelate(x, y), .repeat(x, y), .scroll(x, y, speedX, speedY), .scrollX(amount, speed), .scrollY(amount, speed).
COLOR: .color(r, g, b), .colorama(amount), .hue(amount), .saturate(amount), .contrast(amount), .brightness(amount), .invert(), .luma(threshold), .thresh(threshold), .posterize(bins, gamma), .shift(r, g, b).
BLENDING with a second chain: .add(chain, amount), .blend(chain, amount), .diff(chain), .mult(chain, amount), .mask(chain), .layer(chain).
MODULATE moves pixels by a second chain: .modulate(chain, amount), .modulateRotate(chain, multiple), .modulateScale(chain, multiple), .modulateKaleid(chain, sides), .modulateScrollX(chain, amount), .modulatePixelate(chain, multiple).
OUTPUTS: .out() draws to o0, which is what shows. src(o0).scale(1.01).blend(…, 0.1).out() makes trails and tunnels.

MOTION. Any number can be a function of time: () => time * 0.1, () => Math.sin(time) * 0.5. An array steps through values: shape([3, 4, 5]).
SOUND. a.fft[0] (bass) to a.fft[3] (treble) run from 0 to about 1 while sound is on: () => 0.2 + a.fft[0] * 0.5.

OUTPUT. Only Hydra code: one chain ending in .out(), or a few lines each ending in .out(oN) with render() or one final .out(). No imports, no comments, no prose. Keep amounts small: rotate speeds under 0.5, modulate amounts under 0.5, scale feedback between 0.95 and 1.05.

EXAMPLES
Request: slow purple kaleidoscope
Sketch: osc(6, 0.05, 1).color(0.7, 0.2, 1).kaleid(6).rotate(() => time * 0.05).out()
Request: a glowing tunnel that pulls you in
Sketch: src(o0).scale(1.02).rotate(0.005).blend(shape(32, 0.3, 0.02).color(0.2, 0.8, 1), 0.08).out()
Request: glitchy green rain
Sketch: noise(8, 0.3).thresh(0.6).color(0, 1, 0.4).pixelate(60, 20).scrollY(0, -0.2).out()
Request: pulse with the bass
Sketch: shape(4, () => 0.2 + a.fft[0] * 0.6, 0.02).rotate(() => time * 0.2).modulate(noise(3), () => a.fft[1] * 0.2).out()`;

/** The full prompt for one request: the spec, the ask, and the sketch that is showing now. */
export function rewritePrompt(request, currentCode) {
  return `${SKETCH_PROMPT}\n\nRequest: ${request}\nCurrent sketch (change it, or replace it when the request asks for something new):\n${currentCode}\n\nReturn JSON with "name" (two or three words) and "code" (the sketch).`;
}

The interface: the stage, an editor, example chips, a prompt box that asks callAI to rewrite the sketch, a "Listen to the room" button that makes a.fft follow the microphone, full screen, and the saved sketches from Fireproof with every write gated on can.

App.jsx

jsximport React, { useEffect, useMemo, useRef, useState } from "react";
import { useFireproof } from "use-fireproof";
import { callAI, useVibe, useViewer } from "use-vibes";
import { Badge, Button, Card, Field, Heading, Input, Row, Skeleton, Spacer, Stack, Text } from "@vibes.diy/ui";
import { EXAMPLES, attach, listen, run } from "./lib/hydra.js";
import { rewritePrompt } from "./lib/sketch-prompt.js";

export default function App() {
  const { database, useLiveQuery } = useFireproof("sketchPad");
  const { me, can, ready } = useVibe("sketchPad");
  const { requestLogin, ViewerTag } = useViewer();
  const { docs } = useLiveQuery("type", { key: "sketch" });
  const saved = useMemo(() => [...docs].sort((a, b) => (b.createdAt ?? 0) - (a.createdAt ?? 0)), [docs]);

  const stage = useRef(null);
  const [code, setCode] = useState(EXAMPLES[0].code);
  const [live, setLive] = useState(false);
  const [problem, setProblem] = useState(null);
  const [ask, setAsk] = useState("");
  const [asking, setAsking] = useState(false);
  const [name, setName] = useState("");
  const [hearing, setHearing] = useState(null); // the stop function while the mic is on
  const alive = useRef(true); // false once the pad unmounts, so a late microphone is closed at once

  // The engine arrives after first paint; the stage shows a skeleton until the first frame.
  useEffect(() => {
    let gone = false;
    let detach = () => {};
    attach(stage.current).then(async (d) => {
      if (gone) return d?.();
      if (!d) return setProblem("the video engine did not load — check the connection and reload the page");
      detach = d;
      const result = await run(code);
      if (!result.ok) setProblem(result.message);
      setLive(true);
    });
    return () => {
      gone = true;
      detach();
    };
  }, []);
  useEffect(() => () => hearing?.(), [hearing]);
  useEffect(() => () => void (alive.current = false), []);

  async function show(source = code) {
    const result = await run(source);
    setProblem(result.ok ? null : result.message);
  }

  function pick(next) {
    setCode(next);
    show(next);
  }

  async function rewrite() {
    if (!ask.trim() || asking) return;
    setAsking(true);
    try {
      const reply = await callAI(rewritePrompt(ask, code), {
        schema: { properties: { name: { type: "string" }, code: { type: "string" } } },
      });
      const next = JSON.parse(reply);
      setAsk("");
      pick(next.code);
    } catch {
      setProblem("the rewrite did not come back — press Rewrite to ask again");
    } finally {
      setAsking(false);
    }
  }

  async function toggleListening() {
    if (hearing) {
      hearing();
      setHearing(null);
      return;
    }
    try {
      const stop = await listen();
      if (!alive.current) return stop(); // the pad closed while the permission prompt was up
      setHearing(() => stop);
    } catch {
      setProblem("the microphone stayed off, so a.fft reads zero — sketches still play");
    }
  }

  const draft = { type: "sketch", name: name || "untitled", code, authorHandle: me?.userHandle };
  const verdict = can.create(draft);

  async function keep() {
    setName("");
    const { ok } = await database.put({ ...draft, createdAt: Date.now() });
    if (!ok) setName((now) => now || name);
  }

  return (
    <main className="min-h-screen bg-background text-foreground p-4 md:p-8">
      <Stack gap={4}>
        <Row gap={2} align="center" wrap>
          <Heading level={1}>Sketch Pad</Heading>
          <Spacer />
          <Badge tone={live ? "up" : "idle"}>{live ? "Live" : "Warming up the synth…"}</Badge>
        </Row>

        <div ref={stage} className="relative aspect-video w-full overflow-hidden rounded-md bg-neutral">
          {!live && <div aria-busy="true" className="absolute inset-0"><Skeleton variant="card" /></div>}
        </div>

        <Row gap={2} align="center" wrap>
          <Button onClick={() => show()}>Run</Button>
          <Button variant="quiet" onClick={() => stage.current?.requestFullscreen?.()}>Full screen</Button>
          <Button variant="quiet" onClick={toggleListening}>{hearing ? "Stop listening" : "Listen to the room"}</Button>
          <Text size="sm" muted>Ctrl+Enter runs the editor</Text>
        </Row>

        <textarea
          value={code}
          onChange={(e) => setCode(e.target.value)}
          onKeyDown={(e) => { if ((e.ctrlKey || e.metaKey) && e.key === "Enter") { e.preventDefault(); show(); } }}
          spellCheck={false}
          rows={7}
          aria-label="Hydra sketch"
          className="w-full rounded-md border border-border bg-card p-3 font-mono text-sm text-foreground outline-none"
        />
        {problem && <Text size="sm" tone="down">That sketch needs a tweak: {problem}. The last good one keeps playing.</Text>}

        <Row gap={2} wrap>
          {EXAMPLES.map((ex) => (
            <Button key={ex.name} size="sm" variant="quiet" onClick={() => pick(ex.code)}>{ex.name}</Button>
          ))}
        </Row>

        <Card variant="tight">
          <Row gap={2} align="top" wrap>
            <Field label="Ask for a change">{(id, d) => <Input id={id} describedBy={d} value={ask} onChange={setAsk} placeholder="slower, and more purple" />}</Field>
            <Button size="sm" onClick={rewrite} disabled={asking}>{asking ? "Rewriting…" : "Rewrite"}</Button>
          </Row>
        </Card>

        <Card variant="tight">
          <Stack gap={3}>
            <Heading level={2}>Keep this sketch</Heading>
            {!ready && <div aria-busy="true"><Skeleton variant="row" /></div>}
            {ready && !me && <Row gap={2} align="center" wrap><Text muted>Sign in and your sketches stay with you.</Text><Button size="sm" variant="quiet" onClick={requestLogin}>Sign in to keep sketches</Button></Row>}
            {ready && me && verdict.ok && (
              <Row gap={2} align="top" wrap>
                <Field label="Name">{(id, d) => <Input id={id} describedBy={d} value={name} onChange={setName} placeholder="midnight bloom" />}</Field>
                <Button size="sm" onClick={keep}>Keep</Button>
              </Row>
            )}
            {saved.map((s) => (
              <Row key={s._id} gap={2} align="center" wrap>
                <Button size="sm" variant="quiet" onClick={() => pick(s.code)}>{s.name}</Button>
                {s.authorHandle && <ViewerTag userHandle={s.authorHandle} />}
                <Spacer />
                {can.delete(s).ok && <Button size="sm" variant="quiet" tone="down" onClick={() => database.del(s._id)}>Remove</Button>}
              </Row>
            ))}
          </Stack>
        </Card>

        <Text size="sm" muted>
          Video by <a className="underline" href="https://hydra.ojack.xyz" target="_blank" rel="noreferrer">Hydra</a> (AGPL), fetched from its own address when the page opens.
        </Text>
      </Stack>
    </main>
  );
}

access.js

js// sketch: a saved Hydra sketch. Anyone signed in keeps their own; everyone can read the gallery.
export function sketchPad(doc, oldDoc, user, ctx) {
  if (!user) throw { forbidden: "sign in to keep sketches" };
  if (doc.type === "sketch" || (doc._deleted && oldDoc && oldDoc.type === "sketch")) {
    const author = oldDoc ? oldDoc.authorHandle : doc.authorHandle;
    if (author !== user.userHandle || (!doc._deleted && doc.authorHandle !== user.userHandle)) throw { forbidden: "sign your sketch with your own handle" };
    return { channels: ["sketches"], grant: { public: ["sketches"] } };
  }
  throw { forbidden: "unknown document type" };
}

The language in one table

A sketch is a chain: one source, then any number of transforms, ending in .out(). Every argument can be a number, a function of time (() => Math.sin(time) * 0.5), or an array that steps through values (shape([3, 4, 5]), with .fast(2) to step twice as often).

Feedback is src(o0) inside the chain that draws to o0: src(o0).scale(1.01).rotate(0.01).blend(shape(4), 0.1).out() leaves trails, tunnels and smears. Keep the scale near 1 (0.95 to 1.05) and the blend small, or the frame floods to one colour in a second.

Four outputs (o0–o3) and four sources (s0–s3). An output is a chain's canvas; a source is outside pixels: s0 holds the camera after startCamera("s0"), s1.initImage(url) an image, s2.initVideo(url) a looping video (both need a CORS-open URL, or the frame stays black).

speed = 0.5 slows every clock in the sketch; bpm = 120 sets the rate an array steps at. Both are plain assignments in the sketch, and both land on the synth through the with block.

The interface writes the sketch

The pad above is the editor shape. The other shape is the one most people want: knobs, buttons and grids that compile to a sketch string and re-run it as they move. A knob is a number dropped into the string; the effect that runs the sketch depends on the string, so every move swaps the picture without a gap. Show the string the knobs wrote, so the person can carry it into any Hydra editor, and learn the language by watching it change.

A video synth whose knobs write the sketch, with the camera as a source:

App.jsx

jsximport React, { useEffect, useRef, useState } from "react";
import { Badge, Button, Card, Heading, Row, Skeleton, Spacer, Stack, Text } from "@vibes.diy/ui";
import { Slider } from "@vibes.diy/ui/interactive";
import { attach, run, startCamera } from "./lib/hydra.js";

// Each knob is a number dropped into one sketch string; every move re-runs it.
const KNOBS = [
  { key: "sides", label: "Mirror", min: 1, max: 12, step: 1 },
  { key: "warp", label: "Warp", min: 0, max: 0.6, step: 0.01 },
  { key: "trail", label: "Trails", min: 0, max: 0.95, step: 0.01 },
  { key: "spin", label: "Spin", min: -0.5, max: 0.5, step: 0.01 },
  { key: "hue", label: "Colour drift", min: 0, max: 0.3, step: 0.01 },
];
const START = { sides: 4, warp: 0.2, trail: 0.6, spin: 0.1, hue: 0.05 };

function toSketch(k, camera) {
  const source = camera ? "src(s0).saturate(1.4)" : "osc(10, 0.08, 1.2)";
  return [
    `${source}`,
    `  .modulate(noise(2.5), ${k.warp})`,
    `  .kaleid(${k.sides})`,
    `  .rotate(() => time * ${k.spin})`,
    `  .colorama(${k.hue})`,
    `  .blend(src(o0).scale(1.01), ${k.trail})`,
    `  .out()`,
  ].join("\n");
}

export default function App() {
  const stage = useRef(null);
  const [knobs, setKnobs] = useState(START);
  const [camera, setCamera] = useState(false);
  const [live, setLive] = useState(false);
  const [note, setNote] = useState(null);
  const cameraOff = useRef(null);
  const alive = useRef(true); // false once the synth unmounts, so a late camera is stopped at once
  const asking = useRef(false); // one camera request at a time
  const code = toSketch(knobs, camera);

  useEffect(() => {
    let gone = false;
    let detach = () => {};
    attach(stage.current).then((d) => {
      if (gone) return d?.();
      if (!d) return setNote("The video engine did not load. Check the connection and reload the page.");
      detach = d;
      setLive(true);
    });
    return () => {
      gone = true;
      alive.current = false;
      detach();
      cameraOff.current?.();
    };
  }, []);

  // The interface writes the sketch: any change re-runs it, and the synth swaps without a gap.
  useEffect(() => {
    if (live) run(code).then((r) => setNote(r.ok ? null : r.message));
  }, [live, code]);

  async function cameraOn() {
    if (asking.current) return;
    asking.current = true;
    try {
      const off = await startCamera("s0"); // asks from this press; the sketch reads it as s0
      if (!alive.current) return off(); // the synth closed while the permission prompt was up
      cameraOff.current = off;
      setCamera(true);
      setNote(null);
    } catch {
      setNote("The camera stayed off, so the synth plays its own stripes. Allow the camera and press again.");
    } finally {
      asking.current = false;
    }
  }

  function cameraBack() {
    cameraOff.current?.();
    cameraOff.current = null;
    setCamera(false);
  }

  return (
    <main className="min-h-screen bg-background text-foreground p-4 md:p-8">
      <Stack gap={4}>
        <Row gap={2} align="center" wrap>
          <Heading level={1}>Video Synth</Heading>
          <Spacer />
          <Badge tone={live ? "up" : "idle"}>{live ? (camera ? "Camera in" : "Live") : "Warming up…"}</Badge>
        </Row>

        <div ref={stage} className="relative aspect-video w-full overflow-hidden rounded-md bg-neutral">
          {!live && <div aria-busy="true" className="absolute inset-0"><Skeleton variant="card" /></div>}
        </div>

        <Card variant="tight">
          <Stack gap={3}>
            {KNOBS.map((k) => (
              <Row key={k.key} gap={3} align="center">
                <Slider value={[knobs[k.key]]} onChange={([v]) => setKnobs((now) => ({ ...now, [k.key]: v }))} min={k.min} max={k.max} step={k.step} label={k.label} />
                <Text size="sm" muted>{k.label} {knobs[k.key]}</Text>
              </Row>
            ))}
          </Stack>
        </Card>

        <Row gap={2} wrap>
          <Button onClick={camera ? cameraBack : cameraOn}>{camera ? "Back to stripes" : "Use my camera"}</Button>
          <Button variant="quiet" onClick={() => setKnobs(START)}>Reset knobs</Button>
          <Button variant="quiet" onClick={() => stage.current?.requestFullscreen?.()}>Full screen</Button>
        </Row>
        {note && <Text size="sm" tone="down">{note}</Text>}

        <Card variant="tight">
          <Stack gap={1}>
            <Text size="sm" muted>The sketch these knobs write — paste it into any Hydra editor</Text>
            <pre className="whitespace-pre-wrap font-mono text-sm">{code}</pre>
          </Stack>
        </Card>

        <Text size="sm" muted>
          Video by <a className="underline" href="https://hydra.ojack.xyz" target="_blank" rel="noreferrer">Hydra</a> (AGPL), fetched from its own address when the page opens.
        </Text>
      </Stack>
    </main>
  );
}

The camera is a press, then s0. startCamera("s0") asks for the camera from the press that called it, plays it into a <video> with playsInline (iOS shows a camera inline only with it), and hands it to the slot; the sketch reads it as src(s0). When the person says no, the call throws, and the app says the synth plays its own stripes until the camera is allowed. The function it returns stops the camera; call it on the way back and in the unmount cleanup, so the camera light goes off with the page.

Sound makes it move

A sketch reads sound as a.fft[0] (bass) to a.fft[3] (treble), each from 0 to about 1, the same names Hydra's own editor uses, so sketches people share in the Hydra community run as written. The engine's own audio reader asks for the microphone when the page opens, so it stays off (detectAudio: false) and lib/hydra.js owns a instead. Two things fill it:

• The room, through listen(): from a "Listen" press it opens the microphone, reads four bands from an AnalyserNode every frame, and returns the function that closes the microphone. A sketch such as shape(6, () => 0.25 + a.fft[0] * 0.5).out() breathes with whatever is playing in the room.
• The app's own music, through hit(band, amount): each event the app plays kicks one band to amount, and the frame loop lets it fall back (each band keeps 90% per frame), so a kick is a flash that fades in about half a second.

A Strudel pattern driving a Hydra sketch is the second case. Strudel's onTrigger hands every sounding event to the app at the moment it is scheduled; a setTimeout of (targetTime - time) * 1000 lands the hit on the beat, and the event's sound name picks the band (kick to a.fft[0], snare and clap to a.fft[1], hats to a.fft[2], notes to a.fft[3]). The pattern side is the Strudel doc's lib/strudel.js, unchanged; this is the visual half, and a set is the pair:

lib/sets.js

js// lib/sets.js — a set is a Strudel pattern and the Hydra sketch it drives.
// The beat reaches the sketch as a.fft: kick → a.fft[0], snare and clap → a.fft[1], hats → a.fft[2], notes → a.fft[3].
export const SETS = [
  {
    name: "kick bloom",
    pattern: `stack(\n  s("bd*4, ~ cp ~ cp, hh*8").bank("RolandTR909"),\n  note("<c2 c2 eb2 g1>").s("sawtooth").lpf(700).gain(0.5)\n).cpm(124/4)`,
    sketch: `shape(4, () => 0.15 + a.fft[0] * 0.35, 0.02)\n  .color(1, 0.3, () => 0.5 + a.fft[1] * 0.5)\n  .rotate(() => time * 0.2)\n  .modulate(noise(3), () => a.fft[2] * 0.15)\n  .out()`,
  },
  {
    name: "tunnel break",
    pattern: `s("bd ~ [~ bd] ~, ~ sd ~ sd, hh*8").bank("RolandTR808").cpm(92/4)`,
    sketch: `src(o0)\n  .scale(() => 1.01 + a.fft[0] * 0.05)\n  .rotate(0.004)\n  .blend(osc(20, 0.05, 1).kaleid(4).mask(shape(4, 0.2, 0.05)), () => 0.05 + a.fft[1] * 0.3)\n  .out()`,
  },
  {
    name: "acid voronoi",
    pattern: `stack(\n  s("bd*4"),\n  note("c2 [eb2 c2] g1 [c2 bb1]").s("sawtooth").lpf(sine.range(300, 2000).slow(4)).resonance(10).gain(0.6)\n).cpm(130/4)`,
    sketch: `voronoi(6, 0.3, 0.2)\n  .color(0.2, () => 0.6 + a.fft[3] * 0.4, 1)\n  .modulateScale(osc(4), () => a.fft[0] * 0.4)\n  .colorama(() => a.fft[3] * 0.1)\n  .out()`,
  },
];

/** Which band of a.fft a sounding event kicks. */
export function bandFor(value) {
  const s = String(value?.s ?? "");
  if (/bd|kick/.test(s)) return 0;
  if (/sd|cp|rim|snare|clap/.test(s)) return 1;
  if (/hh|oh|cr|hat/.test(s)) return 2;
  return 3;
}

App.jsx

jsximport React, { useEffect, useMemo, useRef, useState } from "react";
import { useFireproof } from "use-fireproof";
import { useVibe, useViewer } from "use-vibes";
import { Badge, Button, Card, Field, Heading, Input, Row, Skeleton, Spacer, Stack, Text } from "@vibes.diy/ui";
import { attach, hit, run } from "./lib/hydra.js";
import { onPatternError, play, stop } from "./lib/strudel.js";
import { SETS, bandFor } from "./lib/sets.js";

const editorClass = "w-full rounded-md border border-border bg-card p-3 font-mono text-sm text-foreground outline-none";

export default function App() {
  const { database, useLiveQuery } = useFireproof("beatVisuals");
  const { me, can, ready } = useVibe("beatVisuals");
  const { requestLogin, ViewerTag } = useViewer();
  const { docs } = useLiveQuery("type", { key: "set" });
  const saved = useMemo(() => [...docs].sort((a, b) => (b.createdAt ?? 0) - (a.createdAt ?? 0)), [docs]);

  const stage = useRef(null);
  const timers = useRef([]);
  const [pattern, setPattern] = useState(SETS[0].pattern);
  const [sketch, setSketch] = useState(SETS[0].sketch);
  const [live, setLive] = useState(false);
  const [sound, setSound] = useState("idle"); // idle | loading | playing
  const [problem, setProblem] = useState(null);
  const [name, setName] = useState("");

  useEffect(() => {
    let gone = false;
    let detach = () => {};
    attach(stage.current).then(async (d) => {
      if (gone) return d?.();
      if (!d) return setProblem("the video engine, which did not load — check the connection and reload the page");
      detach = d;
      await run(sketch);
      setLive(true);
    });
    return () => {
      gone = true;
      detach();
      stop();
      timers.current.forEach(clearTimeout);
    };
  }, []);
  useEffect(() => onPatternError((e) => setProblem(`the pattern: ${e.message || e}`)), []);

  // Every sounding event kicks a band of a.fft at the moment it sounds.
  function onEvent(hap, time, cps, target) {
    const delay = Math.max(0, (target - time) * 1000);
    timers.current.push(setTimeout(() => hit(bandFor(hap.value), hap.value?.gain ?? 1), delay));
    if (timers.current.length > 256) timers.current = timers.current.slice(-128);
  }

  async function go(nextPattern = pattern, nextSketch = sketch) {
    setProblem(null);
    const wasPlaying = sound === "playing";
    if (!wasPlaying) setSound("loading");
    // play() first: it unlocks audio before its first await, while this is still the press.
    const playing = play(nextPattern, onEvent);
    const shown = await run(nextSketch);
    if (!shown.ok) setProblem(`the sketch: ${shown.message}`);
    const ok = await playing;
    setSound(ok ? "playing" : wasPlaying ? "playing" : "idle");
  }

  async function halt() {
    await stop();
    timers.current.forEach(clearTimeout);
    timers.current = [];
    setSound("idle");
  }

  function pick(set) {
    setPattern(set.pattern);
    setSketch(set.sketch);
    if (sound === "playing") go(set.pattern, set.sketch);
    else run(set.sketch);
  }

  const draft = { type: "set", name: name || "untitled", pattern, sketch, authorHandle: me?.userHandle };
  const verdict = can.create(draft);

  async function keep() {
    setName("");
    const { ok } = await database.put({ ...draft, createdAt: Date.now() });
    if (!ok) setName((now) => now || name);
  }

  const runKeys = (e) => {
    if ((e.ctrlKey || e.metaKey) && e.key === "Enter") {
      e.preventDefault();
      go();
    }
  };

  return (
    <main className="min-h-screen bg-background text-foreground p-4 md:p-8">
      <Stack gap={4}>
        <Row gap={2} align="center" wrap>
          <Heading level={1}>Beat Visuals</Heading>
          <Spacer />
          <Badge tone={sound === "playing" ? "up" : "idle"}>{sound === "playing" ? "Playing" : sound === "loading" ? "Fetching the drum kit…" : live ? "Press play" : "Warming up the synth…"}</Badge>
        </Row>

        <div ref={stage} className="relative aspect-video w-full overflow-hidden rounded-md bg-neutral">
          {!live && <div aria-busy="true" className="absolute inset-0"><Skeleton variant="card" /></div>}
        </div>

        <Row gap={2} align="center" wrap>
          <Button onClick={() => (sound === "playing" ? halt() : go())}>{sound === "playing" ? "Stop" : "Play"}</Button>
          <Button variant="quiet" onClick={() => go()}>Update</Button>
          <Button variant="quiet" onClick={() => stage.current?.requestFullscreen?.()}>Full screen</Button>
          {SETS.map((s) => (
            <Button key={s.name} size="sm" variant="quiet" onClick={() => pick(s)}>{s.name}</Button>
          ))}
        </Row>

        <div className="grid gap-4 md:grid-cols-2">
          <Stack gap={1}>
            <Text size="sm" muted>Pattern (Strudel)</Text>
            <textarea aria-label="Strudel pattern" value={pattern} onChange={(e) => setPattern(e.target.value)} onKeyDown={runKeys} spellCheck={false} rows={7} className={editorClass} />
          </Stack>
          <Stack gap={1}>
            <Text size="sm" muted>Sketch (Hydra) — kick is a.fft[0], snare a.fft[1], hats a.fft[2], notes a.fft[3]</Text>
            <textarea aria-label="Hydra sketch" value={sketch} onChange={(e) => setSketch(e.target.value)} onKeyDown={runKeys} spellCheck={false} rows={7} className={editorClass} />
          </Stack>
        </div>
        {problem && <Text size="sm" tone="down">Needs a tweak in {problem}. The last good one keeps going.</Text>}

        <Card variant="tight">
          <Stack gap={3}>
            <Heading level={2}>Keep this set</Heading>
            {!ready && <div aria-busy="true"><Skeleton variant="row" /></div>}
            {ready && !me && <Row gap={2} align="center" wrap><Text muted>Sign in to keep sets and share them here.</Text><Button size="sm" variant="quiet" onClick={requestLogin}>Sign in</Button></Row>}
            {ready && me && verdict.ok && (
              <Row gap={2} align="top" wrap>
                <Field label="Set name">{(id, d) => <Input id={id} describedBy={d} value={name} onChange={setName} placeholder="friday opener" />}</Field>
                <Button size="sm" onClick={keep}>Keep</Button>
              </Row>
            )}
            {saved.map((s) => (
              <Row key={s._id} gap={2} align="center" wrap>
                <Button size="sm" variant="quiet" onClick={() => pick(s)}>{s.name}</Button>
                {s.authorHandle && <ViewerTag userHandle={s.authorHandle} />}
                <Spacer />
                {can.delete(s).ok && <Button size="sm" variant="quiet" tone="down" onClick={() => database.del(s._id)}>Remove</Button>}
              </Row>
            ))}
          </Stack>
        </Card>

        <Text size="sm" muted>
          Sound by <a className="underline" href="https://strudel.cc" target="_blank" rel="noreferrer">Strudel</a> and video by <a className="underline" href="https://hydra.ojack.xyz" target="_blank" rel="noreferrer">Hydra</a> (both AGPL), each fetched from its own address.
        </Text>
      </Stack>
    </main>
  );
}

The sets are a public gallery with author-owned writes, the same rule as the pad's:

access.js

js// set: a Strudel pattern and the Hydra sketch it drives. Anyone signed in keeps their own; everyone can read the gallery.
export function beatVisuals(doc, oldDoc, user, ctx) {
  if (!user) throw { forbidden: "sign in to keep sets" };
  if (doc.type === "set" || (doc._deleted && oldDoc && oldDoc.type === "set")) {
    const author = oldDoc ? oldDoc.authorHandle : doc.authorHandle;
    if (author !== user.userHandle || (!doc._deleted && doc.authorHandle !== user.userHandle)) throw { forbidden: "sign your set with your own handle" };
    return { channels: ["sets"], grant: { public: ["sets"] } };
  }
  throw { forbidden: "unknown document type" };
}

The text-to-sketch prompt, verbatim

lib/sketch-prompt.js above is the system prompt for a text box that turns a sentence into a sketch, or an app that rewrites the sketch on request. Use it as written: it names only functions the engine has, with their argument order, its four examples were run on the engine as the doc was written, and it keeps the amounts in the range that stays watchable (a feedback scale past 1.05 floods the frame). rewritePrompt(request, code) appends the ask and the sketch that is showing; the callAI schema is { name: string, code: string }, and the app shows code through run(), so a sketch the model got wrong surfaces as a message while the last good one plays on.

The stage and its lifecycle

• One canvas, made once by the module, moved into the stage. attach(el) appends it and starts the loop; the function it returns cancels the loop and takes the canvas out. A remount appends the same canvas again, so the WebGL context and every output survive (a second new Hydra on a second canvas is a second GPU context and the first one's sketch is lost).
• The mount effect guards the async attach: when the component unmounts before the engine arrives, the late detach runs at once (the gone flag in the examples), so a fast remount never leaves two loops.
• The canvas renders at 1280×720 and CSS scales it into a 16:9 stage (aspect-video). That is sharp on a laptop and light enough for a phone; hydra.setResolution(w, h) changes it when an app needs more.
• hush() blanks every output and forgets update; a "Blank" button and a scene change between sets call it. The loop keeps running, so the next run() shows at once.
• Full screen is the stage element's own requestFullscreen(), which the vibe frame allows; the canvas fills it and keeps its aspect.

Mobile

The stage sits at the top at full width, and the editor or knobs follow under it. Typing a sketch on a phone is slow, so a phone-first app leads with example chips, knobs and a "describe it" box, and keeps the editor one tap away. The camera button names what it does ("Use my camera"); the front camera is the default. Phones render Hydra at highp precision (the engine picks it on iOS); a 1280×720 canvas keeps frames smooth on a recent phone.

Persistence — the sketch text is the document

Save the sketch text itself (code, or the knob values a sketch compiles from) with a name, the author's handle and a timestamp; the engine rebuilds the picture from it on every run, so a saved document is tiny and remixable. A set that pairs music and picture saves both strings. The shared gallery is one useLiveQuery over the sketch type, and the access rule above makes it public to read and author-owned to write; a signed-out visitor plays with the pad and is invited to sign in at "Keep".

Shapes a Hydra vibe takes

• A sketch pad — stage, editor, examples, a prompt box that rewrites the sketch, saved sketches. The starter above.
• A video synth — knobs that write the sketch, the camera as a source, the written sketch shown underneath. The second example above.
• Beat visuals — a Strudel pattern and a Hydra sketch played together, the beat kicking a.fft. The third example above.
• A VJ deck — a row of saved sketches as pads, hush() between them, a full-screen stage on a second screen, the room's sound through listen().
• A photo booth — the camera through kaleidoscopes and feedback, and a "Keep this frame" press. Add a snapshot() to lib/hydra.js that wraps the engine's hydra.getScreenImage((blob) => …), which captures the next frame as a PNG blob; a plain canvas.toDataURL() reads blank, because the WebGL canvas is cleared after every frame.
• A jam room — the sketch document is shared, every member's edit is a version, and the gallery doubles as the set list.

Common gotchas

• Every name in the sketch is undefined — the sketch went through hydra.eval() with makeGlobal: false. Run it through run() and its with blocks.
• The microphone prompt appears on load — detectAudio was left on. Set it false and call listen() from a press.
• a.fft reads undefined — the sketch ran outside run(), which passes the module's audio as a.
• The picture floods to one colour — a feedback chain scales or blends too much. Keep src(o0).scale within 0.95 to 1.05 and the blend under 0.2.
• An image or video source stays black — the URL is not CORS-open. Use a host that sends Access-Control-Allow-Origin (jsDelivr, a vibe's own assets), or the camera.
• The camera light stays on after leaving — the stop function from startCamera() was dropped. Keep it in a ref and call it in the cleanup.
• Two pictures fight or the frame rate halves — two loops or two engines. One memoized loader, one canvas, one attach per stage.
• A sketch built from typed text breaks on a quote — strip or escape the quotes before dropping a person's text into a sketch string.