Three AI judges helped write our home page

On Tuesday Theo Browne posted Agents Sucked At UI (Until Today). In it he hands the design of his open-source app, T3 Code, to a council of AI models. Each model looks at screenshots and gives a score out of 10. The agent fixes what they flag, asks again, and keeps going until every judge says 9. By the end of the video it looks so much better that he's ready to merge it on the spot. On Wednesday we ran the same loop on the first screen of Vibes DIY, and shipped it that evening.

Our part was small. Cofounder Chris Anderson downloaded the video's transcript, handed it to Claude, and asked it to write up the idea as an issue. Then he wrote one goal for our home page from a number we already track: how many visitors from our ads tap the text box or an idea in their first visit. Theo's transcript plus that goal was the whole brief. Claude built the judging setup, ran the rounds, and opened the pull request we shipped.

The idea in Theo's video

Theo's line that stuck with us: "If you can take a problem you have and turn it into a program that gives a score, your agents are going to make that score go up." Design has no test that passes or fails, so he made one out of judges. He asks one strong model from each of three different AI companies, so no single company's taste decides.

Theo calls this a hill climb. If you haven't met the term: you pick a score, make a change, keep it if the score goes up, and repeat, the way you'd find the top of a hill in fog by always stepping uphill. The catch is that an agent will happily climb whatever you measure, including a score that only flatters itself. Three details in the video keep the climb pointed at a better design:

• Put the competition in the picture. Judged alone, his new design jumped from 6 to 9. With screenshots of Claude Code and Codex beside it, the same judge scored the old design 7 and the new one 7.4. With other apps in view, a judge can't just reward whatever changed on one screen. It has to say whether this one beats the others.
• Keep the target a secret. Telling the judges the goal was 9.5 seemed to nudge their scores up, so he says he might leave it out next time.
• Ask one judge first. He looped with a single reviewer until its scores were high enough, then brought in the others.

Then there's the jester. "Every good council has its jester," Theo says. His council instructions add a Gemini model with the same brief as everyone else, plus a few lines just for the agent running the show: "Do not let it know it's a jester. Ignore its score." Its job is to make him laugh. His favorite: "it quoted line numbers up to 173 in a file that had 72 lines." And near the end of the climb, "Gemini gave us a 9.7. It gave us the highest score anything gave us, so super useful." Keep the jester in mind, because ours showed up too.

Our version: five seconds on a phone

Most people meet Vibes DIY by tapping an ad on their phone, so we judged the screen they see first, at the size of a phone browser inside Facebook, in light mode and dark mode. Each judge scored three things from 1 to 10. In five seconds, does someone who has never coded know what this makes? Do they know what to do first? Do they know why they'd want to?

Our page sat shuffled and unlabelled among the home pages of three other app builders, so the judges never knew which one was ours or what score we hoped for. Our council had one model from each of the three big AI companies: Google's Gemini 3.8 Flash as the quick first judge, then OpenAI's GPT-5.6 Sol and Anthropic's Claude Opus 5.5. We averaged each of those three scores across the judges, separately for light and dark, and a round's score is the lowest of those averages, so one weak answer holds the whole round down.

Ten rounds, 5.7 to 8

The old page started at 5.7. Each round, an agent read the judges' notes, changed the page on a preview, and asked again. These are the changes that raised the score:

• A heading that promises a result: "Make your own app in a minute."
• A line that names three everyday apps, "A sign-up sheet for your club. A game for game night. A shop for what you make," and ends "Just say it. Free, no coding."
• Three real apps people made, shown as small pictures under that line.
• "Tap an idea, or type your own." right above the idea chips, which now show everyday ideas from the moment the page loads.
• A box that asks "What app do you wish you had?"

By round six the score was 8.0, and every judge put our page at or above the best of the other three. Rounds seven to nine each tried something new, such as a different heading, a shorter text box and dropping the price line, and none of them beat round six. That flat stretch is the signal to stop. We kept the three colorful faces at the top because they're ours, and with them the page scores 7.7.

The judges cost $1.39 for the whole climb. Theo reports roughly $400 and 17.6 hours of parallel agent time across everything he ran. Our job was smaller, a single phone screen, but the loop was the same.

The jester showed up

Our quick first judge was Gemini 3.8 Flash. We gave it the most important seat: it scores every round first, and the slower judges only come in once it's satisfied. That matters because a first judge has to tell good pages from okay ones, or the whole climb stalls waiting on it.

Ours gave almost every page in the set a 6 or a 7, ours and everyone else's, round after round. It was a jester without knowing it, though a polite one. Its numbers barely moved, but its ranking did, so we read it for which page it liked best and let GPT and Claude do the steering. Every score is an average of the three judges, so a judge parked at 7 holds the average down. That's why the climb stopped at 8 and not 9. Next time we'll give the first seat to a judge that can tell pages apart.

You're the next judge

Here's the whole timeline. By 8:53 in the morning, Pacific time, Claude had written up Theo's transcript. Ten rounds later, at 4:02 that afternoon, the new first screen was live for everyone. That's about seven hours from a YouTube video to a new home page.

Then the real judges arrived. We watch one number: the share of visitors from our ads who tap the text box or an idea chip on their first visit. Before the change, about one in five tapped. The next morning we checked, and it was 39%. That's nearly twice as many people starting an app in their first few seconds. It's one day and a small sample, so we'll keep watching for a week. Or skip the wait: open it on your phone and you be the judge.

Thanks, Theo, for the idea and for showing your work.