We Know Too Much About How Mercury Doesn't Think

When you describe an app on Vibes DIY, the first thing that happens is invisible. A model reads your sentence and writes a plan: what the app keeps track of, what screens it has, who gets to see what. Then a second model builds the app from that plan. The plan is the part you wait through before anything appears on screen.

Mercury is a language model from Inception that writes differently from most. Instead of producing one word after another, it drafts a whole answer at once and refines it in a few passes, which makes it very fast and very cheap. Put it in the planner's seat and your first app starts two to three times sooner, and the whole build costs 20–40% less. We wanted that badly.

So we spent a week trying to make it plan as well as our careful model. It didn't work, thoroughly, and the job we found for it instead is live now: when you describe an app, your first version is on screen in about half the time. Here's how we got there.

Round after round

We judge a plan the only way that counts: build the app from it. Each round, the same set of app ideas gets planned twice, once by Mercury and once by the careful model, both plans get built, and a judge compares the finished apps side by side without knowing which planner wrote which. (Our eval showcase is the kind of app these rounds build.) Between rounds we changed the prompt, the examples, the format, the order of the questions.

The score moves around. The direction doesn't.

It isn't the format

The obvious suspect was us. Mercury's plans come out in a format we designed for it, tighter and more fill-in-the-blank than the careful model's. Maybe the format was starving the builder.

So we ran the control: the careful model wrote its plan in Mercury's format, and we built from that. Against its own usual plans it went 8 wins, 10 losses and 5 ties, which is close. Going the other way, Mercury writing in the careful model's looser prose beat Mercury in its own format 6 to 3, with 2 ties. The format costs a little. The model costs the rest.

The cabin weekend

The example that made it click was an app for a group cabin trip, where friends split the costs. Mercury's plan was fluent and well organized. It had the people, the expenses, a balance screen, all where you'd expect them.

And it counted the things people offered to bring as money they had paid. Volunteer to bring supplies, and the app would settle up as if you'd handed over cash.

Every sentence in that plan reads fine. The structure is right. The cause and effect is wrong, and a builder handed a confident plan builds exactly what it was told. That is the whole pattern in one app: fluent, structurally right, causally wrong.

So we asked it for stories

A story only works when each event follows from the one before, so it's a clean place to watch cause and effect. Ask Mercury for a known tale and you get every beat in order. Ask it to invent, and it makes a list: an Odyssey in a strange new setting walked its episodes as "First… Next came… Deeper still… Finally."

It can also be lovely. One ghost story, about a night archivist who finds an unlabeled reel with his own voice on it, ends like this:

He picked up the next box. A small card fell out. It read: "Session 415 - To Be Recorded." He put on the headphones.

That's a real turn, and it was one story out of fourteen.

We asked it to plan the chain first, because this happened, therefore that must happen next, and cause and effect did show up. A town loses its clock tower, everyone goes by their own watch, the watches drift apart, and the council declares "a moratorium on punctuality." But a link can be fluent and wrong:

Because the crew stops working, the ship's rudder rusts shut during a storm, as no hands were available to grease the gears or inspect the hull.

A rudder doesn't rust shut in an afternoon because the crew sat down. It's the cabin weekend again: the grammar of a cause with nothing inside it.

As a judge of other plans, it mostly picked whichever one it read first, in 62–70% of its decisive calls. It praised a plant-care plan for storing each plant's last-watered date as its own field, right next to a care log that already records it. It read as thorough, so Mercury liked it.

Let the sprinter sprint

Here's the turn. Every experiment asked the same thing of Mercury: be the careful model, only faster. Every answer was the same. It is fast, it is broad, it writes a fluent first impression of anything in a couple of seconds, and it doesn't check that impression against itself.

That's a real skill, just a different one. It's what a usability tester gives you when they think out loud: someone sits down with your sketch and says "so I guess this is where I'd put what I'm bringing?" Those guesses are valuable because they're quick and unfiltered, and nobody mistakes them for the spec.

So we gave Mercury that job: read the idea the way a tester does on a first look, out loud, with every hunch labeled as a guess. On the cabin weekend, with nothing in the prompt about bringing versus paying, it raised the question on its own both times:

I'd guess it's an even split of the total costs, but I'm not sure if items brought are counted as payments.

Both of those plans split only the expenses and flagged the choice for the builder to confirm.

We also checked whether the first read gets better if Mercury reasons privately before it talks. Across 120 readings it didn't: thinking first took about 2.3 times as long, cost about three times as much, wrote around 40% fewer visible words, caught no consequence the quick read missed, and added one to three confidently wrong lines to each set of plans. The quick read stays quick.

Building apps from the first read

Plans on paper are a promise. Apps are the test, so we built them.

The think-aloud read, handed straight to the builder, went 1 win and 5 losses against the careful model over six builds. The judge kept naming whole flows the apps didn't have. A tester's hunches are a great start, and a builder needs a little more shape than hunches.

So we gave the first read a shape that suits a fast model: user-story cards. Each card is one thing a person does in the app, written the way a person says it, with any rule in plain English and no units or formulas:

As a cabin guest, I add what I am bringing to the list, so that others know I have contributed.

Then a second fast step reads the cards and adds two lines under each one: what I see when I open this, and what I expect when I do it. That step uses Inception's coder model doing real fill-in-the-middle, writing into a gap with the card above it and the rest of the plan below:

Expect: "When I do this, I expect the expense to appear in the list with my name and the amount I paid."

Cards with those lines beat plain cards 6 wins, 3 losses and 3 ties over 12 builds. Against the careful model, on the nine openings both rounds shared, they went 2 wins, 3 losses and 4 ties. That's the closest Mercury has ever come, and it arrives fast: the first build starts at about 11 seconds instead of 23, and the whole first turn finishes in about 70 seconds instead of 145.

Two plain things made it dependable. Patient retries wait out a busy moment at the provider instead of giving up, and our own key with Inception gives the calls their own lane. With both, Mercury planned 24 builds out of 24.

What a fill-in step is good at

The fill-in step taught us something worth knowing about fast models generally. It can't add logic. If a card has the rule wrong, the lines under it narrate the wrong rule faithfully. What it does beautifully is describe the experience inside a structure someone else got right: the form you'll see, the list that updates, the balance that changes when you add an expense. That's exactly the voice a builder needs to make screens that feel the way you'd expect.

What happens when you describe an app now

Mercury and the careful model start on your idea at the same moment. It's a real race, and whichever claims the build first builds your app. In our test round, Mercury's quick read started a build in about 11 seconds where the careful model took about 23, and the first turn finished in about 70 seconds instead of 145. If Mercury stumbles, the careful model carries on with no delay.

When Mercury wins, the careful model's plan still gets built, as a second app at its own address. When it's finished, the agent drops a short ⭐ link into your chat. Nothing waits for it: open it if you're curious, or keep going with the app you have. The ⭐ app's chat links back to your first one with a ⚡, and you can keep talking to either.

The agent also knows the first build came from a quick guess that wasn't its own. The builder is told "Guesses only; the creator's words win," and the agent reads the first read the way you'd read a tester's notes: where something differs from what you asked, it mentions it and asks whether you'd like it changed. It only changes things on its own when the app is broken.

What the week bought us is a better question than the one we started with. It was "can a fast model replace the careful one?" It's now "where does a quick first read help?" The answer is right at the start, with the careful version a ⭐ away.