What a small decision model sees in a blurred picture: building Prompt Guess with Jev and Claude Haiku 5.5

Prompt Guess is a game in the AIKIZI Lab. A picture starts as a coarse mosaic and sharpens in five steps. Under it are four short prompt lines, one true and three decoys. A person races two AI models to name the true line first. This post is about how we built the race, what it costs, and what two very different models do with a blurred picture.

The two rivals are Jev, the decisions model from TypeSafe, and Claude Haiku 5.5 from Anthropic. One is a small model built to choose between options; the other is a general model that can see. We wanted to know which reads a picture sooner, and how to show that fairly.

What is TypeSafe's Jev?

Jev is a decisions model. You give it a question and a list of options, and it returns one choice with a probability for every option. It does not write prose. It is built for fast, cheap, structured decisions, such as picking one of a few options.

We use its image preview build, which reads pictures between 256 and 1568 pixels a side. It is billed per input token, at about $42 per billion input tokens, and a 1024 pixel picture comes to roughly 3,500 tokens. For four short lines and one picture, the picture is almost all the cost.

How did we ask Claude Haiku 5.5 the same thing?

Claude Haiku 5.5 is a general model, so we asked it for structured JSON with a choice and a confidence, at medium effort. Same picture, same four lines.

Haiku's confidence is a number it writes about its own answer; Jev's is a distribution over the options. They are not the same kind of number, and that shows below.

Why does the game use pixels and not the picture's description?

Every picture in the game has a decode on AIKIZI, a long text description, and the true line is a short piece of it. We tried letting Jev read the decode text instead of the pixels.

It did not work. Reading the text, Jev was right 100 % of the time. The true line is taken from that text, so matching it is a lookup, not a reading. That is a leak.

So the game is pixels only: the models see the frames the player sees and nothing else.

What does each blur step cost a vision model?

The picture is shown as a mosaic whose blocks halve every 1.5 seconds: 64 pixels across, then 32, 16, 8, then sharp. The first version started one step coarser, at 128, and we ran a spike on 16 pictures with that step still in, to see where each model starts to read.

Both curves climb the same way: below one in four, which is chance, at 128, shapes starting to count between 32 and 16, close to nine in ten right by 8. That was the curve we wanted, with room in the middle for a person to win.

How do you make a multiple-choice round fair?

A round is only fair if exactly one line is true of the picture. Many pictures share colours and moods, so a decoy from another picture can easily be true of this one too.

We built the rounds in two passes. The first is a scorer. For the true line it prefers short pieces that are distinctive: a colour, a material, a subject noun. For decoys it takes lines from other pictures and rejects any that are also true of this one, for example a decoy whose colour is in this picture's decoded colours.

That check reads the decode's words, not the pixels, so a decoy can pass it and still look like the blurred frame. In one round the decoy "Cool monochrome palette of slate blue" passed, because grey was not among the picture's decoded colours. At 64 that frame is mostly blue sky, which the decoy fits well. That is the trap a player meets.

The second pass is a check by a model. Jev looks at each round at sharp and must pick the true line at 80 % or more. If it does not, one decoy is swapped and the round is checked again. Of 294 rounds, 264 passed first time, 29 needed a swap, and 1 stayed ambiguous. That one is never drawn.

Using Jev to check rounds it then plays could look circular. But the check is only at sharp, so it says nothing about the blurred frames, which are the part being raced.

Why were precomputed answers not good enough?

The first version asked both models once, when the deck was built, and the game replayed their stored answers. It was cheap, but it was not a race. The answer times were measured on another day, and every player raced the same recording.

A rival that is called live should be live. So for the first games of the day the models are asked while you play, about the round you are about to see.

How does the live race work?

When a round starts, the page sends one POST for the next round to our Cloudflare Worker. The Worker makes ten asks: five frames times two models. It runs them six at a time, and each ask has eight seconds before it is given up.

The stored mosaics are tiny. At 64 a picture is only about sixteen blocks across, so the file is about sixteen pixels wide, far below Jev's 256 pixel floor. The Worker scales each mosaic up with nearest-neighbour, which keeps the blocks hard-edged, as the player sees them, instead of smoothing them into a soft blur. The sharp frame comes straight from the image CDN.

The answers come back in about 4 to 6 seconds, which is less time than a round takes to play. So the answers for round two are ready when round two starts.

The cost is about $0.0013 per round, about 1.3 cents for a ten-round game. To keep that bounded, every visitor gets one free game, and every game after it costs 2 credits. The free rounds are counted in D1 per IP address and across everyone, with a daily cap. A game bought with credits gets a ticket in D1 with its ten live rounds, and each race spends one; the ticket answers only the player who bought it. Games bought from one network have a daily ceiling too, because accounts are cheap to make. When a cap is reached, an ask fails, or the counters cannot be read, the Worker asks nobody and the page replays the answers stored in the deck instead, and says so. The game never stops because a model is slow or a budget is spent.

How fast are Jev and Claude Haiku 5.5 on one frame?

Jev answers a frame in roughly 250 to 450 milliseconds. Haiku takes 1.3 to 2.5 seconds. Both are fine for a game that runs on a 1.5 second step, because the asks happen before the round starts.

On the player's clock the gap still shows: if both are first sure at the same step, Jev buzzes about one to two seconds earlier.

How do you compare a person and two models on one clock?

The person watches the picture sharpen in real time. The models are asked about all five frames at once, out of band. A fair reveal has to admit that.

Our answer is a shared clock. A rival buzzes at the first step where it names the true line at 80 % or more. Its buzz is placed at the time that step appeared on screen, plus the time its own answer took. Step two appears at 1.5 seconds, so a model that was first sure at 32 and took 319 milliseconds buzzes at about 1.8 seconds. The player's buzz is when they clicked.

The reveal then shows one stopwatch for all three. It does not claim the models were watching alongside the player. It answers a narrower question: at which step could each one name the picture, and how long did it take to say so.

One part of this is kinder to the rivals than to the player. A player who picks a wrong line scores zero. A rival that is confidently wrong early is just not counted yet. In one round Jev was 85 % sure of a decoy at 64, and then buzzed on the true line at 32.

How early does each model read a picture across the whole deck?

Across all 294 rounds, here is the first step where each model was at least 80 % sure of the true line.

Jev was sure first on 138 pictures and Haiku on 26. They tied on 129. By 16, Jev was sure on 106 rounds and Haiku on 64. Two changes got us here. A second pass took away easy wins on the coarsest mosaic: where Jev was already sure there, usually because the true line named the colours, one decoy was swapped for a line naming the same colours, kept only if Jev still found the true line on the sharp picture. Then the 128 step went: at 128 a picture did not read as a picture to a person at all. The game now starts at 64, where Jev is sure on 20 rounds and Haiku on 3.

Is a small decision model better than an LLM at reading pictures?

For this task, a choice between four short lines on a blurred frame, the small model was quicker to commit and quicker to answer. That is not the same as better at seeing. In the spike both models were right about as often at most steps. The gap opens in confidence.

Jev returns a probability for each option, and on this task those probabilities reached 80 % earlier. Haiku writes its confidence as part of its answer, and it is more cautious with it: on 44 rounds it never reached 80 % on the true line, even at sharp. If your product needs a model to say "this one, and I am sure", the shape of the output matters as much as the model behind it.

Being sure early has a cost. Jev's early certainty is sometimes about a decoy that shares the picture's colours, as in the round above. On a few dozen coloured blocks, a high probability is a statement about the blocks, not about the picture.

Cost was the easy part: ten asks for about an eighth of a cent. Speed was harder, and the way through was to ask before the player needs the answer, then be honest about when it arrived.

What comes next for Prompt Guess?

These are plans, not promises. We would like to let you play on your own picture. We would like a wider pool than the 300 or so showcase pictures from AIKIZI Explore the deck uses today. And a leaderboard, for the shared first game of the day, would make scores worth comparing.

Play Prompt Guess    Browse the Lab

Jev also plays a word game in the Lab: the Jev word race, where it walks from one word to another one link at a time. There is also a player's guide to Prompt Guess. More posts are on the AIKIZI blog.

FAQ

What is TypeSafe's Jev?

A decisions model: give it a question and options, and it returns one choice with a probability for each. Its preview build reads images 256 to 1568 pixels a side.

How much does a live round of Prompt Guess cost?

The cost is about $0.0013 per round, about 1.3 cents for a ten-round game.

Why can the models not read the picture's description?

Reading the decode text, Jev was right 100 % of the time, because the true line is taken from that text. That is a leak, so the game is pixels only.

How do you stop a decoy from also being true?

A scorer rejects decoys that are also true of the picture. Then Jev looks at each round at sharp and must pick the true line at 80 % or more, or one decoy is swapped.