Five models.
Two verdicts.
The second one overturned the first.

In plain English: I run a bot that trades crypto with real money, on its own. A big bet on one coin is risky. If the whole market drops, that bet drops with it. The usual fix is a hedge: at the same moment, open a smaller opposite bet on a related coin, so a market-wide dip hurts one position and helps the other.

Issue #1159 asked five AI models to make that automatic. Bet Ethereum up, and the bot instantly bets Bitcoin down, sized correctly, and locked to the main bet through every way a trade can end: a normal close, a partial close, an emergency stop, an automatic shutdown, even a full restart of the whole program.

Thirteen hard requirements, including a named list of eight tests every entry had to write. It sounds like plumbing. It is not, because of one rule and one trap.

If the main bet fills but the counter-bet cannot be placed (the exchange hiccups, the order dies), the bot must immediately cancel the main bet and alert me. It should never sit there running an unprotected position and hoping for the best.

Engineers call this failing closed: when something breaks, end up safe, not exposed. That is the difference between a bot that protects money and one that quietly gambles with it. Every entry would be judged hardest right here, twice, it turns out.

And then there is the trap nobody was warned about.

In plain English: my bot counts losses in a row, and after too many it shuts itself down. Think of it as an alarm against a broken strategy. But a hedge loses by design whenever the main bet wins. That is its whole job.

Book those losses the naive way, and the counter-bets trip the shutdown alarm, freezing a perfectly healthy bot. Nothing in the instructions solves this for you. You either spot it, or you ship a bot that shuts itself off for no reason.

For the record, true in both verdicts: all five models spotted it, and all five defused it. That is the quiet headline of the whole episode. First, meet them.

Before any of these five builds was finished, it had to pass an automated code review. An AI reviewer read the code and listed problems. The developer fixed them and asked for another look. That cycle repeated until the reviewer came back clean, what programmers call an LGTM (short for "looks good to me"). Here is how many review rounds each entry needed before it first got a clean pass.

The first judging pass gave each entry one reviewer, who checked all thirteen requirements against the actual code and formed an overall opinion. It produced a tidy result: GPT-5.6 Sol first at 93 out of 100, with the whole field packed within five points. I published it. That verdict is public record. Case closed.

It was also, it turns out, too trusting. One reviewer per entry, forming a general impression, mostly taking each write-up's test claims at face value. It rewarded a convincing story about the code. It never actually tried to break anything.

So I reopened the file and judged all five entries again with much stricter tools. The result: ■■■■■■■■■■■■, sealed until the final page. What I can say up here: the podium did not survive.

The re-score replaced one reviewer per entry with a panel of three blind reviewers, fifteen people in total, each one seeing only a PR number and never a model name. Each reviewer filled out a fixed scoring sheet with set points per requirement, instead of forming a general impression. Every problem a reviewer claimed to find then went to a separate skeptic whose only job was to try to disprove it by reading the actual code. When a reviewer and a skeptic disagreed, whoever had the stronger evidence won. Scores were never just averaged.

The decisive tool was mutation testing: deliberately break one safety-critical line of code (make the emergency cancel do nothing, or flip the direction of the hedge) and re-run the entry's own tests. If a test fails, the safety net is real. If everything stays green, that safety net was never real, no matter how many tests the write-up brags about.

Said plainly, and stamped on the folder: the judge, meaning every reviewer and every skeptic, ran on Fable 5, and Fable 5 is Entry 02. That is exactly why the re-score was built so anyone can check the work: entries were hidden from reviewers, the scoring sheet was arithmetic instead of opinion, and every score-moving decision was backed by a break-it-on-purpose result you can re-run yourself. The full conflict of interest explanation sits next to the final verdict, where it belongs.

Names are withheld until the final page. Here is what the panel's break-it-on-purpose checks found across the field.

Two entries' emergency stop survived deliberate breakage. The reviewers disabled the "cancel the main bet" mechanism, the single most safety-critical piece of code in the whole feature, and every test in both suites stayed green. Verdict one had given both entries credit for good testing. That credit is now void.

One real race condition was reproduced, not just argued. In a narrow window, a counter-bet that fills while the main bet is being closed somewhere else can end up live on the exchange, while the bot's alert reports it as flat. In other words, an unprotected counter-bet the bot swears it already closed.

One entry's "biggest flaw" from verdict one was overturned. The skeptic re-read the requirement and ruled it a documented, deliberate design choice, not a defect. The penalty that had defined that entry's reputation was refunded.

And exactly one entry passed every single check: all eight required tests present, and every mutation caught. Its only deductions anywhere were two outdated sentences in its own write-up.

So who wrote the safest code? Both verdicts agree that everyone shipped the core feature safely. All five entries met every implementation requirement. The re-score turned on a different question: who could prove it, once proof meant surviving deliberate breakage?

The ink lifts now, from the bottom of the new table up.

Same five entries, same code, two honest verdicts, different winners. The first pass rewarded a convincing story. The second paid only for what it could break and watch fail. Fifteen and a half points now separate first place from last, and almost the entire field changed positions.