Our earlier review measurements put models on different seats. Flash and gpt-oss ran on a tool-free seat. Opus ran on an agentic seat that could read outside its working directory, including the answer key. EXP 017 puts four models on one seat: one request per change, the same prompt, reasoning on, no tools and no file access, all through OpenRouter on one dedicated key. Each model reviewed the same 58 private changes twice. Those changes hold 30 known defects and 30 clean cases.

The models caught about the same number of defects. The price per review differed by more than a hundredfold. Pooled over two runs, they caught between 26 and 31 of 60 defects, and every interval overlaps every other. A review cost 0.06 cents with gpt-oss-120b, about 1.5 cents with MiniMax M3.1 Flash, 4.6 cents with Claude Sonnet 5.5 and 8.3 cents with Claude Opus 5.5, as billed by the broker. Under the rule we wrote down in advance, gpt-oss-120b, Flash and Opus are on the frontier, and Sonnet is not. Opus is on it by one catch, which is within run-to-run noise.

What was fixed in advance

The question, the four models, the seat, the prompt, the scorer, the analysis and the decision rule were published before the first counted call (merged 23 October 2026, 05:45:56 UTC). The private harness that ran the reviews is pinned there by hash. Its runner refuses to start if any pinned file differs. Nothing was amended.

The result

Per model, runs a / b and pooled over both, with Wilson 95% intervals. Cost is what the broker billed, per review.
ModelDefects caughtFalse alarmsCost per reviewMedian seconds, a / bNo verdictOn the frontier
MiniMax M3.1 Flash14 / 16 · 30/60 [38–62%]22 / 26 · 48/60 [68–88%]1.5 cents187.5 / 10521yes
gpt-oss-120b15 / 14 · 29/60 [36–61%]22 / 20 · 42/60 [57–80%]0.06 cents17.5 / 190yes
Claude Sonnet 5.513 / 13 · 26/60 [32–56%]15 / 15 · 30/60 [38–62%]4.6 cents19 / 17.50no
Claude Opus 5.518 / 13 · 31/60 [39–64%]16 / 20 · 36/60 [47–71%]8.3 cents22 / 230yes

Sonnet is off the frontier because Flash and gpt-oss both catch more defects for less money. Gpt-oss and Flash are on it as the two cheapest seats, and Opus is on it because it caught the most. These are differences of one to five catches out of 60, and the per-run counts move by up to five between replicates of the same model. So the frontier this rule draws is mostly a price ranking. The catches do not separate the models.

What else the runs show

False alarms separate the models more than catches do. Sonnet flagged 15 of 30 clean changes in both runs. Opus flagged 16 and 20, gpt-oss 22 and 20, and Flash 22 and 26. No model comes near the bar of at most 2 that EXP 012 used. A secondary three-way frontier (catches, false alarms, cost) keeps all four models, because Sonnet has the fewest false alarms.

Flash's false-alarm count is partly an artefact of this seat. Flash gave no verdict on 21 of its 116 reviews. Nineteen of those ran into the seat's 32,000-token output limit while still reasoning and returned no reply; one timed out and one replied without a verdict. As pre-registered, each counts as a miss on a defect and an alarm on a clean change. In EXP 012, on Ollama Cloud with no output limit set, Flash's first pass gave no verdict on 1 and 2 of 58 changes. Its catches here (14 and 16) are close to the 16 and 16 it caught on that seat in note 005. The other three models never failed to give a verdict.

gpt-oss-120b did better here than on Ollama. It caught 15 and 14 defects, against 11 and 12 on Ollama Cloud's tool-free seat with the same prompt. On OpenRouter, its 116 reviews were served by 15 different upstream hosts. This experiment does not separate a host effect from run-to-run variation.

Cost and time. The 464 counted reviews are charged $16.86 in the results: Opus $9.66, Sonnet $5.37, Flash $1.76 and gpt-oss $0.07. Of that, $16.78 is the cost the broker reported in its responses, and $0.08 is the pre-registered upper bound for 4 attempts that got no response at all (all Flash). The dedicated key's own usage counter rose by $16.81 over the window, so those 4 attempts were billed about $0.03, below the bound. Sonnet and Opus took 20–25 minutes per run of 58 changes. Flash took 4 to 5.6 hours, because of its long reasoning and the retries.

Reading the data files. The analysis was frozen before the runs, and three of its fields need a gloss.

What it means

Measured: on this corpus and this seat, paying about 130 times more per review (Opus against gpt-oss-120b) bought 2 more catches out of 60, which is inside the noise. It also bought fewer false alarms: 36 against 42 of 60. Measured: the cheaper Anthropic model, Sonnet 5.5, caught the fewest defects and also flagged the fewest clean changes. Expected, not measured: a review seat chosen on price alone would do about as well at catching defects as any of these, and the remaining difference between models is in false alarms. A council or a cheaper prompt aimed at false alarms is a separate experiment.

Evidence and its boundary

This is a publicly pre-registered experiment on an internal corpus. The diffs come from private repositories, so cases, findings and answers are not published. The counts are, along with every run's per-case outcomes (numbered, not named), each review's billed cost, latency and served model, and a per-review spend ledger. node experiments/review-frontier/analyse.mjs --check recomputes the results from those files, and the site's test suite runs it.

The counted runs ran on one laptop, on 23 October 2026 from 05:46:13 to 11:24:34 UTC. There were 8 runs of 58 reviews each, at most 3 at a time, and each ended with the canonical review log byte-for-byte unchanged. The scorer credits a catch when the verdict is not "ship" and a finding names the defective file; it does not check the diagnosis. The fleet reviewed some of these changes live, so a model may recognise one; that is not measured.

What remains open

Pre-registered experiment. Private diffs; aggregates and per-case outcomes only. One broker (OpenRouter), whose terms we read before naming it here. Prices are the broker's on 23 October 2026.