What the evidence
supports.
The claims below cite published experiment records. Every number links to its source. We also list claims we do not make, with the relevant results or work still pending. Publishing a result does not by itself establish a broader claim. These are selected claims and open questions, not a complete inventory of everything the lab has not established.
Corrected on
Wording clarification · . The earlier title and social preview promised “what we have not measured” and “an explicit list of what is not measured yet”; the earlier lead said “Everything else is in the list of what we have not measured”. Those statements implied a complete inventory that this page does not provide. The lists below are selected published results and open questions. Our publication standard explains this correction.
What we have measured.
Each claim below is a sentence whose numbers are read from the experiment record it cites, at build time. Each number links to that record.
On a sealed corpus of private changes, reviewed by every model through the same tool-free seat, the cheapest model,
openai/gpt-oss-120b, cost $0.000631 per review and caught 29 of 60 seeded defects; the dearest,anthropic/claude-opus-5.5, cost $0.083274 per review and caught 31 of 60. The catch intervals of all models in the experiment overlap, so on this corpus the price of a review varied far more than what it caught.Measured . Source:
data/review-frontier/results.json· the field note.That experiment's 464 counted reviews were billed $16.85832 in total by the broker, read from the broker's own per-call cost.
Measured . Source:
data/review-frontier/results.json· the field note.A reviewer asked to re-check its own findings before answering still raised 20 and 22 false alarms on clean changes in its counted runs. The pre-registered answer to whether it can talk itself out of its false alarms is no.
Measured to . Source:
data/self-verify/results.json· the field note.In the week of , the weekly broker probe found that OpenRouter answered 20 of 20 calls for
claude-haiku-4-5and 20 of 20 forgpt-4o-mini, with 0 and 0 answered by a different model than the model asked for, and 0 and 0 server errors.Measured . Source:
data/broker-probe/2026-W43.json.
Claims we do not make.
A published result does not necessarily support the broader claim listed here. Each entry links to the available record or explains what is still missing.
- That Odin's routing saves money against a fixed-model baseline. EXP 018, “Does routing by our own rulebook beat one fixed model on real tasks?”: Results are published: the record. This page does not summarise them yet.
- That Odin's per-call cost estimates match what the brokers bill. EXP 015, “Does Odin's cost estimate match what the brokers bill?”: Pre-registered; no result is published yet. Its measurement window: to .
- That the same model costs the same through every broker. EXP 016, “Does the same model cost the same through every broker?”: Results are published: the record. This page does not summarise them yet.
- That Odin has negotiated better prices with providers. No negotiated price exists. Every price the lab has paid so far is a broker's list price, a promotion or prepaid credit.
- That Odin has provider partnerships. None is signed. A lead is not a partner, and this site names a partner only after a signed agreement is on record.
- How reliable each broker is over time. A trend needs many weeks of the weekly probe, and a broker is probed only when its terms allow a published measurement.
The rule for this page.
No projection, market size or forecast appears here, and no partner is named. Each number is read from the record it links to when the site is built, and the build stops if a number is not read from an experiment record or no longer matches it (scripts/number-provenance.mjs). A sentence that compares results, such as intervals overlapping, is checked against the record the same way. These checks establish consistency with a record, not the truth of its observations, independent replication or applicability beyond its stated sample.