Provider
economics.

Odin sends model calls through several brokers and to many models. Whether that is smart, and whether it saves money, are questions we answer with experiments rather than with a sales line. This page collects what the series has measured so far. Every number on it is read from a published experiment record when the site is built, and links to that record. Questions without a published result are listed as such.

wording clarification: the earlier Evidence link “What we can and cannot claim yet” suggested a complete list. The linked page shows selected published claims and open questions.

Published results.

EXP 017: What does a defect caught in review cost, model by model?

Pooled over each model's runs (the Runs column). The interval is the Wilson interval on defects caught. Cost is what the broker billed, per review.
SeatModelDefects caughtFalse alarmsCost per reviewRunsFrontier
flashminimax/minimax-m3.1-flash30 of 60 (38% to 62%)48 of 60$0.0151362on
gpt-ossopenai/gpt-oss-120b29 of 60 (36% to 61%)42 of 60$0.0006312on
sonnetanthropic/claude-sonnet-5.526 of 60 (32% to 56%)30 of 60$0.046292off
opusanthropic/claude-opus-5.531 of 60 (39% to 64%)36 of 60$0.0832742on
cost per review, log scale →defects caught ↑ (axis from none caught)flashgpt-osssonnetopus
Each point is a model: further right costs more per review, higher catches more defects. Filled points are on the cost/recall frontier. The numbers are in the table above.

The broker billed $16.85832 for all 464 reviews. The field note gives the method, the limits and what the result does not show; the record holds every figure.

EXP 016: Does the same model cost the same through every broker?

Results are published: the record. This page does not summarise them yet.

EXP 018: Does routing by our own rulebook beat one fixed model on real tasks?

Results are published: the record. This page does not summarise them yet.

The weekly broker probe, latest week

The week of . Latency in milliseconds, measured from a GitHub-hosted runner. A broker shown by letter has terms that do not yet allow naming it next to a number.
BrokerModelAnswered5xxOther model servedp50 / p95
OpenRouterclaude-haiku-4-520 of 2000636 / 776
OpenRoutergpt-4o-mini20 of 2000568 / 717
Broker Bnot probed: its terms are under review before any measurement is published
Broker Cnot probed: its terms are under review before any measurement is published

Every week of the probe and how it is measured.

No published result yet.

These questions are part of the series. Until a result is published here, this site makes no claim about them.

  • EXP 015: Does Odin's cost estimate match what the brokers bill? Pre-registered; no result is published yet. Its measurement window: to .

How to read this page.

Each experiment's question, measure and decision rule were frozen and published before any counted call; the methods page explains how. A broker is named next to a number only when its published terms allow it, as recorded in data/provider-terms.json; otherwise it is shown by letter.

Run models or route calls to them? What the lab offers providers, and what it asks.

The build refuses this page if any number on it is not read from an experiment record, or no longer matches it. The check is scripts/number-provenance.mjs.