Field notes.
A field note records one experiment or change: what we set out to test, what actually happened and what is still open. The latest is at the top.
NOVEMBER 2026
OCTOBER 2026
- FIELD NOTEEXP 021: Kill the database mid-session, does anyone notice? Lab 03 (failover), historical plan and amendmentsLab 03 (failover): while the agent space writes proof records, the database's primary fails. Which store keeps every acknowledged write and the feed intact, with the shortest write outage and the fewest moving parts: a MongoDB replica set, or PostgreSQL with Patroni and etcd? Two rounds of advocate review fixed the plan first. Historical plan; the counted result is now linked with unresolved strict-ack and disturbance limits.
- FIELD NOTEWhen a cached yes answers the opposite questionA fixed real-embedding experiment tests policy versions, opposite questions and memory corrections through MongoDB’s AI adapters.
- FIELD NOTE · EXP 020EXP 020: MongoDB on its home ground: where does it fit Odin? Pre-registered, S2 complete, amended five timesOur two earlier MongoDB studies tested what PostgreSQL does well. EXP 020 tests what MongoDB is built for, on four problems Odin has on record, with a reviewer for each database checking the plan first. The verdict is per problem; there is no overall winner. Pre-registered; S2 is complete, and amendment 05 records how its run went; S1, S3 and S4 have not run.
- FIELD NOTEMongoDB beyond JSON: agents, memory and modelsReal local Voyage embeddings, agent checkpoint recovery and memory/cache checks extend the MongoDB investigation beyond document storage. Both databases agree; a platform advantage remains unproven.
- FIELD NOTE · EXP 016EXP 016: Does the same model cost the same through every broker?Almost. On 720 identical calls, Broker A and Broker C billed exactly list price for Claude Sonnet 5.5 and GPT-4o mini, and Broker B billed 10% above it on both. No broker served a different model. On Sonnet, 4 to 7 of 120 calls per broker came back without an answer.
- FIELD NOTE · EXP 017EXP 017: What does a defect caught in review cost, model by model?Four models reviewed the same 58 private changes twice through one tool-free seat. They caught between 26 and 31 of 60 defects, all inside one another's intervals. The price per review ranged from 0.06 to 8.3 cents. Under the pre-registered rule, gpt-oss-120b, MiniMax M3.1 Flash and Claude Opus 5.5 are on the frontier and Claude Sonnet 5.5 is not.
- FIELD NOTE · EXP 019EXP 019: Routing by our own rulebook, measured againThree fresh, complete runs of the routed arm on the same eight tasks, against EXP 018's fixed arms as published. Routing paid 14.3 cents per case fixed against 24.7 for always-Opus and 4.3 for always-Luna. Under the pre-registered rule, routing beats always-Opus and does not beat always-Luna. It fixed 7.8 points fewer cases than Opus, inside the rule's allowance only because its own runs varied by 11.3 points.
- FIELD NOTE · EXP 018EXP 018: Does routing by our own rulebook beat one fixed model on real tasks?Three arms did eight real coding tasks from our repository three times each in a sealed sandbox: a frozen table copied from our routing rule, always Claude Opus 5.5, and always GPT-6 Luna. Per case fixed, the routed arm paid 13.3 cents, Opus 24.7 and Luna 4.3. The pre-registered verdict is incomplete: one routed cell was cut by its spend stop, leaving two of three complete routed replicates.
- FIELD NOTE · EXP 012EXP 012: Can a reviewer talk itself out of its false alarms?No. Asked to name a concrete failing input for every finding or drop it, MiniMax M3.1 Flash kept 249 of 269 of its own findings. False alarms stayed at 20 and 22 of 30 clean changes, against a pre-registered bar of 2.
- FIELD NOTEA free model on our review benchOn a sealed corpus of 30 real defects and 30 clean changes, MiniMax M3.1 Flash's counts sat in Claude Opus's range at about 1.5 cents a review. No reviewer flags few enough clean changes to vote, and an agent-run reviewer can read its own answer key.
- FIELD NOTECan a free model find real bugs in our private code?On bugs in our private fix history that can be found from the code alone, MiniMax M3.1 Flash's first guess landed within two lines of the fix 63% of the time, against 23% for our current default. Only about 1 in 10 of the small fixes we sampled could be found that way.
- FIELD NOTE · EXP 014Where would MongoDB earn a place in Odin?A follow-on experiment tests bounded session updates and dynamic metadata queries against strong PostgreSQL alternatives, with explicit gates for a selective MongoDB primitive.
- FIELD NOTE · EXP 013Does flexible workflow data need MongoDB?MongoDB and PostgreSQL JSONB both passed a local Temporal work-order experiment: 20,000 operations, 42 correctness checks and 10 evolution suites. A MongoDB advantage remains unresolved.
- FIELD NOTE · EXP 011EXP 011 — Where does a model reviewer earn its keep? Pre-registered, not yet runThe pre-registration of EXP 011, published before any item or paid call exists: Every model-reviewer measurement so far used rules a hand-written checker already decides. EXP 011 asks the opposite: on the 18 rules EXP 007's census found no bce 0.3.1 constraint can decide, and which a code change can still break, does nina 0.34.0's reviewer tell violating changes from clean ones better than the frozen rule-filtered off-the-shelf tools, with both held to a false-reject rate of at most 15%?
- FIELD NOTE · EXP 009EXP 009: Unload leaves no trace, the resultsThe measured results of EXP 009, judged against its published pre-registration: P1 refuted: 1 of 400 H1 trials diverged from R0 (95% Wilson upper bound 1.40%). P2 supported: 0 silent-inert and 0 false-inactive kernel configurations of 120.
- FIELD NOTEEXP 009: Unload leaves no trace, the pre-registrationThe pre-registration of EXP 009, shown before the counted run: Does hot load, unload and reconfigure of research components leave the harness indistinguishable from a fresh process, and does every unmet need show up as a named inactive component instead of silently doing nothing?
- FIELD NOTE · EXP 008EXP 008: Hand over the cache, not the text, the pre-registrationThe pre-registration of EXP 008, shown before the counted run: Two small open-weight models from different families, with different tokenizers. The sender reads a change and the rules once; the receiver must make the EXP 005 gate decision from the sender's KV cache, mapped into its own, without a forward pass over that context. EXP 008 measures whether that matches text re-prefill closely enough, and fast enough, to be worth having, on public baselines only.
SEPTEMBER 2026
- FIELD NOTE · EXP 006EXP 006: nina reviews the change, measuredEXP 006, nina reviews the change, measured 1 Oct 2026: the bar was met, the manipulation check passed (3 of 180 runs diff-blind), and nina is in the spotlight. Pre-registered 30 Sep 2026; amendment 01 1 Oct 2026.
- FIELD NOTE · EXP 007EXP 007 — Which of your gate's rules need a model at all? Measured: the premise is refutedEXP 007, measured 7 Oct 2026: The median expressible share across the 7 plugins with rules is 0.0%, below the pre-registered 25.0% bar: the premise is refuted: stage 2 is cancelled and the census is the result. 175 rules that the plugins ship (145 primary, 30 sampled) and 13 controls (6 written for this experiment, 7 from EXP 005); the controls met the calibration bar. Pre-registered 30 Sep 2026; amendment 01 6 Oct 2026.
- FIELD NOTE · EXP 005Jev as a fast gate: measuredEXP 005, Jev as a fast gate, measured 29 Sep 2026: the claim is refuted; nina's reviewer met the spotlight bar, and the spotlight is held. Pre-registered 26 Sep 2026; amendment 01 28 Sep 2026, amendment 02 28 Sep 2026.
- FIELD NOTE · EXP 004System-1 decisions without lock-in: Jev vs open-weight LayaCan an open-weight model answer typed System-1 decisions locally, so a hosted model becomes a comparison rather than a dependency?
- FIELD NOTEA passing pipeline was not yet a verified release.A green release status sounds like a simple answer: the software is ready. In this rollout, it left a question unanswered. Had the release actually produced the coverage report we expected?
- FIELD NOTEWhy open the factory floor?Most of the interesting work happens before the launch announcement. A rule is written. A test fails for the right reason. A beautiful plan meets a real system and needs to change.