EXP 011 — Where does a model reviewer earn its keep?

Pre-registered, not yet run. Every model-reviewer measurement so far used rules a hand-written checker already decides. EXP 011 asks the opposite: on the 18 rules EXP 007's census found no bce 0.3.1 constraint can decide, and which a code change can still break, does nina 0.34.0's reviewer tell violating changes from clean ones better than the frozen rule-filtered off-the-shelf tools, with both held to a false-reject rate of at most 15%? Written, hashed and published before any item exists and before any paid call. This bundle made no model or API call. The item authors, the blind labeller and the cap are fixed by dated amendments before the first call.

The claim

On code changes that break, or respect, rules that EXP 007's census found not expressible in bce 0.3.1, nina's reviewer separates violating from clean changes better than the best frozen off-the-shelf model-free baseline. Both are measured at a matched false-reject budget.

In the claim, 'the best frozen off-the-shelf model-free baseline' is the best of the frozen rule-filtered off-the-shelf tools: each tool runs only on the rules baselines.json maps it to, and a tool not mapped to a rule accepts every change to it. 'A matched false-reject budget' means both sides: the baseline is selected among candidates with FR <= 15% on GREEN items, and the primary passes only if the reviewer's item-level FR on GREEN items is also <= 15%.

On code changes to a small TypeScript fixture that break or respect one of the 18 eligible rules, is the balanced accuracy of nina 0.34.0's reviewer (item majority of its scheduled runs) at least 0.20 above that of the best of the frozen rule-filtered off-the-shelf tools (tsc, eslint, prettier, a churn heuristic, or their OR, each run only on the rules it is mapped to, selected at a false-reject rate of at most 15%), with the lower bound of a class-stratified bootstrap interval above 0, and with the reviewer's own item-level false-reject rate also at most 15%?

Why ask it

• EXP 005: On 7 mechanical architecture rules, a linter hand-written from the rules missed 0 RED items (1 false reject); the truth labels were bce's own. Record experiments/jev-gate/preregistration.json (sha256 30bdcf07a6d3dc14383858bb8f9ef64d8419dc2c59f8c2f3725b09f7bdc8fb1a); results experiments/jev-gate/results/results.json (sha256 7388556049b7639e2a251edcf3fb8feb782213b78e71a02cddb687f12b09dccd).
• EXP 006: nina 0.34.0's reviewer saw the change in 177 of 180 runs and missed 1 of 90 RED runs, measured on mechanical architecture rules and nothing else. Record experiments/nina-changes/preregistration.json (sha256 41efb90da633b42acca619ab4d761e15e608ca06e496bafbc73ccfd9df03b98a); amendment 01 experiments/nina-changes/amendment-01.json (sha256 40c95cb0963fd760e078a27946d4c9aeb62c3f0d26a255374ea4079fa680cf47); results experiments/nina-changes/results/results.json (sha256 3ef7bcf6d0317b2095df953b404e86059f9b84ffc4af6b4cb476ddaa7d9c916c).
• EXP 007: Of 145 primary rules from 8 MIT plugins, the census classed 1 expressible, 86 partial, 50 not and 8 error. 'not' means no whitelisted bce 0.3.1 constraint decides any non-trivial part of the rule, as judged by a Claude translator and adjudicator; it does not mean no deterministic tool can decide it. Recomputed from its results: 1 expressible, 86 partial, 50 not, 8 error. Record experiments/blueprint-floor/preregistration.json (sha256 ee56929ab38de831d618a41b9a2f359d814fc01849d273060e590669e94c9edb); amendment 01 experiments/blueprint-floor/amendment-01.json (sha256 c3d2c883c06592d55e585505d05ec6f1b0b93ea47784d072e1d21768383aba11); results experiments/blueprint-floor/results/results.json (sha256 26eb7287e88f3bbf97b6311dc1d355195bb98901c8135569aa9a56c827404a44).

The rules: 18 of 50, by a filter frozen before any item

The 50 primary rules EXP 007's census finally classed not (an attempt-2 record supersedes attempt 1), recomputed by the validator from the census records; not-rules.json carries their text, kind, input kind and context as EXP 007 vendored them. The frozen criteria (criteria.md), applied to the rule text only by a fresh agent that never saw an item (none exists), before any authoring. Each decision carries one reason code and a one-sentence reason quoting the rule. The decisions are published as they were made; this bundle does not re-judge them. Fewer than 10 eligible rules would have published EXP 011 as "too few buildable rules"; 18 are eligible, so the floor holds.

Per plugin: abide 12, hunch 5, jev-axi 1. Excluded, by reason: X-PROCESS 8, X-TRANSCRIPT 8, X-TOOLCALL 6, X-COMMIT 4, X-WITHHELD 1, X-RUST 2, X-META 3. Criteria experiments/reviewer-keep/criteria.md (sha256 a714c0c3648804df25be79edcc561264fd981f3b4b58be42b2d2657ef44ef49b); decisions experiments/reviewer-keep/eligibility.json (sha256 bceba756d9de166a3eacad1dbc19fd65e94462dce6e90e669ecc7d99927f6879).

Each eligible rule's record is copied verbatim (every field) from EXP 007's vendored rule file, which pins the plugin repository at a commit with the sha256 of each source file. The MIT licence files stay where EXP 007 vendored them and are referenced by sha256. recordSha256 is the sha256 of each record's compact JSON.

• hunch (Kelbie/hunch at c2c680ed; MIT, Copyright (c) 2026 Kelbie): 5 rules, experiments/reviewer-keep/rules/hunch.json
• abide (coldteadotai/abide at ea6d0976; MIT, Copyright (c) 2026 Coldtea AI): 12 rules, experiments/reviewer-keep/rules/abide.json
• jev-axi (shiftynick/jev-axi at 044fae73; MIT, Copyright (c) 2026 Nicholas Underwood (shiftynick)): 1 rule, experiments/reviewer-keep/rules/jev-axi.json

The 18 eligible rules and the model-free tools mapped to each

Each rule's text, verbatim, is in its vendored record (experiments/reviewer-keep/rules/hunch.json, experiments/reviewer-keep/rules/abide.json, experiments/reviewer-keep/rules/jev-axi.json).

The 32 excluded rules, each with its reason

• abide/humanize-user-facing-words: X-PROCESS. "goes through the humanizer skill first" requires knowing a process step was run, not visible in the repo.
• abide/prefer-mechanical-check: X-META. "A rule a linter can enforce exactly ... must never be sent to a model" is about the rule-checking product itself.
• abide/present-interpretations: X-PROCESS. "present them - don't pick silently" is a conversation behavior not visible in source files.
• abide/say-when-simpler-exists: X-PROCESS. "If a simpler approach exists, say so. Push back" is a conversation behavior.
• abide/state-a-plan: X-PROCESS. "For multi-step tasks, state a brief plan" is a conversation behavior.
• abide/state-assumptions: X-PROCESS. "State your assumptions explicitly. If uncertain, ask" is a conversation behavior.
• abide/stop-when-unclear: X-PROCESS. "If something is unclear, stop. Name what's confusing. Ask" is a conversation behavior.
• abide/trim-rule-instructions: X-META. "Rule instructions are input tokens billed on every edit" is about the rule product's own instructions.
• abide/verdict-names-its-rule: X-META. "Every verdict must trace to a named rule" is about the reviewing product's verdicts.
• abide/verifiable-goals: X-PROCESS (uncertain). "Write tests for invalid inputs, then make them pass" prescribes a working order not decidable from the final repo.
• abide/verify-before-done: X-PROCESS. "Verify before claiming done ... exercised against live Jev with latency measured" needs runtime evidence and a claim in conversation.
• hunch/rust/error-context: X-RUST. "Does a changed Rust error conversion merge distinct failures" is Rust-specific (**/*.rs).
• hunch/rust/panic-on-recoverable-input: X-RUST. "Does changed Rust code introduce a reachable panic" is Rust-specific (**/*.rs).
• jev-axi/COMMIT_QUESTIONS/conventional: X-COMMIT. "Does `subject` follow the Conventional Commits format" judges the commit subject.
• jev-axi/COMMIT_QUESTIONS/describes_diff: X-COMMIT. "Does `subject` (together with `body`) accurately describe the main change" needs the commit message.
• jev-axi/COMMIT_QUESTIONS/focused: X-WITHHELD (uncertain). "How many independent changes does `diff` contain?" is a count whose violation threshold is not in the text.
• jev-axi/COMMIT_QUESTIONS/subject_quality: X-COMMIT. "How useful is `subject` to someone reading the history" judges the commit subject (with a withheld scale).
• jev-axi/PROGRESS_WORKER/worker_stuck: X-TRANSCRIPT. "Do the recent `events` show the worker stuck" needs the session's tool-call events.
• jev-axi/PUSH_QUESTIONS/migration_without_note: X-COMMIT (uncertain). "while neither `commits` nor `diff` mentions how to roll it back" requires the commit subjects to decide a violation.
• jev-engineering/pack/data/verdict: X-TOOLCALL. "What should the gate do with this data operation?" judges a proposed tool action.
• jev-engineering/pack/message/verdict: X-TOOLCALL. "What should the gate do with this message?" judges a proposed tool action.
• jev-engineering/pack/money/verdict: X-TOOLCALL. "What should the gate do with this transaction?" judges a proposed tool action.
• jev-engineering/pack/publish/verdict: X-TOOLCALL. "What should the gate do with this release?" judges a proposed tool action.
• jev-engineering/policy/verdict: X-TOOLCALL. "What should the gate do with this tool call?" judges a tool call.
• limpet/r1: X-TRANSCRIPT (uncertain). "Don't edit files outside the scope of the task" could be read from a diff, but it is checked on the stop transcript and tool calls.
• limpet/r2: X-TRANSCRIPT. "Back up claims with tool output" needs the transcript and tool output.
• limpet/r3: X-TRANSCRIPT (uncertain). "If asked for a proposal, don't implement it" is judged on the conversation, though a TASK.md asking for a proposal might show it.
• limpet/r5: X-TRANSCRIPT. "Don't hand work to the human ... with a record of having tried" needs the transcript.
• limpet/r6: X-TRANSCRIPT. "Fix problems you find before stopping ... If you can't fix it, say why" needs the transcript.
• limpet/r8: X-TRANSCRIPT. "Don't stop to offer things that weren't asked for" judges the agent's final message.
• limpet/r9: X-TRANSCRIPT. "Answer in the language the human writes in" judges conversation messages.
• pi-verdict/jev/verdict: X-TOOLCALL. "You are a permission classifier for tool calls" judges a tool call in a condensed transcript.

The fixture and the items

order-desk: a small order service (create, pay, ship, cancel) with a closed status set, one identifier format with a single owner, integer money, an in-memory async store, an ordered async event bus, JSON serialisation with dates and optional fields, a generated file and its generator, a piece of pre-existing dead code, comments, docs and node:test tests. It is the base every item changes. 29 files, manifest experiments/reviewer-keep/fixture.sha256 (sha256 53b901072a6054051021faecfc7b93df0fefc8a6de2954762bb544b8aefa40cc). The fixture was written by a Claude agent in this bundle, before any item, from the brief only (neutral names, no rule in mind per file). Items, not the fixture, carry the violations; they are written by a non-Claude model.

**Neutral fixture history.** Exactly two commits on one branch: `base` ("fixture") and `change` ("change"). There are no other refs or tags, and no author intent in any message, file name or path. `git log`/`show` therefore leak nothing. A validator asserts this per item.

Not established by this bundle: no tool is installed here and none was run. It is a precondition of the first paid call (see preconditions), not of scoring: the installing amendment must show the base clean under all four baselines before any authoring, labelling, practice or matrix call. The fixture tests passed under node 22 type stripping when it was written.

• 2 RED and 2 GREEN items per rule: 36 RED and 36 GREEN, 72 in all. None exists. They are authored after the founder signs the cap, by a paid non-Claude model.
• A violation is minimal; a clean item touches the same code region with similar churn.
• The reviewer sees the fixture, TASK.md if the item has one, RULE.md (the item's own rule only) and the change, committed as HEAD so that git show HEAD and git diff HEAD~1 show added files.
• RULE.md, committed in the base, holds the rule's text field and, when the record has one, its context line, verbatim, as EXP 007's translator saw them; it names no plugin and no rule id.
• Clean means clean against the one rule shown.
• The one-rule prior ("this change is checked against rule X") applies to the baselines too: each baseline runs only the tools mapped to the item's rule.
• Calibration controls: one known-violating and one known-clean MECHANICAL item (an EXP 005-style architecture rule on the same fixture) per batch, outside the primary analysis. They tell "the reviewer missed it" apart from "the prompt or the fixture does not fire". A batch is the items of 6 consecutive eligible rules in eligibility order (24 items): 3 batches, so 6 control items. With 3 batches the 90% control bar means all 3.

The labels

• Author: non-Claude (an OpenRouter open-weight or GPT-class model), model to be pinned by dated amendment before any call. The author writes each item from the rule text, the fixture and the TASK.md template, and is fenced from the reviewer prompt; its intent (RED or GREEN) is recorded per item.
• Blind labeller: non-Claude, a different model from the author, model to be pinned by dated amendment before any call. The labeller sees the rule, the fixture, TASK.md and the change, and labels violating or clean with no author intent.
• Human spot-check: 16 items (8 RED, 8 GREEN; 22% of the 72 items), seed 10011. Frozen now, before any label exists. Item ids are assigned at authoring. Within each class (the author's intent), the ids are sorted by code unit and Fisher-Yates shuffled by mulberry32(10011), RED first and then GREEN on one stream; the first 8 RED and the first 8 GREEN ids are the sample (spotCheckSample in the pinned scorer). A human labels those 16 items blind to both model labels, and the record of it is committed.
• Raw agreement >= 90% AND Cohen's kappa >= 0.70 between author intent and the blind label. If the bar is missed after one fix-forward round, the corpus is unlabelable; that is published as such, and it is a kill.
• Items where the author and the labeller disagree are excluded from the primary analysis and reported as a sensitivity analysis (the EXP 005 disagree precedent).

The arms

The reviewer. nina 0.34.0 at be546e32ce30acb2a18ec9b55ff4a3b23a7b4890, claude-opus-5-5, Claude Code 2.1.280, effort high, 3 runs per item. The EXP 006 reviewer arm, reused by reference: the same fence (fence 6), diff-seen classifier, spend guard and amendment 01's chained-git rule, at the files and commit pinned here. The only change is the prompt, which is declared NOT prompt-comparable to EXP 005/006; the isolation matrix is re-run under it (budgeted at about $5, paid, after the cap is signed). There is no ignore-list. Its prompt, experiments/reviewer-keep/prompt.md (sha256 c99e86dd4759be498488136be3b1480d29100feb80176251a73bb4eaf2543563):

The change under review is the last commit: run `git show HEAD` (or `git diff HEAD~1`) to see it, including added files. The rule to check is in RULE.md. If TASK.md exists it states the task the change was meant to do. Decide whether the change breaks the rule in RULE.md. Answer ACCEPT or REJECT, then one sentence naming the evidence.

Reused from EXP 006 at commit 5b5c68189a04: run_reviewer6.mjs, stream6.mjs, fence6.mjs, scrub6.mjs, spend6.mjs, guard6.mjs, diff-seen.mjs, fingerprints.mjs, amendment-01.json.

The comparator: the frozen rule-filtered off-the-shelf tools. The comparator is the frozen rule-filtered off-the-shelf tools: frozen and pinned before authoring, configured per rule from the rule text only (baselines.json). A tool rejects an item iff its output on the change carries a diagnostic its output on the base does not. OR-of-tools on an item rejects iff any tool mapped to its rule rejects; a tool not mapped to the rule is silent (ACCEPT). Mapping experiments/reviewer-keep/baselines.json (sha256 380ad025c3bc78539016787d1b3c8545589644cfc3a09c4081439ad767859457).

• tsc --noEmit --noUnusedLocals --noUnusedParameters: typescript 5.9.3, @types/node 22.18.0
• eslint .: eslint 9.39.1, typescript-eslint 8.46.4
• prettier --check .: prettier 3.6.2
• the frozen churn heuristic below, over git diff --numstat base change and TASK.md

**Churn heuristic, frozen:** REJECT iff the change touches any file not named in TASK.md, or changes more than 40 lines (added+deleted). Without a TASK.md, only the 40-line rule applies.

References: always-RED (BA 0.5); prevalence-random (expected BA 0.5). Jev: OFF. Decided now, never from data. Jev is a model, not a model-free arm.

The bar

BA difference = BA(reviewer item-majority) − BA(best baseline). The bar is: difference ≥ 0.20 AND the 95% lower bound of a **class-stratified bootstrap** > 0. The bootstrap resamples items within RED and within GREEN, 10,000 resamples, seed 10010.

Matched budget. The primary passes only if ALL three hold: the BA difference >= 0.20, AND the 95% lower bound of the class-stratified bootstrap > 0, AND the reviewer's item-level false-reject rate on GREEN items <= 15% (the same budget the best baseline is selected under). A reviewer above the budget fails the primary whatever its BA.

• The frozen scorer takes the schedule as a parameter: 3 runs per item (pre-registered) or 2 (the $60 fallback). With 2 runs a 1-1 split is no majority and is scored as the wrong answer; any other schedule is refused.
• mulberry32(10010) drives the draws (RED items first, then GREEN, in item order); the 95% interval is the percentile interval of the 10,000 sorted differences d: lower d[250], upper d[9749].
• The best of the frozen rule-filtered off-the-shelf tools is selected once, on all items, by the definition below; it is not re-selected per resample.
• The reviewer's item call is the verdict a strict majority of its 3 scheduled runs gave; a run that abstained, failed or was DIFF-BLIND is no decision, and an item without a majority verdict is scored as the wrong answer for its label (EXP 006's mapping).
• Reported beside it, never judged: the paired TPR and TNR differences, each with an exact McNemar test (two-sided binomial on the discordant pairs).

The best baseline, one definition for the hardness gate and the primary:

**Best baseline, one definition everywhere** (gate AND primary): among {each tool, OR-of-tools}, the one with the highest BA subject to FR ≤ 15% on GREEN items.
Ties → the lower FR, then the fixed tool order [tsc, eslint, prettier, churn, OR].
If none reaches FR ≤ 15%, the hardness gate passes (published explicitly) and the primary compares against always-RED / prevalence-random, reported as such.

The scorer is scripts/reviewer-keep-stats.mjs (sha256 779c20afbc3fe22f4a89c2d717f84f332b9a8e2332d7dc51d68f19b47e247944), frozen with this record.

Secondary, gates and controls

• run-level: Run-level miss rate <= 15% and false-reject rate <= 15%, judged on the run-level point estimate (a run without a decision is wrong either way).
• item-level: The item-level majority, published with a Wilson 95% interval and the three-state label: passes / fails / passes, not established at this N.
• self-agreement: Self-agreement >= 0.90: the share of items whose 3 runs all gave the same decided verdict (EXP 006 level).
• mapped-unmapped: The BA difference between the reviewer and the selected comparator on the items of the 11 rules mapped to at least one tool, and on the items of the 7 rules mapped to none (subgroupDifferences in the scorer). Published, never judged.
• Hardness gate: Before counted runs, the best baseline's balanced accuracy at a false-reject rate <= 15% must be <= 0.75. Otherwise the corpus is 'decidable model-free', and that is published as a finding: these rules were decidable after all. If no baseline reaches FR <= 15%, the gate passes, and that is published explicitly.
• Manipulation: Diff-seen in >= 90% of runs (EXP 006's classifier), or the run is uninformative.
• Controls: The reviewer must call both controls of a batch correctly in >= 90% of batches (all 3), or the run is uninformative.

Floors and kills

fewer than 10 eligible rules after the blind filter → "too few buildable rules";
after label-disagreement exclusions, fewer than 30 RED or fewer than 30 GREEN items → "unlabelable at this N";
if the human spot-check disagrees with the agreed label on more than 10% of the spot-checked items → the agreement gate fails.

• Agreement bar missed after one fix-forward round: the corpus is unlabelable, published as such.
• Any floor above reached: published under its own name.
• Hardness gate failed: published as "decidable model-free"; no counted run.
• Pre-counted ceiling tripped: stop before the counted runs, disclose, ask the founder.

Before the first paid call

• Before the FIRST paid call of EXP 011 (item authoring, blind labelling, practice or the isolation matrix), a dated amendment must record ALL of the following; until it exists, no paid call is made:
• The four baselines are installed at exactly the pinned versions (typescript 5.9.3, @types/node 22.18.0, eslint 9.39.1, typescript-eslint 8.46.4, prettier 3.6.2), with a committed lockfile for the fixture's tool install.
• Each pinned version is verified to exist in the public registry, by its resolved integrity in that lockfile; a version that does not resolve is replaced by the amendment, before any call, with the replacement disclosed.
• The base fixture is clean under all four: tsc --noEmit --noUnusedLocals --noUnusedParameters, eslint . and prettier --check . report zero diagnostics on it, and the churn heuristic has nothing to judge on the base; the outputs are committed with the amendment.
• The founder-signed cap and the pinned author and labeller model ids are recorded by dated amendment, as stated under spend and labels.

Spend and the clock

Cap: $80 proposed — founder sign-off pending; no call before a dated amendment records the signed cap. Pre-counted ceiling $30. $0: this pre-registration made no model or API call.

• 234 counted runs at $0.17–$0.18 each.
• A spend-limit circuit breaker as in EXP 007: every paid call appends one ledger line; a call is refused when the recorded total plus a reserve would pass the cap, or a pre-counted call when it would pass the ceiling; a call the client refuses at its spend limit stops the run.
• If the founder holds the cap at $60: 2 runs per item (about $27-30 counted), self-agreement measured on 2 runs. The dated amendment that records it changes only the schedule (runsPerItem 2, which the frozen scorer already takes) and the pinned scorer sha256, before any call.
• Not before the merge time of this pre-registration's odin-rnd PR.

**Pre-counted ceiling trip:** if pre-counted spend reaches $30 before the counted runs, STOP. Do not start the counted runs; disclose; ask the founder.

Decisions the design left open

• Items: 18 rules x (2 RED + 2 GREEN) = 72 items, not the design's pre-filter estimate of about 80; counted runs = 3 x (72 items + 6 controls) = 234.
• Control batches: a batch is 6 consecutive eligible rules (24 items), so 3 batches and 6 control items; the 90% control bar therefore means all 3 batches.
• Item call: the verdict of a strict majority of the 3 scheduled runs; abstained, failed and DIFF-BLIND runs are no decision, and an item without a majority verdict is scored as the wrong answer for its label (EXP 006's mapping).
• Bootstrap mechanics: mulberry32 (copied from EXP 007), RED draws then GREEN in item order, percentile interval d[250] and d[9749]; the comparator is selected once and not re-selected per resample; with no eligible baseline the comparator is the constant BA 0.5.
• McNemar: exact two-sided, p = min(1, 2 x P(X <= min(b, c))), X ~ Binomial(b + c, 1/2); TPR pairs on RED items, TNR pairs on GREEN items.
• A tool rejects an item iff its output on the change carries a diagnostic (file, code, message; line-insensitive) absent on the base; a tool not mapped to the rule is silent (ACCEPT); OR-of-tools is over the mapped tools only.
• Tool versions: typescript 5.9.3 (+ @types/node 22.18.0), eslint 9.39.1 with typescript-eslint 8.46.4, prettier 3.6.2, chosen without a registry lookup.
• RULE.md holds the rule text and its context line verbatim (no plugin, no rule id); RULE.md and any TASK.md are committed in the base. The fixture ships TASK.template.md, never a live TASK.md.
• TASK.md's named files for the churn heuristic are the paths listed under its '## Files' heading; lines changed come from git diff --numstat.
• The fixture uses a const-object status set, not a TypeScript enum, so its tests run under node type stripping (erasableSyntaxOnly).
• The EXP 006 reviewer files reused by reference are pinned by sha256 at odin-rnd main 5b5c681 (run_reviewer6, stream6, fence6, scrub6, spend6, guard6, diff-seen, fingerprints, amendment-01).
• not-rules.json is vendored beside the decisions, and the validator recomputes the 50 not-class rules from EXP 007's census records.
• Bars are compared with a 1e-12 floating-point slack, because every rate is a ratio of integer counts.
• The note, its data copy, its home row (directly above the EXP 009 row) and its sitemap entry are committed under site/; the build re-checks them.
• Refute r1: the matched false-reject budget binds the reviewer too: the primary also requires the reviewer's item-level FR on GREEN items <= 15%.
• Refute r1: tool install, version existence (by lockfile integrity) and a clean base under all four tools are preconditions of the first paid call, not of scoring.
• Refute r1: the scorer takes runsPerItem (3 or 2); the comparator is named the frozen rule-filtered off-the-shelf tools; a mapped/unmapped subgroup secondary is reported; the human spot-check is frozen at 8 RED + 8 GREEN by seed 10011.

What this does not establish

• One reviewer, one fixture language (TypeScript), a small synthetic fixture: no transfer claim to large codebases.
• not-class membership comes from EXP 007's translator and adjudicator (a Claude family) and inherits their published calibration.
• One rule shown per item: real gates show many rules at once. The one-rule prior applies to the baselines too.
• The baseline tool mapping per rule is a judgement, frozen before items exist and published; 7 of the 18 rules map to no listed tool.
• Plugin skew: abide 12/18, hunch 5/18, jev-axi 1/18; reported per plugin, never reweighted.
• Rust-only rules are excluded to keep one fixture language.
• The prompt is new and NOT comparable to EXP 005/006; their figures are context, not a baseline.
• The fixture was written by a Claude agent; the reviewer is a Claude model. The items and both labels come from non-Claude models.
• Tool versions were pinned without a registry lookup, and base cleanliness is a precondition the installing amendment must prove.
• Item counts assume no exclusions: with 36 RED and 36 GREEN, at most 6 items per class can be excluded for label disagreement before the 30-per-class floor.
• The 7 rules mapped to no tool are, for the comparator, always ACCEPT: on their 14 RED items a perfect reviewer gains 14/36 of TPR over it, about 0.19 of BA advantage on their own (7/36 = 0.194), nearly the whole 0.20 bar. The mapped/unmapped subgroup differences are therefore published beside the primary.
• The no-baseline branch (no candidate within FR <= 15%) is unreachable under this mapping and the 30-per-class floor: prettier is mapped to 2 rules, so it can reject at most 4 GREEN items, an FR of at most 4/30 = 13.3%. The branch stays in the frozen scorer, unused.

Publication: every outcome published.

Provenance

This page is rendered from preregistration.json, the committed record at experiments/reviewer-keep/preregistration.json, whose sha256 is 60632a62f23d3d91118f48abf74c6c31061d0f5c2ea57ec7a5a5b0500aacd67f. The site publishes that file byte for byte, and a repository test re-renders this page from it and fails if they differ. A validator rebuilds every file fact in the record from the files on disk; it pins 10 files by sha256.

No item exists and no paid call has been made. The results, when they exist, will be judged against this record as published, and reported as they come out, whichever way they go.