CI Witness

The target is fixed.
The correction still fails.

An agent can make the failing assertion pass while breaking something else—or by removing the check. CI Witness deliberately tries both and requires the acceptance check to reject them.

Inspect the recorded evidence Explore a scoped pilot

A locale fix has more than one obligation.

Under Turkish locale, the fixture lowercases TITLE to tıtle, with a dotless ı. The contract expects title. A correction must also preserve the boundary and regression checks.

Authored Java fixture · cases from the recorded experiment below
Candidate	What actually happens	Recorded evidence
Controlled baseline	`TITLE → tıtle` The explicit Turkish locale reproduces the target failure.	Inspect this case
Correction	`Target + boundary + regressions` All four required checks execute. The declared boundary preserves straße.	Inspect this case
Seeded bad correction	`Target passes · regression fails` The original symptom is gone, but the arithmetic regression rejects this candidate.	Inspect this case

A fix must retain the evidence that could reject it.

The runner requires the exact target assertion, the case-specific boundary and both regression assertions. Suppressed checks cannot qualify. Unknown exceptions stay unresolved. Timezone, ordering and configuration cases apply the same discipline under explicit conditions.

These examples test the acceptance mechanism. They do not measure an agent’s performance on unseen work.

Inspect the recording.

Recorded fixture run; the execution environment is identified below. These Java examples, candidate tests and corrections were authored for this demonstration. Selecting a case reads saved evidence; it does not execute code or an agent.

Recorded fixture run · GitHub Actions · 2026-10-11T05:47:49.747Z

Run 29bb03f4-8bc4-4d79-9e8e-fc92a604d1f2 · base source revision 6a375f0b5f0c0c26c9563613e5aa0cbb808c94b9

Uncommitted working tree; base revision does not identify these edits. Input hashes identify executed sources.

GitHub Actions recording · attempt 1

Four deterministic authored timezone, locale, order and configuration cases; explicit fixed order, reproduced target assertions and hand-authored corrections with one case-specific boundary and two regression assertions per case.

Runtime and experiment boundaries

{
  "node": "v22.23.3",
  "java": "openjdk version \"21.0.12.1\" 2026-08-18 LTS\nOpenJDK Runtime Environment Temurin-21.0.12.1+1 (build 21.0.12.1+1-LTS)\nOpenJDK 64-Bit Server VM Temurin-21.0.12.1+1 (build 21.0.12.1+1-LTS, mixed mode, sharing)",
  "javac": "javac 21.0.12.1"
}

No statistical flaky-rate estimate or general diagnosis claim.
No AI calls, automatic repair, customer inputs or measured ROI.
Unknown failures remain inconclusive, including the expected unknown negative control.
Trusted bundled code only; temporary directories are not a security sandbox.
Hashes establish input consistency, not malicious-tamper authenticity.

Raw execution record · Source hashes and recording manifest

controlled-corrections: 4 / 4 cases
Declared cases with all four exact assertions passing after the authored correction.
discrimination: 32 / 32 observations
Expected outcomes matched across 28 executions and four boundary observations, including rejection and unresolved unknown controls; not a behavioral pass rate.
unresolved: 4 / 28 executions
Unresolved executions, including four deliberately unknown failures.

Recorded case

timezone

rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-unrelated-failure

Actual

rejected-unrelated-failure

Actual command transcript

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-7: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness unrelated timezone
cwd: <workdir>
completed; exit 1; 47 ms
stdout:
ASSERT|unrelated|FAIL|actual=crash|expected=ready
CHECKS|1|FAILURES|1

stderr:

Acceptance concerns the stated evidence check. An expected rejection can demonstrate that the check works; it does not mean the tested behavior passed. Fixture results are not customer savings, held-out performance or a guarantee of broader correctness.

All recorded observations, including negative controls

timezone-baseline · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-4: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness baseline timezone
cwd: <workdir>
completed; exit 1; 112 ms
stdout:
ASSERT|target.timezone|FAIL|actual=2026-01-02|expected=2026-01-01
ASSERT|boundary.timezone|PASS|actual=2024-02-29|expected=2024-02-29
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

timezone-reproducer · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-5: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness reproducer timezone
cwd: <workdir>
completed; exit 1; 72 ms
stdout:
ASSERT|target.timezone|FAIL|actual=2026-01-02|expected=2026-01-01
ASSERT|boundary.timezone|PASS|actual=2024-02-29|expected=2024-02-29
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

timezone-corrected · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

verified-correction

Actual

verified-correction

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-6: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness corrected timezone
cwd: <workdir>
completed; exit 0; 52 ms
stdout:
ASSERT|target.timezone|PASS|actual=2026-01-01|expected=2026-01-01
ASSERT|boundary.timezone|PASS|actual=2024-02-29|expected=2024-02-29
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

timezone-boundary · accepted

Independent case-specific boundary assertion must execute alongside the target and regression checks.

Expected

2024-02-29

Actual

2024-02-29

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-6: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness corrected timezone
cwd: <workdir>
completed; exit 0; 52 ms
stdout:
ASSERT|target.timezone|PASS|actual=2026-01-01|expected=2026-01-01
ASSERT|boundary.timezone|PASS|actual=2024-02-29|expected=2024-02-29
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

timezone-unrelated · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-unrelated-failure

Actual

rejected-unrelated-failure

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-7: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness unrelated timezone
cwd: <workdir>
completed; exit 1; 47 ms
stdout:
ASSERT|unrelated|FAIL|actual=crash|expected=ready
CHECKS|1|FAILURES|1

stderr:

timezone-suppressed · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-missing-checks

Actual

rejected-missing-checks

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-8: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness suppressed timezone
cwd: <workdir>
completed; exit 0; 38 ms
stdout:
CHECKS|0|FAILURES|0

stderr:

timezone-regression · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-surviving-regression

Actual

rejected-surviving-regression

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-9: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness regression timezone
cwd: <workdir>
completed; exit 1; 64 ms
stdout:
ASSERT|target.timezone|PASS|actual=2026-01-01|expected=2026-01-01
ASSERT|boundary.timezone|PASS|actual=2024-02-29|expected=2024-02-29
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|FAIL|actual=5|expected=4
CHECKS|4|FAILURES|1

stderr:

timezone-unknown · inconclusive

Unknown failure remains unresolved; no diagnosis inferred from exception text.

Expected

inconclusive

Actual

inconclusive

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-10: java -Dfile.encoding=UTF-8 -Duser.timezone=Europe/Amsterdam -Duser.language=en -Duser.country=US -cp <workdir> Witness unknown timezone
cwd: <workdir>
completed; exit 1; 27 ms
stdout:

stderr:
Exception in thread "main" java.lang.IllegalStateException: UNCLASSIFIED_CONTROL
	at Witness.main(Witness.java:31)

locale-baseline · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-11: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness baseline locale
cwd: <workdir>
completed; exit 1; 87 ms
stdout:
ASSERT|target.locale|FAIL|actual=tıtle|expected=title
ASSERT|boundary.locale|PASS|actual=straße|expected=straße
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

locale-reproducer · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-12: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness reproducer locale
cwd: <workdir>
completed; exit 1; 55 ms
stdout:
ASSERT|target.locale|FAIL|actual=tıtle|expected=title
ASSERT|boundary.locale|PASS|actual=straße|expected=straße
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

locale-corrected · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

verified-correction

Actual

verified-correction

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-13: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness corrected locale
cwd: <workdir>
completed; exit 0; 47 ms
stdout:
ASSERT|target.locale|PASS|actual=title|expected=title
ASSERT|boundary.locale|PASS|actual=straße|expected=straße
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

locale-boundary · accepted

Independent case-specific boundary assertion must execute alongside the target and regression checks.

Expected

straße

Actual

straße

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-13: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness corrected locale
cwd: <workdir>
completed; exit 0; 47 ms
stdout:
ASSERT|target.locale|PASS|actual=title|expected=title
ASSERT|boundary.locale|PASS|actual=straße|expected=straße
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

locale-unrelated · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-unrelated-failure

Actual

rejected-unrelated-failure

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-14: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness unrelated locale
cwd: <workdir>
completed; exit 1; 62 ms
stdout:
ASSERT|unrelated|FAIL|actual=crash|expected=ready
CHECKS|1|FAILURES|1

stderr:

locale-suppressed · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-missing-checks

Actual

rejected-missing-checks

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-15: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness suppressed locale
cwd: <workdir>
completed; exit 0; 36 ms
stdout:
CHECKS|0|FAILURES|0

stderr:

locale-regression · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-surviving-regression

Actual

rejected-surviving-regression

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-16: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness regression locale
cwd: <workdir>
completed; exit 1; 104 ms
stdout:
ASSERT|target.locale|PASS|actual=title|expected=title
ASSERT|boundary.locale|PASS|actual=straße|expected=straße
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|FAIL|actual=5|expected=4
CHECKS|4|FAILURES|1

stderr:

locale-unknown · inconclusive

Unknown failure remains unresolved; no diagnosis inferred from exception text.

Expected

inconclusive

Actual

inconclusive

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-17: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=tr -Duser.country=TR -cp <workdir> Witness unknown locale
cwd: <workdir>
completed; exit 1; 27 ms
stdout:

stderr:
Exception in thread "main" java.lang.IllegalStateException: UNCLASSIFIED_CONTROL
	at Witness.main(Witness.java:31)

order-baseline · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-18: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness baseline order
cwd: <workdir>
completed; exit 1; 54 ms
stdout:
ASSERT|target.order|FAIL|actual=beta,alpha|expected=alpha,beta
ASSERT|boundary.order|PASS|actual=alpha,beta,gamma|expected=alpha,beta,gamma
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

order-reproducer · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-19: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness reproducer order
cwd: <workdir>
completed; exit 1; 59 ms
stdout:
ASSERT|target.order|FAIL|actual=beta,alpha|expected=alpha,beta
ASSERT|boundary.order|PASS|actual=alpha,beta,gamma|expected=alpha,beta,gamma
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

order-corrected · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

verified-correction

Actual

verified-correction

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-20: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness corrected order
cwd: <workdir>
completed; exit 0; 46 ms
stdout:
ASSERT|target.order|PASS|actual=alpha,beta|expected=alpha,beta
ASSERT|boundary.order|PASS|actual=alpha,beta,gamma|expected=alpha,beta,gamma
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

order-boundary · accepted

Independent case-specific boundary assertion must execute alongside the target and regression checks.

Expected

alpha,beta,gamma

Actual

alpha,beta,gamma

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-20: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness corrected order
cwd: <workdir>
completed; exit 0; 46 ms
stdout:
ASSERT|target.order|PASS|actual=alpha,beta|expected=alpha,beta
ASSERT|boundary.order|PASS|actual=alpha,beta,gamma|expected=alpha,beta,gamma
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

order-unrelated · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-unrelated-failure

Actual

rejected-unrelated-failure

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-21: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness unrelated order
cwd: <workdir>
completed; exit 1; 52 ms
stdout:
ASSERT|unrelated|FAIL|actual=crash|expected=ready
CHECKS|1|FAILURES|1

stderr:

order-suppressed · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-missing-checks

Actual

rejected-missing-checks

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-22: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness suppressed order
cwd: <workdir>
completed; exit 0; 39 ms
stdout:
CHECKS|0|FAILURES|0

stderr:

order-regression · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-surviving-regression

Actual

rejected-surviving-regression

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-23: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness regression order
cwd: <workdir>
completed; exit 1; 52 ms
stdout:
ASSERT|target.order|PASS|actual=alpha,beta|expected=alpha,beta
ASSERT|boundary.order|PASS|actual=alpha,beta,gamma|expected=alpha,beta,gamma
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|FAIL|actual=5|expected=4
CHECKS|4|FAILURES|1

stderr:

order-unknown · inconclusive

Unknown failure remains unresolved; no diagnosis inferred from exception text.

Expected

inconclusive

Actual

inconclusive

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-24: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.order=beta,alpha -cp <workdir> Witness unknown order
cwd: <workdir>
completed; exit 1; 29 ms
stdout:

stderr:
Exception in thread "main" java.lang.IllegalStateException: UNCLASSIFIED_CONTROL
	at Witness.main(Witness.java:31)

config-baseline · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-25: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness baseline config
cwd: <workdir>
completed; exit 1; 53 ms
stdout:
ASSERT|target.config|FAIL|actual=legacy|expected=strict
ASSERT|boundary.config|PASS|actual=relaxed|expected=relaxed
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

config-reproducer · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

target-failure-reproduced

Actual

target-failure-reproduced

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-26: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness reproducer config
cwd: <workdir>
completed; exit 1; 55 ms
stdout:
ASSERT|target.config|FAIL|actual=legacy|expected=strict
ASSERT|boundary.config|PASS|actual=relaxed|expected=relaxed
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|1

stderr:

config-corrected · accepted

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

verified-correction

Actual

verified-correction

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-27: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness corrected config
cwd: <workdir>
completed; exit 0; 48 ms
stdout:
ASSERT|target.config|PASS|actual=strict|expected=strict
ASSERT|boundary.config|PASS|actual=relaxed|expected=relaxed
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

config-boundary · accepted

Independent case-specific boundary assertion must execute alongside the target and regression checks.

Expected

relaxed

Actual

relaxed

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-27: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness corrected config
cwd: <workdir>
completed; exit 0; 48 ms
stdout:
ASSERT|target.config|PASS|actual=strict|expected=strict
ASSERT|boundary.config|PASS|actual=relaxed|expected=relaxed
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|PASS|actual=4|expected=4
CHECKS|4|FAILURES|0

stderr:

config-unrelated · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-unrelated-failure

Actual

rejected-unrelated-failure

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-28: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness unrelated config
cwd: <workdir>
completed; exit 1; 56 ms
stdout:
ASSERT|unrelated|FAIL|actual=crash|expected=ready
CHECKS|1|FAILURES|1

stderr:

config-suppressed · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-missing-checks

Actual

rejected-missing-checks

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-29: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness suppressed config
cwd: <workdir>
completed; exit 0; 39 ms
stdout:
CHECKS|0|FAILURES|0

stderr:

config-regression · rejected

Exact assertion identities, values, execution count and exit agree with the declared oracle.

Expected

rejected-surviving-regression

Actual

rejected-surviving-regression

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-30: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness regression config
cwd: <workdir>
completed; exit 1; 54 ms
stdout:
ASSERT|target.config|PASS|actual=strict|expected=strict
ASSERT|boundary.config|PASS|actual=relaxed|expected=relaxed
ASSERT|regression.empty|PASS|actual=|expected=
ASSERT|regression.total|FAIL|actual=5|expected=4
CHECKS|4|FAILURES|1

stderr:

config-unknown · inconclusive

Unknown failure remains unresolved; no diagnosis inferred from exception text.

Expected

inconclusive

Actual

inconclusive

Execution receipt

command-3: javac -encoding UTF-8 -d <workdir> <repo>/demos/ci-witness/fixtures/Witness.java
cwd: <workdir>
completed; exit 0; 572 ms
stdout:

stderr:


command-31: java -Dfile.encoding=UTF-8 -Duser.timezone=UTC -Duser.language=en -Duser.country=US -Dfixture.mode=legacy -cp <workdir> Witness unknown config
cwd: <workdir>
completed; exit 1; 28 ms
stdout:

stderr:
Exception in thread "main" java.lang.IllegalStateException: UNCLASSIFIED_CONTROL
	at Witness.main(Witness.java:31)

Reproduce on your machine.

From the repository root, with Node 22 and JDK 21 installed. Runs trusted bundled fixtures locally.

node demos/ci-witness/run.mjs --out /tmp/ci-witness.json

Read the demo source and limitations

Bring one decision to the bench.

A proposed pilot for a team deciding whether this evidence would improve its engineering review.

Proposed delivery cap: 12 engineering hours for this bounded scope, subject to input review and agreement. A planning hypothesis, not a service promise or measured saving.

Prepare a pilot brief Download CI Witness proposal

Good fit: One known environment-sensitive Java failure in a trusted nonproduction test suite.
You bring: A runnable fixture; approved environment configurations; target assertion and expected boundary and regression outcomes.
You take away: A controlled matrix, baseline/reproducer/correction transcripts and an acceptance receipt with unresolved cases disclosed.
Acceptance to agree: The same target failure reproduces; the correction passes its target, boundary and regression checks without suppressing them.
Outside this pilot: General autonomous diagnosis or repair, production changes, flaky-rate estimates and arbitrary untrusted code.

Measure whether it earns its place.

Agree a comparable baseline, acceptance criteria and a stop condition before work starts. Record setup, execution, human review and rework time, plus tool costs and the full cost of labor. Compare total effort and accepted outcomes; disclose unresolved work.

The demo establishes a mechanism to inspect. Whether it reduces your team's cost or risk is a pilot question. Agree scope, fees, access, data handling and delivery terms before sharing code or data. The brief is a manual preparation step; nothing is sent automatically.

What would convince your team that an agent’s fix is safe to accept?

Bring one reproducible failure and the checks your team cannot afford to lose. We can define the reproduction and acceptance conditions, compare candidate corrections, and record investigation and review effort alongside failures.

The proposed work is the acceptance system around your workflow: an explicit contract, checks that can reject a plausible but wrong result, and a record another engineer can reproduce. The small Java example above makes one part of that approach inspectable.

Discuss your acceptance problem

Part of a larger engineering problem.

When agents produce more changes, someone still has to decide which changes deserve acceptance. Odin explores that decision from the rule to the observed outcome.

Blueprint Conformance Engine: Published architecture constraints checked against code. Public, reproducible examples.
A passing pipeline was not yet a verified release: Our internal release account follows coverage artifacts, a qualified image and a user journey. Author-reported; its private receipts are not independently inspectable here.
These Witness experiments: Public adversarial examples: tests that miss defects, matching implementations that share a bug, and corrections that lose required checks.

CI Witness

The target is fixed.The correction still fails.

A locale fix has more than one obligation.

A fix must retain the evidence that could reject it.

Inspect the recording.

timezone

Expected

Actual

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

Expected

Actual

Execution receipt

The target is fixed.
The correction still fails.