Skip to content

Errata meta-verification

Independent re-check of every errata claim that asserts an error, run outside the session that recorded the claim. The goal is to catch wrong error findings before they harden into the appendix, and to confirm real ones with a second system.

Run a meta-verification pass after a large batch of error rows lands in errata.csv, or when preparing to treat the errata list as settled for a release. It is not a substitute for the five-step check on each row; it audits that check.

  1. Build claim batches from every original-error, edition-error, open and rejected row. Each card carries id, claim_as_printed (the was text), appears_in, section and original_source only — never the recorded verdict.
  2. Split the cards into six batches. Run three independent checkers per batch (best of three). Each run sees one batch file and none of the others.
  3. Checkers may open only primary evidence: ncurses/, form/, menu/, panel/, progs/, include/, Ada95/, c++/, misc/, man/ (*.3x, *.1m, *.5), doc/html/, and top-level INSTALL, NEWS, README, ANNOUNCE. They must not open docs/appendix/ or site/.
  4. Use the Merlon Docs evidence check (install with npx skills add Merlon-Docs/merlon --skill merlon-docs). Each card records claim, opened, quote, verdict, confidence and earlier.
  5. Tally best-of-three majorities, list splits, and compare against errata.csv with the framing below.
  6. File the report and run outputs under meta-verification/, update Lessons learned, and open a pull request. Hand the report to a separate agent for follow-up after that pull request merges.
System What matches means
Meta-verification run The printed text is correct given primary evidence (the original was right).
An errata error row The error claim is confirmed (the original was wrong).

So an original-error row whose errata verdict is matches expects a meta majority of disagrees. Keep that map explicit in every report.

Commit 568319e5809ad365ba195ebbd100946c53b42cb4, Cursor Cloud Agent, six batches × three runs, Merlon Docs skill loaded.

Artifact Path
Cover note for the acting agent cover-note.md
Full evidence report handoff-report.md
Bo3 rollups findings/
Per-run cards and usage runs/
Batch inputs inputs/
Adjudication of the misaligned rows and splits adjudication.md

Bo3 on 178 claims: 78 disagrees, 70 matches, 16 can't tell, 8 superseded, 1 missing, 5 three-way splits (L56, L57, L203, L215, L287). Against errata.csv under the map above: 76 aligned, 97 misaligned. The acting agent starts from the cover note and the handoff report’s checklist. The adjudication rechecked every misaligned row and split from primary evidence: of 112 errata rows checked, 91 are true positives and 3 true negatives, 10 are type I errors (the original was right), none is a type II error, and 8 can’t be settled from the repository. The checkers’ Bo3 majority made no type I errors on those rows, but it made 49 type II errors against 30 true positives.

A prior attempt inside the Claude docs session failed (safety filter / interrupted runs) and must not be treated as a completed pass; it is recorded in Lessons learned.