Commit graph

2 commits

Author SHA1 Message Date
f1c2c9eb4a fix(review): write the gold-scorer key separator as an escape, not a raw NUL byte
lib/review/gold-scorer.mjs held its (file, rule_key) separator as a literal NUL byte
inside a string literal. Git's binary heuristic therefore classed the whole file as
binary. Every diff of the scorer read "Binary files differ", `--numstat` printed "-  -",
and plain grep matched nothing in it. The separator is now written as the escape
sequence '\x00', which is the same one-character string at runtime.

Proof that the change is behaviour-neutral:
- A snapshot script recorded every number the scorer produces, before and after:
  - the committed run-perfect fixture through the coordinator contract;
  - empty, half, spurious (keys containing spaces and colons), duplicated and empty-gold
    runs;
  - all 9 verdict pairs.
  `diff` of the two outputs is empty (141 lines each).
- tests/lib/gold-scorer.test.mjs + tests/lib/gold-eval.test.mjs: 13/13 before and after.
- `git diff --no-index --numstat /dev/null <file>`: the new file reads "89 0" (text); the
  old blob, as a control, reads "- -" (binary).

End-state gate D-06: open -> closed (defects 4 -> 3 of 7). Suite 1059 (1057/0/2), run
on a clean export of the index.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 15:36:18 +02:00
440594f1b2 feat(eval): SKAL-1·4b offline gold-scored output eval
Scores committed agent-run fixtures against the golden corpus at
(file, rule_key) granularity, building on the deterministic coordinator
contract (4a). Offline: committed reviewer payloads, no live agent spawn,
no LLM, no network (the LLM-in-the-loop grading is the separate 4c tier).

- lib/review/gold-scorer.mjs: scoreFindings (precision/recall/f1 at
  (file,rule_key) granularity, line+severity ignored) + scoreVerdict; pure,
  with documented vacuous-set conventions.
- tests/fixtures/bakeoff-rich/runs/run-perfect.json: committed run that
  reproduces all 5 seeded gold findings through runContract.
- tests/lib/gold-eval.test.mjs: the scoring RUN (precision/recall/f1 = 1.0,
  verdict == expected_verdict BLOCK, nothing suppressed/skipped).
- lib/util/test-census.mjs: third census category (goldEval) — a scoring run
  is neither behavior coverage nor a doc-pin; honest-count invariant now 3-way.
- docs/eval-corpus/README.md: 4b moved from Future hardening to implemented.

Suite 809 -> 822 (820/0/2). gold-scorer covers TP+FP+FN+degenerate paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJQYC5vpkJWxndS55vQQZ6
2026-06-30 09:00:33 +02:00