Commit graph

2 commits

Author SHA1 Message Date
440594f1b2 feat(eval): SKAL-1·4b offline gold-scored output eval
Scores committed agent-run fixtures against the golden corpus at
(file, rule_key) granularity, building on the deterministic coordinator
contract (4a). Offline: committed reviewer payloads, no live agent spawn,
no LLM, no network (the LLM-in-the-loop grading is the separate 4c tier).

- lib/review/gold-scorer.mjs: scoreFindings (precision/recall/f1 at
  (file,rule_key) granularity, line+severity ignored) + scoreVerdict; pure,
  with documented vacuous-set conventions.
- tests/fixtures/bakeoff-rich/runs/run-perfect.json: committed run that
  reproduces all 5 seeded gold findings through runContract.
- tests/lib/gold-eval.test.mjs: the scoring RUN (precision/recall/f1 = 1.0,
  verdict == expected_verdict BLOCK, nothing suppressed/skipped).
- lib/util/test-census.mjs: third census category (goldEval) — a scoring run
  is neither behavior coverage nor a doc-pin; honest-count invariant now 3-way.
- docs/eval-corpus/README.md: 4b moved from Future hardening to implemented.

Suite 809 -> 822 (820/0/2). gold-scorer covers TP+FP+FN+degenerate paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJQYC5vpkJWxndS55vQQZ6
2026-06-30 09:00:33 +02:00
5144e53129 docs(eval): establish eval-corpus frozen-failure home 2026-06-26 11:57:39 +02:00