chore(voyage): release v5.8.0 - SKAL-1·4b offline gold-scored output eval
Version sync 5.7.1 -> 5.8.0 across plugin.json, package.json, package-lock.json, README badge, and CHANGELOG top entry (guarded by doc-consistency.test.mjs). Suite 822 (820/0/2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJQYC5vpkJWxndS55vQQZ6
This commit is contained in:
parent
cb5dba9542
commit
0c634b6636
5 changed files with 21 additions and 6 deletions
15
CHANGELOG.md
15
CHANGELOG.md
|
|
@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.
|
|||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## v5.8.0 — 2026-06-30 — offline gold-scored output eval (SKAL-1·4b)
|
||||
|
||||
Additive — no behavior change, no breaking change. Internal eval infrastructure only (`lib/` + `tests/` + docs); no command, agent, profile, or Handover contract touched.
|
||||
|
||||
### Gold-scored output eval (SKAL-1·4b)
|
||||
|
||||
- The review-coordinator self-eval gains its **scoring run**, building on the deterministic coordinator contract + golden corpus shipped as the 4a foundation in v5.7.0.
|
||||
- `lib/review/gold-scorer.mjs`: `scoreFindings(runFindings, goldFindings)` matches at **`(file, rule_key)` granularity** (line + severity deliberately ignored) → `{ tp, fp, fn, precision, recall, f1, matched, missed, spurious }`; `scoreVerdict` checks exact verdict match. Pure — no I/O, no LLM, no network. Vacuous-set conventions (empty run → recall 0; empty gold → precision 0; f1 collapses to 0) are documented in the module header.
|
||||
- `tests/fixtures/bakeoff-rich/runs/run-perfect.json`: the first **committed run** — reviewer payloads that, fed through `runContract`, reproduce all 5 seeded gold findings. The regression guard: any future contract change that silently suppresses or skips a seeded finding breaks the eval.
|
||||
- `tests/lib/gold-eval.test.mjs`: the **scoring run**, wired into `node --test`. Asserts `run-perfect` scores precision/recall/f1 = 1.0 and `verdict == expected_verdict` (BLOCK). Offline: committed payloads, no live agent spawn (the LLM-in-the-loop grading is the separate 4c tier).
|
||||
- **Third test-census category.** `lib/util/test-census.mjs` now reports a `goldEval` bucket (matched by `GOLD_EVAL_FILE_RE`) separately from `behavior` and `docPins` — a scoring run is neither behavior coverage nor a prose pin, so the honest-count invariant is a 3-way sum.
|
||||
- `docs/eval-corpus/README.md`: SKAL-1·4b moved from "Future hardening" to an implemented section documenting the scorer, run format, and census category.
|
||||
- Suite: **822 (820 pass / 0 fail / 2 skip)**, up from 809 (+10 scorer behavior tests, +3 eval scoring-run tests). The scorer test covers the discriminating paths (false negatives + false positives) and the degenerate empty-run / empty-gold cases, not merely the all-match case.
|
||||
- Version sync: `plugin.json`, `package.json`, `package-lock.json`, README badge, CHANGELOG top entry all at `5.8.0`, guarded by `doc-consistency.test.mjs`.
|
||||
|
||||
## v5.7.1 — 2026-06-29 — relocate agent `<example>` blocks to body (always-loaded token trim)
|
||||
|
||||
Performance/packaging change — no behavior change, no breaking change. (M4)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue