diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 095328f..0476d8b 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "voyage", "description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any element, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.", - "version": "5.7.1", + "version": "5.8.0", "author": { "name": "Kjell Tore Guttormsen" }, diff --git a/CHANGELOG.md b/CHANGELOG.md index 6aac024..5827ae6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +## v5.8.0 — 2026-06-30 — offline gold-scored output eval (SKAL-1·4b) + +Additive — no behavior change, no breaking change. Internal eval infrastructure only (`lib/` + `tests/` + docs); no command, agent, profile, or Handover contract touched. + +### Gold-scored output eval (SKAL-1·4b) + +- The review-coordinator self-eval gains its **scoring run**, building on the deterministic coordinator contract + golden corpus shipped as the 4a foundation in v5.7.0. +- `lib/review/gold-scorer.mjs`: `scoreFindings(runFindings, goldFindings)` matches at **`(file, rule_key)` granularity** (line + severity deliberately ignored) → `{ tp, fp, fn, precision, recall, f1, matched, missed, spurious }`; `scoreVerdict` checks exact verdict match. Pure — no I/O, no LLM, no network. Vacuous-set conventions (empty run → recall 0; empty gold → precision 0; f1 collapses to 0) are documented in the module header. +- `tests/fixtures/bakeoff-rich/runs/run-perfect.json`: the first **committed run** — reviewer payloads that, fed through `runContract`, reproduce all 5 seeded gold findings. The regression guard: any future contract change that silently suppresses or skips a seeded finding breaks the eval. +- `tests/lib/gold-eval.test.mjs`: the **scoring run**, wired into `node --test`. Asserts `run-perfect` scores precision/recall/f1 = 1.0 and `verdict == expected_verdict` (BLOCK). Offline: committed payloads, no live agent spawn (the LLM-in-the-loop grading is the separate 4c tier). +- **Third test-census category.** `lib/util/test-census.mjs` now reports a `goldEval` bucket (matched by `GOLD_EVAL_FILE_RE`) separately from `behavior` and `docPins` — a scoring run is neither behavior coverage nor a prose pin, so the honest-count invariant is a 3-way sum. +- `docs/eval-corpus/README.md`: SKAL-1·4b moved from "Future hardening" to an implemented section documenting the scorer, run format, and census category. +- Suite: **822 (820 pass / 0 fail / 2 skip)**, up from 809 (+10 scorer behavior tests, +3 eval scoring-run tests). The scorer test covers the discriminating paths (false negatives + false positives) and the degenerate empty-run / empty-gold cases, not merely the all-match case. +- Version sync: `plugin.json`, `package.json`, `package-lock.json`, README badge, CHANGELOG top entry all at `5.8.0`, guarded by `doc-consistency.test.mjs`. + ## v5.7.1 — 2026-06-29 — relocate agent `` blocks to body (always-loaded token trim) Performance/packaging change — no behavior change, no breaking change. (M4) diff --git a/README.md b/README.md index d920b6d..144bc26 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # trekplan — Brief, Research, Plan, Execute, Review, Continue -![Version](https://img.shields.io/badge/version-5.7.1-blue) +![Version](https://img.shields.io/badge/version-5.8.0-blue) ![License](https://img.shields.io/badge/license-MIT-green) ![Platform](https://img.shields.io/badge/platform-Claude%20Code-purple) @@ -10,7 +10,7 @@ A [Claude Code](https://docs.anthropic.com/en/docs/claude-code) plugin for deep implementation planning, multi-source research, autonomous execution, independent post-hoc review, and zero-friction multi-session resumption. Six commands, one pipeline: -> **What's new — v5.7.1: leaner always-loaded agent listing (round 2).** The 17 example-bearing agents now keep their `` blocks in the body (under `## When to use — examples`) instead of the `description:` frontmatter — voyage launches its agents by name, so the auto-selection examples were cost without function there. This trims ~3,180 always-loaded tokens (frontmatter `description` chars 17,672 → 4,945) from the agent listing injected into every session, with no behavior change. The saving lands after `/plugin marketplace update` + reload. **v5.7.0:** opt-in per-session token/cost metering (SKAL-2) + eval foundation (SKAL-1·4a). **v5.6.1:** one-line `description:` for the four reference/dormant agents (~700 tok). **v5.5.0:** brief **framing** enforcement (`brief_version 2.2`) + a `/trekreview` reviewer-schema contract. Additive — no breaking changes. **Full version history → [CHANGELOG.md](CHANGELOG.md).** +> **What's new — v5.8.0: offline gold-scored output eval (SKAL-1·4b).** The review-coordinator self-eval gains a scoring run: `lib/review/gold-scorer.mjs` grades a committed agent-run fixture against the golden corpus at `(file, rule_key)` granularity (precision/recall/f1 + verdict match), and the suite census gains a third category (`goldEval`) so a scoring run is counted apart from behavior coverage and doc-pins. Offline + deterministic — committed reviewer payloads, no live agent spawn (the LLM-in-the-loop tier is the separate 4c). Internal eval infrastructure; no command/agent/Handover change. **v5.7.1:** leaner always-loaded agent listing — `` blocks relocated to agent bodies (~3,180 tok/turn, no behavior change). **v5.7.0:** opt-in per-session token/cost metering (SKAL-2) + eval foundation (SKAL-1·4a). **v5.6.1:** one-line `description:` for the four reference/dormant agents (~700 tok). **v5.5.0:** brief **framing** enforcement (`brief_version 2.2`) + a `/trekreview` reviewer-schema contract. Additive — no breaking changes. **Full version history → [CHANGELOG.md](CHANGELOG.md).** | Command | What it does | |---------|-------------| diff --git a/package-lock.json b/package-lock.json index 9efc757..4da4c04 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "voyage", - "version": "5.7.1", + "version": "5.8.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "voyage", - "version": "5.7.1", + "version": "5.8.0", "license": "MIT", "engines": { "node": ">=18" diff --git a/package.json b/package.json index 6c4c740..5720b87 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "voyage", - "version": "5.7.1", + "version": "5.8.0", "description": "Voyage — brief, research, plan, execute, review, continue. Contract-driven Claude Code pipeline. /trekbrief, /trekplan, and /trekreview each end by building a self-contained operator-annotation HTML (scripts/annotate.mjs, modelled on claude-code-100x): select text or click any heading/paragraph/list-item, pick intent (Fiks/Endre/Spørsmål), write comment, copy structured prompt, paste back, Claude revises the .md.", "type": "module", "engines": {