docs(config-audit): v5 implementation log — Session 5 release result

v5.0.0 SHIPPED 2026-05-01. Tag config-audit/v5.0.0 pushed to Forgejo.
SC-6b release-gate PASS at -0.85% delta (CLAUDE.md actual 589 vs
estimated 594, well within ±5% gate).

Per-step:
- Step 28: README/CLAUDE.md straggler-sweep + self-audit counter alignment
- Step 29: version bump 4.0.0 → 5.0.0 + consolidated CHANGELOG
- Step 30: full audit + live SC-6b gate + tag (incl. one in-step bug fix
  for hotspot.path exposure, required to make calibration measurable)

635 tests still green throughout. No blockers carried forward.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-05-01 09:48:40 +02:00
commit 395a9bd947

View file

@ -180,6 +180,44 @@ Written at the end of each session. State for the next session lives in
---
## Session 5 — release (TBD)
## Session 5 — release (2026-05-01)
*Steps 28-30, including SC-6b release gate.*
**Outcome:** All 3 steps shipped. v5.0.0 tagged and pushed (`config-audit/v5.0.0` on Forgejo). 635 tests still green. SC-6b release-gate **PASS** at 0.85% delta.
### Per-step result
| # | Step | Result | Commit |
|---|------|--------|--------|
| 28 | README + CLAUDE.md straggler-sweep | ✓ green; `--check-readme` PASSES (counts: scanners 12, commands 18, tests 635, knowledge 8, agents 6, hooks 4); self-audit also updated to (a) exclude `plugin-health-scanner.mjs` from `countScannerShape` so the orchestrated-scanner count matches the README badge taxonomy, and (b) `countTestCases` runs `node --test` to count test cases (635) instead of test files (36) — required for badge accuracy | `5bf500e` `docs(config-audit): straggler sweep for v5.0.0 — sync all badge counts` |
| 29 | Version bump 4.0.0 → 5.0.0 + consolidated CHANGELOG | ✓ `plugin.json` bumped, README version badge bumped, Version History row added, marketplace root README updated (Config-Audit row v4.0.0 → v5.0.0 + counts), `## [5.0.0]` consolidated entry written from alpha.1/alpha.2/beta.1/rc.1 | `dcf8087` `chore(config-audit): bump version to 5.0.0` |
| 30 | Final self-audit + SC-6b gate + tag | ✓ verdict PASS (config A 97/100, plugin A 100/100, readmeCheck PASS); SC-6b gate PASS at 0.85% delta; tag `config-audit/v5.0.0` created and pushed | `6cfca82` `fix(config-audit): expose hotspot.path for --accurate-tokens calibration + SC-6b PASS` (incl. tag) |
### SC-6b release-gate outcome
- **PASS — verified at release time with live `ANTHROPIC_API_KEY`.**
- Fixture: `tests/fixtures/marketplace-large/`. Top-3 hotspots = 1 file-backed (`CLAUDE.md`) + 2 MCP virtuals.
- MCP entries skipped per design (no readable content; their tokens are formula-based at 500 + toolCount × 200, not file content).
- `CLAUDE.md` actual: **589 tokens** (Anthropic `count_tokens`, default `claude-opus-4-7`).
- `CLAUDE.md` estimated: **594 tokens** (4-bytes/token heuristic via `estimateTokens`).
- Delta: **5 tokens / 0.85%** — well within ±5% gate.
- API cost: ≈ 1 call × ~600 tokens = trivial (< $0.01).
- No tuning of `estimateTokens` heuristic required.
### Notable observations / deviations
- **Step 30 surfaced a latent N5 bug.** The rc.1 implementation of `--accurate-tokens` looked up `hotspot.path` but the scanner only emitted `source` — every iteration hit the `if (!hotspot?.path) continue` guard and `actual_tokens` stayed at 0. Detected when running the gate. Minimal fix: file-backed hotspots now expose `path: h.absPath` in the JSON output; MCP-server hotspots intentionally leave `path` unset. Tests updated coverage already in place; no test changes required (the bug was a missing field, not a logic error). After the fix, the calibration produced the expected 589 actual_tokens for CLAUDE.md.
- **Self-audit `--check-readme` now counts test cases by spawning `node --test`.** Slow (~16s on the full plugin) but produces the canonical test count (635) that matches the README badge. `countTestFiles` retained as fallback when the subprocess fails (timeout, parse failure).
- **`plugin-health-scanner.mjs` excluded from `countScannerShape`.** It exports `scan` but is documented under "Standalone Scanner" in README/CLAUDE.md and runs separately from `scan-orchestrator.mjs`. Aligning self-audit's counter with the human/badge taxonomy.
- **API key retrieved from macOS keychain** via `security find-generic-password -a ktg -s anthropic-api-key -w` per global CLAUDE.md convention. Key was masked to `sk-ant-a...` in all error paths (verified: tokenizer-api.mjs maskKey).
- **`sampled_hotspots: 3`** in the calibration JSON is slightly misleading — the slice length is 3 but only 1 had a readable path (other 2 are MCP virtuals). Substantive result is correct: 1 file-backed sample, 0.85% delta. A follow-up could change this to `samples_calibrated: actualCount` for clarity (v5.0.1 candidate).
- **`pre-commit-docs-gate` hook** did not trigger on Session 5 commits — all were `docs:`, `chore:`, or `fix:` types (gate only blocks `feat:`).
- **Marketplace root README updated** in Step 29 (Config-Audit row v4.0.0 → v5.0.0, counts refreshed: 8→12 scanners, 17→18 commands, 543→635 tests, 4→6 patterns, +manifest, +--accurate-tokens, +CPS/DIS/COL).
### Result
- 3 steps + 1 in-step bug fix shipped. Pushed to Forgejo `main` (autorisert).
- Tag: `config-audit/v5.0.0` (pushed; `git ls-remote --tags origin | grep -c "refs/tags/config-audit/v5.0.0$"` → 1).
- Test count: 635 (unchanged — Session 5 was docs/release-sync, not new functionality apart from the path-field bug fix).
- v5.0.0 release run is **complete**.
**No blockers carried forward.** Backlog items deferred to v5.0.1: plugin-vs-built-in collision (research uncertainty), `CA-TOK-*` glob suppression runtime warning, `samples_calibrated` field rename in calibration output, hook-path-bug in legacy `~/.config-audit/`.