config-audit/docs/v6-quality-plan.md
Kjell Tore Guttormsen e60b80978b docs(plan): the gate defect was a tracking defect, not a model defect (v6 quality plan)
The unenforced scope gate in fix-engine was already written down in STATE's
open-items paragraph, formatted identically to "4 inline copies of a target
guard". A stronger model reading that paragraph reaches the same conclusion,
because nothing in the data says one item can let a write reach
~/.claude/CLAUDE.md unapproved and the others cannot. The missing thing is a
severity axis, not reasoning power.

Second occurrence of one class: #63 was a gate not firing because the command
layer was untested; #65 is a gate not firing because the engine never reads
it. Two instances of "only prose stood behind a write gate".

Measured, not asserted: 9 writers in scanners/, 1 imports the gate; 21 command
templates, 17 name a write, 5 invoke write-scope-cli. The 8 ungated writers
are mostly legitimate — the defect is that nothing declares WHICH, so the
question is answered by reading rather than by a guard.

Plan: Q1 gate into code + explicit exemption table (blocks the release), Q2
contract tests built from each template's own text, Q_AUDIT one Fable session
to find the rest of the class, Q3 severity axis in tracking, Q4 release
v6.0.0. Model routing per chunk — not a blanket upgrade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mCkx9wGywqNQzsXkBMzJ1
2026-08-12 20:48:52 +02:00

5.2 KiB

v6 quality plan — from "green suite" to A+

Written 2026-08-12 (session #65), after a gate defect surfaced by accident rather than by process. This plan is about the process that missed it, not about the one defect.

0. Root cause — and why a bigger model is not the fix

The defect (fix-engine.mjs writes files without ever consulting the scope gate) was already written down. It sat in STATE.md's ÅPNE POSTER paragraph, mid-sentence, between "P6/M-BUG-44" and "M-BUG-26":

scope-gaten for de FEM andre armene er fortsatt prosa-kontrakt — fix-engine leser ikke gate

It was read at session start and not acted on. A stronger model reading the same paragraph reaches the same conclusion, because the paragraph gives it no reason to: an unenforced safety gate is formatted identically to "4 inline copies of a target guard" and "cleanup of invisible session files". The tracking system has no severity axis, so nothing in the data says one of these can let a write reach ~/.claude/CLAUDE.md unapproved and the others cannot.

That is the finding. Model choice does not fix a missing severity axis.

Second occurrence, same class. #63 found a gate silently not firing because the command layer was untested (--repo <scan-target> dropped the gate to silent, 29 removals, no approval asked). #65 finds a gate not firing because the engine never reads it. Two instances of "only prose stood behind a write gate" is a class, not luck (defect-found-in-one-file-is-a-class).

1. The class, measured

Question Measured 2026-08-12
Files in scanners/ that write to disk 9
…that import the scope gate 1 (lib/subtraction-write.mjs)
Command templates 21
…that name a write action 17
…that invoke write-scope-cli.mjs 5

The 8 ungated writers are not 8 bugs. write-output.mjs, backup.mjs, baseline.mjs, scan-orchestrator.mjs write plugin-managed artefacts and are legitimately exempt. The defect is that nothing declares which: "does this write path need the gate?" is answered by reading code, never by a guard. That is precisely what let fix-engine sit unguarded next to subtraction-write, which does it right.

2. What A+ means here, concretely

Not "more care". Three falsifiable properties:

  1. No invariant is enforced only by prose. Every contract a command template states about a write, a gate, or a scope is asserted by a test that fails when the code stops honouring it.
  2. Every open item carries a severity and a consequence sentence. "What breaks if this stays open" is written next to it, and anything touching a write, a gate, or user-scope config never lives in the backlog paragraph.
  3. Shipped ≠ committed. Work that is not released is not quality: the machine runs the released plugin, so 29 unreleased commits are 29 fixes nobody has.

3. Chunks, in order

Q1 — the gate moves from prose into code (BLOCKS the release)

fix-engine calls classifyWriteTarget + strongestGate, exactly as subtraction-write already does — share the constant, do not copy it (two copies of one table drift). Add an explicit exemption table naming every plugin-managed writer and why it is exempt. Verify: a guard that walks scanners/ for write calls and fails on any writer that neither imports the gate nor appears in the exemption table. Seen RED against today's tree first.

Q2 — the command layer gets contract tests

The 17 templates that name a write are today verified by nothing. Build the argv from the template's own text (dogfood-the-command-not-the-cli) and assert: the command a template tells the agent to run parses, targets the file the gate classified, and calls the gate before any write. Verify: delete the write-scope-cli line from one template → its test goes red.

Q_AUDIT — one Fable session: find the rest of the class

A cross-cutting sweep for other invariants that exist only in prose (agent prompts, command templates, .claude/rules/), each rated by what breaks if it silently stops holding. This is review/big-picture work — Fable's documented form strength and a first choice, not a fallback. Output: a rated list, not code. A Fable session runs without advisor.

Q3 — severity axis in the tracking (cheap, rides along)

STATE.md open items become a table with severity + consequence. Rule: safety/write/user-scope items are never in the backlog paragraph. This is the fix for the actual root cause.

Q4 — release v6.0.0

29 commits, 21 of them feat/fix, including breaking ID semantics (7a794b4). Only after Q1. Gate: self-audit --check-readme + full suite + check-versions.mjs 0 ERROR.

Then B3 (two-layer CNF), as planned.

4. Model routing for this plan

Chunk Model Why
Q1, Q2, Q3 Opus 5 / high implementation with strong verification (tests fail loudly)
Q_AUDIT Fable 5 / xhigh cross-cutting review + planning; deliberate override of the rubric, recorded in STATE as an override, no advisor
Q4 release Opus 5 / high mechanical but one-way (a pushed tag)

Not a blanket model upgrade. Escalating every session to compensate for a missing guard is the expensive way to not fix the guard.