open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

docs(plan): the gate defect was a tracking defect, not a model defect (v6 quality plan)

Browse source 
The unenforced scope gate in fix-engine was already written down in STATE's
open-items paragraph, formatted identically to "4 inline copies of a target
guard". A stronger model reading that paragraph reaches the same conclusion,
because nothing in the data says one item can let a write reach
~/.claude/CLAUDE.md unapproved and the others cannot. The missing thing is a
severity axis, not reasoning power.

Second occurrence of one class: #63 was a gate not firing because the command
layer was untested; #65 is a gate not firing because the engine never reads
it. Two instances of "only prose stood behind a write gate".

Measured, not asserted: 9 writers in scanners/, 1 imports the gate; 21 command
templates, 17 name a write, 5 invoke write-scope-cli. The 8 ungated writers
are mostly legitimate — the defect is that nothing declares WHICH, so the
question is answered by reading rather than by a guard.

Plan: Q1 gate into code + explicit exemption table (blocks the release), Q2
contract tests built from each template's own text, Q_AUDIT one Fable session
to find the rest of the class, Q3 severity axis in tracking, Q4 release
v6.0.0. Model routing per chunk — not a blanket upgrade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mCkx9wGywqNQzsXkBMzJ1
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 20:48:52 +02:00
commit e60b80978b
1 changed files with 96 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

96
docs/v6-quality-plan.md Normal file
View file

@ -0,0 +1,96 @@
# v6 quality plan — from "green suite" to A+
**Written 2026-08-12 (session #65), after a gate defect surfaced by accident rather than by
process.** This plan is about the *process* that missed it, not about the one defect.
## 0. Root cause — and why a bigger model is not the fix
The defect (`fix-engine.mjs` writes files without ever consulting the scope gate) was **already
written down**. It sat in `STATE.md`'s ÅPNE POSTER paragraph, mid-sentence, between "P6/M-BUG-44"
and "M-BUG-26":
> *scope-gaten for de FEM andre armene er fortsatt prosa-kontrakt — `fix-engine` leser ikke `gate`*
It was read at session start and not acted on. **A stronger model reading the same paragraph
reaches the same conclusion**, because the paragraph gives it no reason to: an unenforced safety
gate is formatted identically to "4 inline copies of a target guard" and "cleanup of invisible
session files". The tracking system has **no severity axis**, so nothing in the data says one of
these can let a write reach `~/.claude/CLAUDE.md` unapproved and the others cannot.
That is the finding. Model choice does not fix a missing severity axis.
**Second occurrence, same class.** #63 found a gate silently not firing because the *command
layer* was untested (`--repo <scan-target>` dropped the gate to `silent`, 29 removals, no
approval asked). #65 finds a gate not firing because the *engine* never reads it. Two instances
of "only prose stood behind a write gate" is a class, not luck ([[defect-found-in-one-file-is-a-class]]).
## 1. The class, measured
| Question | Measured 2026-08-12 |
|---|---|
| Files in `scanners/` that write to disk | **9** |
| …that import the scope gate | **1** (`lib/subtraction-write.mjs`) |
| Command templates | **21** |
| …that name a write action | **17** |
| …that invoke `write-scope-cli.mjs` | **5** |
**The 8 ungated writers are not 8 bugs.** `write-output.mjs`, `backup.mjs`, `baseline.mjs`,
`scan-orchestrator.mjs` write plugin-managed artefacts and are legitimately exempt. The defect is
that **nothing declares which**: "does this write path need the gate?" is answered by reading
code, never by a guard. That is precisely what let `fix-engine` sit unguarded next to
`subtraction-write`, which does it right.
## 2. What A+ means here, concretely
Not "more care". Three falsifiable properties:
1. **No invariant is enforced only by prose.** Every contract a command template states about a
write, a gate, or a scope is asserted by a test that fails when the code stops honouring it.
2. **Every open item carries a severity and a consequence sentence.** "What breaks if this stays
open" is written next to it, and anything touching a write, a gate, or user-scope config never
lives in the backlog paragraph.
3. **Shipped ≠ committed.** Work that is not released is not quality: the machine runs the
released plugin, so 29 unreleased commits are 29 fixes nobody has.
## 3. Chunks, in order
### Q1 — the gate moves from prose into code (BLOCKS the release)
`fix-engine` calls `classifyWriteTarget` + `strongestGate`, exactly as `subtraction-write` already
does — share the constant, do not copy it ([[two copies of one table drift]]). Add an **explicit
exemption table** naming every plugin-managed writer and *why* it is exempt.
**Verify:** a guard that walks `scanners/` for write calls and fails on any writer that neither
imports the gate nor appears in the exemption table. Seen RED against today's tree first.
### Q2 — the command layer gets contract tests
The 17 templates that name a write are today verified by nothing. Build the argv **from the
template's own text** ([[dogfood-the-command-not-the-cli]]) and assert: the command a template
tells the agent to run parses, targets the file the gate classified, and calls the gate before
any write.
**Verify:** delete the `write-scope-cli` line from one template → its test goes red.
### Q_AUDIT — one Fable session: find the rest of the class
A cross-cutting sweep for other invariants that exist only in prose (agent prompts, command
templates, `.claude/rules/`), each rated by what breaks if it silently stops holding. This is
review/big-picture work — Fable's documented form strength and a **first choice**, not a fallback.
Output: a rated list, not code. A Fable session runs **without advisor**.
### Q3 — severity axis in the tracking (cheap, rides along)
`STATE.md` open items become a table with `severity` + consequence. Rule: safety/write/user-scope
items are never in the backlog paragraph. This is the fix for the actual root cause.
### Q4 — release v6.0.0
29 commits, 21 of them `feat`/`fix`, including breaking ID semantics (`7a794b4`). Only after Q1.
Gate: `self-audit --check-readme` + full suite + `check-versions.mjs` 0 ERROR.
### Then B3 (two-layer CNF), as planned.
## 4. Model routing for this plan
| Chunk | Model | Why |
|---|---|---|
| Q1, Q2, Q3 | **Opus 5 / high** | implementation with strong verification (tests fail loudly) |
| Q_AUDIT | **Fable 5 / xhigh** | cross-cutting review + planning; deliberate override of the rubric, recorded in STATE as an override, no advisor |
| Q4 release | Opus 5 / high | mechanical but one-way (a pushed tag) |
**Not a blanket model upgrade.** Escalating every session to compensate for a missing guard is the
expensive way to not fix the guard.