Anthropic documents that Claude Opus 5 verifies its own work, and that telling
it to double-check or to delegate verification to a subagent causes
over-verification -- token cost with no quality gain. The general subtraction
detector (BP-SUB-001) already surfaces those blocks for every user, with no
model-awareness at all.
`optimize --subtract --for-model <name>` adds the missing half. It ANNOTATES a
subset of the candidates --subtract already produced; it is not a second
detector and can never widen the candidate set. A second SUBTRACT_DETECTORS
entry would have collided with BP-SUB-001 on de-dup, and a prose-only signal in
the agent prompt would have been untestable.
There is no auto-detection, by measurement rather than omission: a CLAUDE.md has
no frontmatter and no resolvable target model, and this operator's own `route`
skill deliberately runs a different model per session -- the same file is read
by whichever model comes next. So the model is named, and the citation is
reported as conditional everywhere a human sees it (agent report copy, and the
Step 7a listing that is the last surface before an approval file).
Precision comes from the TARGET, not the verb list. Measured across the
409-file corpus: 392 BP-SUB-001 candidates, 31 (7.9%) carry a verify verb, and
0 also carry a reflexive or delegated target. Two independent raw-text greps
found 0 as well, so the zero is the corpus rather than an over-narrow regex.
Those 31 verb-only blocks -- "sjekk relevante config-filer", "Type-sjekk:
pyright", "To verify plugin functionality" -- are exactly the false positives a
verb-only version would have produced, which is BP-JUDG-001's 7/7 failure
arriving one lens over. The numbers live in the register entry's note and are
pinned by a test, because a session that cannot see the measurement reads the
zero as a broken detector and loosens it.
`recognized` is reported separately from `matchedCount`: a typo'd model name and
a genuinely clean config both yield zero, and without the distinction the CLI
would report a silent no-op as good news. Dogfooded on the real machine --
`opus-5` gives recognized:true/matchedCount:0, `oppus5` gives recognized:false.
source.published is absent because the guide carries no visible publish date;
its absence is asserted so a later session does not invent one to match the
other entries' shape. Both quoted sentences were verified verbatim 2026-08-12.
The payload stays additive -- forModel and per-candidate modelScope appear only
under the flag, so a plain --subtract run is byte-identical to before (asserted
on the serialized bytes, since a key set to undefined passes a shallow check).
Suite 1724 -> 1752 (+28). The one remaining failure is the pre-existing
drift-cli --output-file crash, untouched by this work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRuXt6tZyowi8QYNKLSHQm
Removes the local copy (byte-identical baseline, md5 736fc9d6) and
repoints the README link to the canonical file in repo-standard, which
differs only by generalising plugin/marketplace wording to
repository/organisation. config-audit's class (plugin) has never
required GOVERNANCE.md in required_files, so nothing was gated on it.
Q1 was gate-in-prose, Q2 was argv-in-prose; the sweep's answer to 'what is
the third' is data-contract-in-prose: templates hand-build files (backup
manifest, session state) that engines and hooks later parse. Rated list of
9, topped by two recovery-path findings: the rollback engine has no CLI
entry (restore runs as model prose with a pre-rendered 'checksum verified'
line), and implement's hand-built manifest format is pinned only by a
hand-written fixture — the seam that already produced one success-shaped
no-op. Every number in the doc is command-produced (Fable session, no
advisor).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BeVSUuNqSCDUgTvfLKv4fK
A command template is a caller with no compiler behind it. It names a scanner and
an argv; nothing checked that the scanner still accepts them. M-BUG-45 measured
what that costs: `--stale-after` arrived malformed, was ignored, and the command
reported "all 14 entries re-verified within the last 90 days" about a threshold
the user had just overridden.
The new guard builds the argv from each template's OWN text (#63 — a hand-typed
call is a path no user takes), reading all three forms a flag appears in,
including the comment-only `GLOBAL_FLAG="" # --global`; that third form is the
one that dies unobserved, since the default path leaves the variable empty.
Measured: 38 invocations, 54 (CLI, flag) pairs, 15 CLIs, 0 dead scanner paths.
Two premises in the plan text were falsified by measuring:
- "the flag exists in the CLI's BOOL_FLAGS/VALUE_FLAGS" — only 3 of 34 scanner
files declare such a surface. The contract is checked on BEHAVIOUR instead:
run the CLI, ask whether it calls the flag unknown.
- `--full-machine` was predicted dead on `posture`. It is live. The fasit was
wrong, not the code.
What the measurement found instead: `campaign-export-cli` was the only one of the
fifteen without the shared `requireValidArgs` gate. Its hand-rolled chain guards
every value branch with `argv[i + 1] !== undefined`, so a trailing `--repo` fell
past all of them to the `startsWith('--')` catch-all and was reported as an
unknown flag — about the flag the CLI itself requires. Classification of "value
flag, no value" across all fifteen: 14 correct, 1 wrong. It now uses ARG_SPEC +
requireValidArgs like the other twelve; valid argv reaches the existing loop
byte-for-byte unchanged. Special-casing it in the test would have rebuilt, in
test code, the prose exception Q1 deleted.
And what the guard itself got wrong, which is worse than what it was looking for:
probing a flag means RUNNING the CLI, and some flags are writers. Its first run
let `drift-cli --save` default its target to the working directory and overwrite
the operator's real ~/.config-audit/baselines/default.json — an ungated write
outside the repo, produced by the guard whose whole subject is ungated writes
outside the repo. Every probe now runs under hermeticEnv() with its own empty
cwd, and the cwd is asserted empty afterwards. Isolation that is only a
convention is not isolation. Side effect: 65s -> 13s, because a hermetic HOME
stops every probe from enumerating ~/.claude.
All six arms seen RED against their own defect, twice — including the ORIGINAL
class (remove --approve-scope from fix-cli) and the plan's own verification
(delete the write-scope-cli line from a template). The non-emptiness arm is
derived from the tree, not pinned to a count that would only be a drift point.
Suite 1707 -> 1724, frozen v5.0.0 + default-output snapshots 0 changed files.
Not fixed here, found while verifying and pre-existing at 749b710: the suite was
NOT green on HEAD. output-file-robustness fails on drift-cli, root cause
diff-engine.mjs:194 — `m.from.severity` where `m.from` is undefined in the moved
section of the drift report. It crashes after the scan, in formatting, so the
CLI exits 3 with no output file. Its own chunk, not this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pj6UoTi6iPsAB2B2j6EZ1k
`write-scope.mjs` has existed since M-BUG-41, but only one writer ever called
it. Measured 2026-08-12: 9 files under `scanners/` write to disk, 1 imported
the gate; 21 command templates, 17 mention a write, 5 call `write-scope-cli`.
Five templates paraphrasing one policy is the shape that put the lever table in
five copies (#61) — one level up.
The defect was never "8 ungated writers = 8 bugs". Four of them write the
plugin's own bookkeeping and must STAY ungated: a gate that fires on every run
gets switched off, and then it guards nothing. The defect is that nothing
declared WHICH, so the question was answered by reading, and answered
differently each time it was asked.
`tests/lib/write-gate-coverage.test.mjs` makes the answer structural: every
writer either imports the gate or holds an EXEMPT entry naming where the bytes
land. Seen RED against today's tree before the fix (4 ungated writers), and
each of its four assertions was separately seen red against its own defect.
Two premises in the plan text were falsified by measuring them first:
- `scan-orchestrator` was carried as "plugin-managed, legitimately exempt".
`--save-baseline` derives its path from the SCAN TARGET, so `--global`
lands `~/.claude/.config-audit-baseline.json` — user-scope, require-ok. It
is gated. `lib/baseline.mjs` is the genuinely exempt one.
- the first sweep scored 9 writers with a regex that could not match
`writeFileSync(`, so `lib/backup.mjs` — a real writer — read as clean. The
guard covers sync and async forms, strips comments before matching, and
asserts non-emptiness so a regex that stops matching cannot make every
other assertion vacuously green (#63, #64).
Gated: fix-engine, rollback-engine, campaign-export-cli, scan-orchestrator.
All five call sites share ONE reduction, `evaluateWriteTargets` — four copies
of classify/strongestGate/dedup is the drift this exists to prevent.
`campaign export` still DISCLOSES rather than refuses: cross-repo is by design
there, and tightening it into a refusal would break the feature. A dry run is
still not a write, so it is never gated (#63). A refusal is a verdict about a
config that WAS examined, so it rides in the payload and keeps the 0/1/2 exit
contract (#62) — and the verdict now reaches the success payload too, since
stderr is discarded by `2>/dev/null` (F3's class).
commands/fix.md carries `--approve-scope` from the answer the user gives, with
the rule stated where it can be read: classifying is not approving.
Dogfooded end to end: a target outside the session root refuses with zero bytes
written, then applies under `--approve-scope`.
Suite 1703 -> 1707/0. Frozen v5.0.0 + default-output snapshots: 0 changed files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pkn22uGCgk6QZA738zNmHL
The unenforced scope gate in fix-engine was already written down in STATE's
open-items paragraph, formatted identically to "4 inline copies of a target
guard". A stronger model reading that paragraph reaches the same conclusion,
because nothing in the data says one item can let a write reach
~/.claude/CLAUDE.md unapproved and the others cannot. The missing thing is a
severity axis, not reasoning power.
Second occurrence of one class: #63 was a gate not firing because the command
layer was untested; #65 is a gate not firing because the engine never reads
it. Two instances of "only prose stood behind a write gate".
Measured, not asserted: 9 writers in scanners/, 1 imports the gate; 21 command
templates, 17 name a write, 5 invoke write-scope-cli. The 8 ungated writers
are mostly legitimate — the defect is that nothing declares WHICH, so the
question is answered by reading rather than by a guard.
Plan: Q1 gate into code + explicit exemption table (blocks the release), Q2
contract tests built from each template's own text, Q_AUDIT one Fable session
to find the rest of the class, Q3 severity axis in tracking, Q4 release
v6.0.0. Model routing per chunk — not a blanket upgrade.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mCkx9wGywqNQzsXkBMzJ1
Article rule 1 ("give Claude judgement instead of rules") gets its register
entry, and deliberately no detector. `BP-JUDG-001` carries `lensCheck: null`.
The cut between deterministic prefilter and prose judge was the open design
decision. It was settled by measurement, and the measurement declined both
halves:
- 409 real CLAUDE.md files (38488 lines, 8689 prose blocks): the caging class
fires 7 times, and all 7 are false positives ("rendered prose-side",
"naming is a flag on the class"). Verified along an independent grep path
that bypasses block-splitting and sentence-splitting entirely, in both word
orders: 5 lines and 1 line, none an instruction.
- The narrow variant (absolute + form noun + numeric cap) fired 8 times —
one duplicated block seen seven times across plugin caches, precision 0 %.
The pre-committed rule required 90 % over 20 distinct fires.
- Where the shape does occur — 45 lines across 4755 skill/agent/command files
— it is the author's editorial policy (emoji, sentence length, slide
titles). Nothing in the text separates that from a vendor's over-tight
guardrail, and the article's reasoning does not transfer: the model is not
the author of a user's config.
So no CA-OPT-002; finding-codes keeps OPT next-free = 2. The numbers live in
the entry's own `note`, so the next session does not re-derive the question.
Two premises the chunk falsified. The brief justified a separate axis by
saying these blocks sit inside `floor-exclusion`'s floor — but the article's
own canonical line carries no floor marker at all, so "inside the floor"
cannot define the axis (the corpus tendency is 76 %, which is a tendency, not
a mechanism). And the fasit's own form-noun vocabulary was wrong: `name` and
`format` alone drove 97 % of fires.
Not folded into `--subtract`: a third "loosen instead of delete" verdict in
the subtraction judge is the AS#5 mixing STATE forbids, and with the corrected
vocabulary there are 0 collisions to arbitrate anyway.
Guards, both seen red against their own defect first: the entry must exist,
be confirmed, date its source and name NO lensCheck; and every lensCheck in
the register must be backed by a real detector.
No behaviour changed — no new finding, no output change, nothing consumes the
entry yet — hence `docs`, not `feat`. Suite 1701 -> 1703/0; frozen v5.0.0 and
default-output baselines 0 changed files. Fasit:
docs/b2-judgment-lens-fasit.local.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mCkx9wGywqNQzsXkBMzJ1
`import-resolver` follows @import targets; a path written in ordinary prose
was checked by nothing. CA-CML-013 resolves those too — one finding per file,
severity low, against both the CLAUDE.md's own directory and the scan root,
because a nested file may legitimately write repo-root-relative paths.
The design work here is the SILENCE list, and every entry on it was measured
against 407 real CLAUDE.md files rather than argued for:
- Bare filenames excluded: admitting them tripled the output (2350 vs 810),
led by name-drops of tools that exist elsewhere on the machine.
- Org/repo slugs, npm packages, pytest node ids and prose enumerations
excluded: 111 fires, inspected, all false positives.
- Bare folder names excluded on the same reasoning one level up: 183 of the
remaining 699 fires (26%), led by `open/` — a Forgejo remote namespace
prefix, not a directory. This one overturned a premise the fasit had
asserted without measuring; the deviation is recorded rather than the
prediction quietly edited.
- Containment is checked against the scan root, not the file's own dir: a
base a `..` chain can escape is not a base. Measured — without it,
`../../../../etc/passwd` resolved to the real file and silenced its own
finding, while a legitimate `../docs/x.md` still resolves.
Rule ORDER is the reported reason (first match wins), so `npm test` is
silenced as a command rather than as a bare token, and two silences with
different causes keep their own fixtures. Twelve classes, pinned by name.
Both load-bearing rules were seen RED against their own defect: deleting
containment fails 1 test, deleting the slug rule fails 6.
Dogfooded through the argv the command template itself constructs, which
found a true positive in our own CLAUDE.md — `lib/humanizer.mjs` where the
file is `scanners/lib/humanizer.mjs`. Fixed here.
Suite 1662 -> 1701, 0 failing. Frozen v5.0.0 and default-output baselines:
0 changed files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HJbfM3N8zWQ1wA2voTrZxz
Found by review after the SUB-WRITE commit, and both defects were in the
template rather than the engine every prediction in the fasit was about.
`--repo` is what a write target is classified AGAINST. The template passed the
SCAN target, and under `--global` that target IS ~/.claude -- so
~/.claude/CLAUDE.md matched `in-repo` and the gate went `silent`. Measured
against the real config: gate silent, scopeClass in-repo, 29 removals applied
with no approval asked. That is the same silent downgrade #62 measured for a
naive .git-upward walk, arriving through a different door, on the one target
this chunk was sequenced behind M-BUG-41 to protect. Every other gated template
already passed `--repo "$PWD"`; this one was the only outlier.
The dry run also could not validate the machine-wide case -- the case that is
mandatory in v1. The gate returned before any file was read, so a dry run there
reported 29 scope-gate refusals and zero checked spans, and the first run able
to find a stale approval would have been the one that writes. A gate guards a
WRITE, and a dry run is not one: `requiresApproval` and the disclosures are
still reported, so the operator is still asked.
The new caller-arm guard was itself red against the corrected template, matching
prose that merely NAMES the CLI. Narrowed to lines that invoke it.
Guards seen red against the original defects: `--repo "<target-path>"` red,
`--repo` omitted red, gate-blocks-dry-run red. Re-dogfooded as the template now
calls it: require-ok / user-scope / 29 spans validated / 0 files written.
Suite 1659 -> 1662/0. Frozen baselines untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017A6vrtPKsVuM4DJ27p7jzw
`optimize --subtract` has only ever proposed. `--apply` executes the blocks the
operator picks, behind a backup whose coverage is verified and a scope gate the
engine enforces rather than describes.
The open design decision from plan §C6 was settled by two measurements, not by
taste. It is NOT a fix-engine action: the subtraction axis appears nowhere in
scan-orchestrator or optimization-lens-scanner, so verifyFixes' re-scan would
mark every removal `verified` whether or not it happened -- a success-shaped
no-op, the same shape that made restoreBackup silently do nothing. It is NOT a
plan/implement step either: that pipeline needs a finding code, and OPT declares
exactly one, for the deterministic check.
The approval artifact is written by main context, not by the lens agent. That is
where the operator's decision actually happens, and it keeps the feature off the
still-unmeasured agent write surface (M-BUG-18 lists optimize as open).
Three properties are load-bearing, and each was seen red against its own defect:
removals validate against the ORIGINAL content and apply in descending line
order; the range check is not redundant with the text check (`line: 0` makes
`slice(-1, 0)` empty, so an empty text MATCHES and `splice(-1, 1)` deletes the
file's last line); and createBackup skips a nonexistent path while still
returning an id, so manifest coverage is asserted before a byte changes.
Two guards were green on their own defect and were fixed after measuring:
`/\b80\s*%\b/` never matches "80% of the file" -- `%` is a non-word character, so
the trailing `\b` demands a word character next. And the caller-arm sweep passed
vacuously against HEAD, iterating an empty list; only the added non-emptiness
assertion caught it.
The floor is repeated, not moved: floor-exclusion still vetoes before anything is
proposed, and the engine refuses a load-bearing block again so a hand-built
approval cannot route around it. `mv` to `_archive/` is a file-level rule and
does not apply to a block excision -- the timestamped backup is the recovery
artifact, and a second copy with no restorer would be worse than none.
strongestGate moves into write-scope.mjs so the gate ordering has one owner.
Dogfooded DRY-RUN against the real ~/.claude/CLAUDE.md: 29 candidates, gate
refused all 29 with exit 0 until the scope was approved, then 29/29 spans
validated with nothing written. ~789 tokens, ~18% of the file -- corroborating
the #40 fasit's ~850, and well short of what a deletion feature is tempted to
promise.
Suite 1625 -> 1659/0. Frozen v5.0.0 and default-output baselines untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017A6vrtPKsVuM4DJ27p7jzw
The chain observed configuration across repos but presented every write it then
proposed as though it landed where the session stands. STATE named two arms;
measuring found five, and two of them are worse than the two already known:
- implement — the approval prompt named NO path at all, only a count, so a plan
editing ~/.claude/CLAUDE.md and one editing ./CLAUDE.md produced byte-identical
prompts.
- rollback — the file list rendered `.claude/settings.json`, a repo-relative
FORM, while the restore writes to the absolute original. The other arms were
silent; this one pointed the wrong way.
- fix — paths were visible but unclassified, and --global mixed machine-wide and
project rows into one unmarked table.
The gate's strength comes from the target's scope class, never from the command
asking: five command-owned policies would drift apart the way five copies of the
lever table did. SCOPE_CLASSES is one source for class, gate, wording and
predicate; templates render `disclosures[]` from the CLI instead of restating
what a class means.
Two orderings in that table are load-bearing, and both were measured:
- plugin-managed before user-scope. Both ~/.claude/config-audit/ and the legacy
~/.config-audit/ are live, and every command writes session state there. The
other order fires the gate on every write ever made and gets it switched off,
which is worse than no gate.
- user-scope before cross-repo. ~/.claude/.git EXISTS, so a plain .git-upward
walk answers "another repo" for ~/.claude/CLAUDE.md and silently downgrades
the strongest gate on the subtraction axis's primary target to disclosure.
disclose is not require-ok: campaign export is cross-repo by design, so the gate
there says so rather than refusing. Distinct from require-target-dir.mjs, which
asks whether a scan ROOT is readable (exit 3) — a different invariant, left
unmerged along with its four inline copies.
Also structural, both found while building this: the hand-maintained GUARDED
list in the unknown-flag sweep now derives its completeness from the directory
(measured complete at 14 of 14 first, so nothing was hiding — but the 15th CLI
would have been swept by nothing); and prose shape-guards use whitespace-
tolerant patterns, after one went red against a command file that did say the
right thing, line-wrapped.
Gated: implement, fix, rollback, plan, campaign export. Suite 1596 -> 1625/0,
frozen v5.0.0 and default-output baselines 0 changed files. No new GAP dimension,
no lever, no finding code — utilization denominators untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013941cEohSD5Aw56FVAtBgZ
Operator decision 2026-08-10 (session #61): the subtraction axis' write half is
wanted in use, so it moves ahead of C3/B2/B3. M-BUG-41 (scope-gate) is promoted
with it as the prerequisite — subtraction-write targets ~/.claude/CLAUDE.md,
which is by definition outside the repo the session stands in, and that is
exactly the gate M-BUG-41 is missing on its two measured arms.
Adds §C6 recording the frames the chunk inherits rather than invents: the
deterministic floor runs before the judge, ~/.claude is archived by mv and never
rm, user level is mandatory in v1, and the honest sizing is ~20% of the file —
not the 80% the framing invites. The apply mechanism is deliberately left open,
to be settled in the chunk's fasit against the gate rather than before it.
Also corrects the release level in the list: MAJOR (v6.0.0), since M-BUG-28
shipped with a BREAKING CHANGE footer that outranks the minor the additions
alone would have implied.
Records the rejection of a sibling /repo-reinit skill so it is not revived: it
would be a third implementation of one judgement alongside this axis and
/doctor Check 3, and regenerating a CLAUDE.md destroys the floor that makes a
mature one valuable. repo-init already owns the fresh-repo case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq3nye21RVYk4pZLeT8pGz
New GAP finding CA-GAP-028: authored subagents exist and not one of them names
`model:` or `effort:`, so every delegated task runs on the main conversation's
model (`model` defaults to `inherit`). Cites BP-MODEL-001/002, landed in C1.
`whats-active` and `manifest` now carry `model`/`effort` per agent.
Shipped as a conditional LEVER rather than a 25th dimension, and the choice was
made by measurement: as a t3 dimension the agent-less marketplace-medium fixture
would count it vacuously-present, moving the denominators 41->42 and utilization
44->45 — which flips `segment` "Developing"->"Competent" in the frozen v5.0.0
posture baseline, a field strip-retired-gap.mjs does not mask. A lever never
enters those denominators. The general rule is now an invariant in CLAUDE.md.
One check across both axes, not one per axis: it fires only when neither is used
anywhere, so a deliberate everything-on-one-model policy stays silent. Cost is
recall, chosen for precision.
Found by dogfooding, fixed red-first: `model: inherit` is the documented default
spelled out, so it must not count as routing — otherwise a config opts out of the
opportunity without changing anything real.
Two pre-existing defects surfaced and closed on the way:
- The humanizer guard asserted TRANSLATIONS.GAP.static EQUALS the dimension
titles, which forbade humanizing any lever — all three existing levers fell
through to the generic "feature opportunity" default, wrong for a budget lever.
Guard now requires coverage of every emittable title, seen red against those
three before the entries were written.
- Two hand-written copies of the lever list (finding-codes guard, humanizer
guard) merged into one exported LEVERS registry carrying code AND title.
- suppression-validation pinned CA-GAP-028 as an unoccupied number; C4 claimed
it. Fixed structurally with a derived first-free id, not by picking a new
literal — same class as #60's "bump this again".
Suite 1596/0. Frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq3nye21RVYk4pZLeT8pGz
C1: two entries the optimization lens can cite for the model/effort axis,
both read out of the primary sources in this session rather than from the
plan's 2026-07-14 summary of them.
Verifying corrected the plan's own numbers: effort is settable in SIX
places, not five (/effort, the /model slider, --effort,
CLAUDE_CODE_EFFORT_LEVEL, settings effortLevel, skill/subagent
frontmatter), the default is high on every supporting model EXCEPT Opus
4.7 (xhigh), and the level count is model-dependent (Opus 4.6 and Sonnet
4.6 have no xhigh).
- BP-MODEL-001: subagent `model` defaults to `inherit`, so a subagent
that names no model costs what the session costs; the documented pin
is overridable by CLAUDE_CODE_SUBAGENT_MODEL and per-invocation model
- BP-MODEL-002: effort is an axis separate from model choice, and higher
is not universally better (`max` "may show diminishing returns and is
prone to overthinking")
- both carry the 2026-07-07 model/effort blog as a corroborating source
with a real `published` date, so B1's evidence-age rule has teeth:
measured stale with reasons ['evidence-age'] at a reference date 378
days past publication while their verified stamps are pristine. The
docs pages themselves get NO published date — they carry none, and
guessing one in the field whose whole job is dating evidence is the
lie the rule exists to catch
- new blanket guard: every corroborating source must carry a parseable
published date. Without it newestEvidenceMs() returns null and the
entry stays green on evidence of any age — a silent hole. Seen red
against its own defect before it was trusted
- knowledge-refresh-cli's stale branch no longer expires on every new
entry: its reference date has to sit after every `verified` stamp, was
bumped once for BP-SUB-001 and would have needed a third bump now, so
it moves to a date no stamp can reach
Register 14 -> 16 entries. Frozen v5.0.0 snapshots untouched (no scanner
output changes); suite 1579/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKjojcdYYiCQP5AudUyQ5e
The three guards landed with M-BUG-28 verify key->number (registry), that every
emitted code is declared (sweep), and that every declared code is claimed by a
call site (orphan check). None of them verifies title->number: that the call
site at PLH source position 3 passes `plugin-json-shadows-default` and not its
neighbour. Transposing two keys that are both valid satisfies all three.
Measured, not assumed: with the two PLH keys swapped, finding-codes.test.mjs,
finding-code-coverage.test.mjs and the orphan check all stayed GREEN. Only this
test goes red.
It reads the ID off a real scan and keys on the finding TITLE -- the assertion
README actually makes. Covers the three places where numbering is deliberately
not source order (CA-PLH-015 and CA-PLH-016 at source positions 3 and 4,
CA-TOK-006 at position 8) plus CA-CML-001. The PLH-016 case also pins the
documented non-uniqueness: four entry problems, one check, one ID.
Suite 1573 -> 1577, 0 failing. Frozen v5.0.0 untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MyqCQKK2ornJ1jFWwqx17E
BREAKING CHANGE: the {NNN} in CA-{SCANNER}-{NNN} identifies the check that
produced the finding. It used to be the finding's position in that scanner's
output for that run, which made it unstable across CONFIGURATIONS, not just
across releases as STATE framed it. Measured on two fixtures: "No custom
subagents" was CA-GAP-007 on minimal-project and CA-GAP-004 on healthy-project.
A user who fixed an unrelated earlier gap silently renumbered every later one,
so a .config-audit-ignore pin retargeted to a neighbouring finding with no
version change at all.
Second measured arm: README already documented the opposite scheme. It and the
scanner headers describe ~20 numbers as check codes (CA-SKL-003 = oversized
body, CA-PLH-015 = folder shadowing, CA-TOK-006 = schema deferral), and the
counter could only produce those in the all-fire case -- source-order positions
are 4, 3 and 8. The documentation described the scheme; the implementation was
what was wrong. Every published number is preserved by construction and pinned
exhaustively in tests/lib/finding-codes.test.mjs.
scanners/lib/finding-codes.mjs is the single authority. Every finding() call
passes a `code`; an undeclared or missing one THROWS. No counter fallback --
that would reproduce D1's findGapId -> 'unknown' silent degradation and let a
half-converted scanner ship IDs that look valid. findingCounter/resetCounter
are deleted outright, not left as no-ops. Retirement is now a mechanism:
RETIRED_CODES tombstones a withdrawn key so its number is never reissued,
seeded with GAP t3_8 -- the D1 removal that opened this chunk.
IDs are consequently NOT unique per finding: one check failing in three files
emits three findings sharing an ID. That inverts which consumer is correct, so
every f.id/findingId site was classified before the change. diff-engine and
most of fix-engine already keyed on scanner+title+file (drift was never lying);
fix-engine's verification did not, and keyed on the ID alone -- fixing one of
two sibling instances marked both fixed, and the untouched one, still present
in the re-scan, was reported as a REGRESSION. Red test first, then keyed on
(findingId, file), which both planFixes and applyFixes already carry.
plugin-health's crossIds Set was measured and is a clean negative: cross
findings are allFindings.slice(crossPluginStart) and codes 18/19 are emitted
only in that tail, so the partition holds by construction.
unknownSuppressions() reports a pin that names no declared check, in the
--output-file payload (ux-rules rule 2 -- a stderr-only warning is invisible to
the commands) and only when one exists, so a clean config is byte-identical.
That is what makes the break safe: a stale pin goes loud instead of dying quiet.
Frozen tests/snapshots/v5.0.0/ untouched on disk. IDs are masked out of that
comparison (mask-finding-ids.mjs) rather than re-derived -- re-deriving
positional IDs would assert the retired scheme against itself, and #58's
isGapEntry off-by-one is the measured example of that misfiring. The dead
re-derivation is removed from strip-retired-gap.mjs. default-output snapshots
re-approved after confirming the diff is IDs and nothing else.
Guards, each seen red against its own defect: a missing code (scanner errors
out mid-sweep), an orphan declaration, a resurrected retired key, and a
documented ID naming no check. The sweep asserts the union across all 16
scanners, never per scanner -- a per-scanner assertion goes green on a partial
conversion.
Fasit written before implementation: docs/mbug28-id-semantics-fasit.local.md,
including one correction made before running (CML has 12 checks over 13 call
sites -- the anchored and calibrated char-budget arms are one check, which a
repeated-title sweep found and my call-site count had missed).
Suite 1535 -> 1573, 0 failing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MyqCQKK2ornJ1jFWwqx17E
CC 2.1.226's /doctor Check 8 covers auto mode with usage-weighted judgement.
The binding positioning forbids carrying a feature whose whole value is
duplicating a /doctor check, so the "adopt this feature" nudge goes. The
deterministic side stays: SET still validates autoMode structure and still
flags it as dead config in shared project settings. GAP dimensions 25 -> 24.
The title lived in FOUR tables, not the two the removal was scoped against:
the dimension list, scoring TITLE_TO_ID, the humanizer's static translations,
and the scoring denominators (TIER_COUNTS t3 8->7, TOTAL_DIMENSIONS 25->24,
MAX_WEIGHTED 42->41) -- the one that moves a user-visible number. findGapId
falls back to 'unknown' silently, so a partial removal would have degraded
without failing. A blanket sync invariant now asserts all four against
GAP_CHECKS instead of comparing occurrences pairwise; each arm was verified
red against its own defect (denominator drift, orphaned humanizer entry,
resurrected dimension).
Frozen tests/snapshots/v5.0.0/ stays untouched. strip-retired-gap.mjs is the
removal twin of strip-added-scanner.mjs: it strips the retired dimension from
whichever side still carries it and re-derives GAP IDs, since retiring a
dimension from mid-list shifts every later ID by one. Derived utilization
figures are dropped from comparison rather than recomputed -- recomputing them
in a test helper would assert the new arithmetic against itself, and
scoring.test.mjs already pins them exactly. Re-seeding was rejected: it would
silently bake in any other drift across every scanner those four files cover.
risk_score, risk_band, verdict, overallGrade, maturity and segment are
byte-identical across the change (severity info carries zero risk weight; GAP
is excluded from the overall grade). Utilization shifts 43 -> 44 on the fixture.
D2 (CA-SKL-002) is NOT removed. Verified against the primary source first: the
CC changelog carries exactly one budget-fraction statement (L3786, 2.1.32) and
nothing supersedes it, so our 2% is current and 002 is not a duplicate with a
stale figure. /doctor's ~1% could not be reconciled from the changelog and it
discloses its own numbers as disk estimates, so it is recorded, not adopted.
Left explicitly unverified in a code note: L3786 says "character budget" while
we express tokens -- a 4x difference nobody can settle from the wording.
Suite 1531 -> 1535, all green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RsfPGxgwbR3MY54wDC6hat
`KNOWN_OPEN` in cli-unknown-flag-rejection.test.mjs named two CLIs as still
carrying the argument-swallow defect. That number was the previous session's
field of view, not a measurement. Measuring all fourteen found **7** open on
the unknown-flag arm and **10** on a second arm the deferral note never
described.
Arm 1 — unknown flag: with no `else` branch, `--zzz` leaves no trace. exit 0,
full payload, a confident answer to a question the caller did not ask.
Arm 2 — the sharper one: `a === '--output-file' && args[i + 1]` asks only
whether a next token EXISTS, never whether it is a value. `manifest`,
`campaign-cli` and `knowledge-refresh-cli` each wrote a file literally named
`--json` into the caller's working directory when handed `--output-file
--json`, exit 0, with `--json` mode silently dropped. A wrong answer is bad;
an unintended file on disk is worse.
Two of the CLIs this catches were already in GUARDED and green on arm 1 while
arm 2 stood open a few lines away — the guard asserted one relation instead of
the invariant.
Fixed with a shared gate (`lib/cli-args.mjs`) that runs BEFORE each CLI's own
parse loop rather than replacing it: valid argv reaches the existing parser
byte-for-byte unchanged, so the byte-stability argument is structural rather
than empirical. `drift-cli`, `fix-cli` and `plugin-health-scanner` were
already correct on both arms and were moved into GUARDED instead of rewritten.
The three CLIs with a bespoke unknown-flag branch had it removed once the gate
made it unreachable.
Suite 1488 → 1531. Frozen v5.0.0 snapshots untouched; `self-audit
--check-readme` passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014P6Rh59Mtj4uYrdYMCYZJE
`fix-engine.mjs` carried its own copy of the valid `effortLevel` list, and
that copy had gone stale on `xhigh` (CC 2.1.154's top Opus tier) while
`settings-validator.mjs` had all five. The nearest-match "fix" therefore
corrected `xhig` — and `XHIGH` — to `high`: the tool silently changed the
tier the user asked for, in the one code path whose whole job is to write
the corrected value back to disk.
Fixed by sharing the validator's table instead of aligning the copy, so the
two cannot drift again. Guard asserts the blanket invariant — a one-character
near-miss on EVERY valid tier corrects back to that same tier — rather than
pinning the one level that happened to be missing; verified red against the
original stale array.
Suite 1488/0. Frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014P6Rh59Mtj4uYrdYMCYZJE
Surfaced by the router dogfood: quoting the placeholder stopped the shell from
swallowing it, which moved the failure down into the CLIs — and revealed that
most of them never check the target at all. Measured:
node scanners/posture.mjs /nonexistent/path/xyz --output-file …
exit 0
Health: B (86/100) — Good shape — a few items to address
Nothing in that output distinguishes it from a real audit: well-formed
envelope, all 10 areas present, 16 opportunities reported. A typo'd path did
not fail — it flattered.
Exit 3 is the right code by the plugin's own contract: 0/1/2 are PASS/WARNING/
FAIL about a configuration that WAS examined, and every command template gates
on exactly that distinction, so a bad path flowed through the whole workflow as
a clean result.
This was a consistency gap, not a design question. Measured across the nine
target-taking CLIs, four already did it right with the same message and the
same exit code (manifest, token-hotspots-cli, whats-active, optimize-lens-cli);
five did not (scan-orchestrator and drift-cli exit 1, posture,
plugin-health-scanner and fix-cli exit 0). The five now share
lib/require-target-dir.mjs, which carries that exact behaviour. The four with
inline copies are left alone — consolidating them is a cleanup, not part of
this fix.
The guard is asserted over ALL nine CLIs, so a new one cannot join the wrong
half, and a third case is covered: a target that exists but is a regular file.
A valid target — including an empty directory — is explicitly unaffected.
Suite 1483 -> 1486, frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
Dogfooding `/config-audit` (the router) against the repo, fasit written before
any run (docs/router-fasit.local.md, untouched). Every claim below is measured
behaviour, not a reading of the source.
1. Bare `<target-path>` inside the step-3 fence is a shell REDIRECTION, not an
argument. Measured in zsh: both CLIs failed before starting, no output file
was written, and the echoed status was 1 — inside the band the router's own
gate calls "continue normally". Quoting makes an unsubstituted placeholder
reach argv, so it fails in the CLI where the exit code means something.
Swept the whole class: 30 sites across 12 further command files, since a
defect in one file is a class until the opposite is measured. New guard:
command-placeholder-shell-safety.test.mjs.
2. The orchestrator's exit code was discarded. Two commands on one line share a
single trailing `echo $?`, which reports only the last: measured, an
orchestrator exit 3 echoed as posture's 0, so the "3 -> stop" gate could
never fire. Both statuses are now captured and echoed.
3. "Running 12 configuration scanners" — the orchestrator registers 16. The new
test binds the narrated count to the registry so the next scanner added
cannot re-stale it silently.
4. The Area Breakdown table hardcoded 7 rows; posture emits 9 quality areas.
Token Efficiency (a B on this repo) and Plugin Hygiene never reached the
user. Rows added, and the row set is now asserted against lib/scoring.mjs.
Label aligned: "MCP Servers" -> "MCP", as posture emits it.
5. Step 6 rendered "the headline line from the humanized stderr scorecard" and
forbade deriving a replacement — while step 3 sent posture's stderr to
/dev/null, as UX rule 2 requires, and the prose is absent from the JSON
payload (measured). The slot could only be improvised. posture's stderr now
goes to a file in the session dir, as commands/posture.md already did; the
user still never sees raw scanner output.
Also: `grep -q -- "--raw"` matched any argument CONTAINING --raw (measured on
`--rawdog` and on a path with --raw in it) — anchored to whole arguments.
SCOPE_FLAGS renamed SCOPE_FLAG, since zsh does not word-split and the plural
invited the M-BUG-45 shape.
command-shell-state-shape.test.mjs only recognised line-initial assignments, so
it reported the idiomatic `node …; STATUS=$?` capture as never assigned. Widened
to assignments after a separator; verified it still fails on a real cross-block
reference before trusting it.
Suite 1477 -> 1483, frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
D12 (org-ops, 2026-08-01) requires three things of the plugin-class
disclosure line: generator, process, and the ownership basis (Anthropic
Consumer Terms §4). Our line carried only the first two — the inline
rewrite in #54 closed catalog's original order, which was measurably
narrower than the decision it derived from.
Adopts the form already live verbatim in llm-security, voyage and
ai-psychosis. Position unchanged (head block, line 9); repo-standard
stays green and `self-audit --check-readme` passes. MIT verified
against LICENSE and .claude-plugin/plugin.json.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
repo-standard v0.2.0 flagged 7 badges, past the measured inflection of 5
(Trockman et al., ICSE 2018) where a badge row reads as clutter rather
than evidence.
Dropped Commands/Agents/Hooks: the counts are stated in the sections
they describe, and every one of them was a hand-synced number that could
silently go stale — the same defect class that made the tests badge wrong
(1441 vs 1477) before it was removed. Kept Version, Platform, Scanners
(the distinguishing number, and the one count `self-audit --check-readme`
still asserts against the filesystem) and License.
`self-audit --check-readme`: PASS, exit 0 — absent badges are skipped by
design, so removing three does not weaken the gate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oUzeHA1Kmi2z8ug83PzbS
repo-standard v0.1.1 gate reported 5 ERROR + 5 WARN + 1 SKIP. All ERROR
fixed, every WARN decided:
ERROR (fixed)
- HEADING-MISSING x3: added `## Install`, renamed "What This Plugin Does
Not Cover" -> `## Non-goals`, "Version History" -> `## Changelog`.
- INSTALL-NO-CLI: `marketplace add` now followed by the actual CLI
command `claude plugin install config-audit@ktg-plugin-marketplace`.
The `enabledPlugins` JSON stays beside it as the second form.
- README-DESC: opening line is now the published forge/plugin.json
description verbatim, so description == catalog == README.
WARN (fixed)
- README-H1: `# Config-Audit Plugin for Claude Code` -> `# config-audit`.
- BADGE-STATIC-CLAIM: dropped the static `tests-1441` badge — it asserted
a run nothing verifies, and it was stale (real count 1477). The
Testing section now carries the measured number plus the fact that no
CI runs it. self-audit --check-readme skips absent badges, still PASS.
- LINK-NON-REPO: `open/claude-code-llm-security` -> `open/llm-security`
(the org's rename).
SKIP (fixed, closes catalog's coord message)
- LINK-OUTSIDE-REPO README:7: the `../../README.md` disclosure link was a
monorepo leftover pointing outside the repo at an anchor that never
existed. Replaced with the inline text the polyrepo migration intended.
Also removed the "What's New in v5.4.0" section: 24 lines of release notes
as the first TOC entry on a v5.13.0 repo, duplicating the v5.4.0 row in
the changelog table it sat above.
Gate re-run: 0 ERROR, 11 checks passed. Full suite 1477/1477.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011oUzeHA1Kmi2z8ug83PzbS
Oppgave A measured (fasit-first, CC 2.1.220): /doctor overlaps our
judgment lenses and alarms, not the deterministic validators. Binding
outcome: 0 whole scanners removed, 2 measured function-duplicates
scheduled for removal (GAP autoMode dimension, CA-SKL-002 alarm role),
5 surfaces repositioned. Plan rewritten with D-chunks + B2-B4 merged
against the existing C-chunks and open dogfood posts; old rejection #5
(prose contradiction detection) superseded by B3 with evidence.
README gains the division-of-labor section vs /doctor; CLAUDE.md gains
the never-duplicate-a-doctor-check invariant.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NeaMRXVGzh9oSwigJDjE9
assessFreshness aged only the entry's own source.verified stamp, so an
entry re-verified against an old source stayed green while a newer source
sat unnoticed (BP-SUB-001 was stamped 2026-07-31, a week after the
superseding-grade article of 2026-07-24 was published). The stamp
certifies the old source; it says nothing about the evidence.
- entries may carry corroborating sources[] with published dates
- new evidence-age rule: stale when the NEWEST published date across all
sources exceeds evidenceStaleAfterDays (default 365); re-verifying the
old source never clears it, only newer evidence does
- source.supersededBy marks a replaced source: stale regardless of stamp
- stale items now carry reasons[] (verified-age / no-verified-date /
superseded / evidence-age)
- BP-SUB-001 gains the 2026-07-24 context-engineering article as a
verified corroborating source (near-verbatim coverage). NOT added to
BP-MECH-*/BP-SIZE-001: own verification found no mechanism-choice or
size-limit content in the article, contrary to the brief's assumption.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NeaMRXVGzh9oSwigJDjE9
Dogfooding `campaign` + `knowledge-refresh` against a throwaway ledger. Seven
defects, all found by running the commands as written and measuring, not by
reading them.
The headline pair only existed together. `knowledge-refresh` built
`STALE_AFTER="--stale-after 30"` and expanded it unquoted, trusting the shell to
split it in two. bash does; zsh — the macOS default, and what the Bash tool runs
here — does not. The CLI got one argv entry, matched no flag, and because it had
no unknown-flag branch, silently kept the 90-day default and reported "✓ All 14
register entries were re-verified within the last 90 days": a true-sounding
sentence about a threshold the user had just overridden. Fixing either half alone
leaves a silent wrong answer or a loud one; both are fixed, and a guard now
rejects any template that packs a flag and its value into one variable.
`knowledge-refresh` also read one register and wrote another: step 6 named an
unanchored `knowledge/best-practices.json` while the CLI reads
`${CLAUDE_PLUGIN_ROOT}/…`, which for an installed plugin is the cache. The
validation gate then ran the cached test against the cached register — green no
matter what was written. The two copies were byte-identical that day, which is
exactly why it was invisible.
`campaign` vouched for repos it could not read. `add /finnes/ikke` returned
`added` + exit 0; `refresh-tokens` then put the phantom in `swept[]` with a
0-token delta and left `skipped[]` empty, so the machine-wide bill claimed
coverage of three repos on a machine with two. Paths stay tracked — an unmounted
volume is a legitimate absence — but are reported as `addedUnverified`, and the
command names them.
Two class sweeps, both measured rather than assumed. `posture` was the single
scanner (1 of 14) whose fatal catch exited 1, which ux-rules defines as a normal
WARNING grade — a crash indistinguishable from a result. And all 13 payload
writers failed on a `--output-file` whose parent did not exist, which on a fresh
machine turned `campaign`'s first run into "the ledger may be corrupt"; they now
share `scanners/lib/write-output.mjs`.
Predicted breadth was too wide for the first time in five sessions: 6 of 8 CLIs
predicted to lack unknown-flag rejection, 4 measured. `drift` and `fix` already
reject them, via a construct the grep did not recognise — a grep matches an
implementation, the invariant is a behaviour. The sweep was rewritten to run each
CLI with a bogus flag and read the exit code.
Suite 1453 → 1469/0. Frozen snapshots untouched. `optimize-lens-cli` and
`token-hotspots-cli` share the unknown-flag defect and are deferred to the v5.14
argument-handling chunk with their positional-swallow arm; the count is recorded
in the guard rather than rounded down to zero.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012NHWjN8EnoxSqRvMTLK2NE
Dogfooding the four read commands (posture, tokens, manifest, whats-active)
surfaced four defect classes, all in the seam between what a command template
promises and what the scanner behind it actually does.
M-BUG-40, fifth arm: posture wrote four temp files it could never read back.
#49 closed the $$/cross-block class in four commands, but posture survived it —
and so did the guard written to prevent exactly this. The guard compared each
$$ path to the block that created it, so a path written once and then read via
prose had no second occurrence to flag. Measured live: written from PID 21614,
read attempted from PID 23772. The invariant is now blanket (no $$ in any temp
path), which also caught fix.md and feature-gap.md.
M-BUG-43: 6 of 7 scanners write their payload to stdout when --raw/--json is
set even when --output-file was given, and the templates redirected only
stderr. Measured: posture 255 182 B, whats-active 35 922 B, drift 28 316 B,
manifest 23 825 B, tokens 8 768 B. fix and feature-gap never read the file they
wrote, so both recovered one letter grade from a quarter-megabyte dump.
tokens swallowed --json and --with-telemetry-recipe: documented, never
threaded, so --json returned the humanized payload where the docs promise
byte-stable v5.0.0 output.
M-BUG-42: manifest's render contract asked for {load}; the payload carries
loadPattern, so the Load column rendered blank for all 96 rows.
Four new tests (1449 -> 1453), each verified red before the fix. The
render-contract test checks {field} names against a live payload from a
fixture, since a hardcoded key list would drift. Frozen v5.0.0 snapshots
untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VGCk9o27eWo9uXLjkZTXEq
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
process.exit() terminates immediately, but Node writes stdout asynchronously
when stdout is a pipe — everything still buffered is dropped. scan-orchestrator
measured 246 854 bytes to a file against 65 536 to a pipe (131 072 on another
run; the cut point is a flush race), so every machine consumer that pipes the
envelope got truncated, unparseable JSON. The failure reads like a corrupt file,
not like a cut-off, which is what made it survive this long. Reported by
org-ops, whose census pipes our output.
Closes the class rather than the one CLI where it was visible. campaign-cli,
campaign-export-cli, campaign-write-cli, knowledge-refresh-cli, drift-cli and
fix-cli all exited the same way on their success paths and were green only
because their payloads fit the pipe buffer today; size is not correctness. All
38 sites across 14 files now set process.exitCode and return, which is the
pattern self-audit.mjs already used.
Two contracts needed care rather than substitution: fail() is a never-returns
guard at ~25 call sites, so it throws a CliUsageError the top-level catch
renders with the identical "Error: " prefix and exit code 3; the path guards
needed an explicit return so main() stops instead of running on. Exit codes and
stderr text are unchanged, and the frozen v5.0.0 snapshots are untouched.
The class sweep is landed as a test, not as fourteen edits — it caught one site
this commit had missed. Suite 1443/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8sS1DuDV6bUJcyumLwbvj
DEL B chunk `interview` (+ discover/status/cleanup/help). Fasit written before
the run predicted 8 defects and refuted 4 candidates; all 8 confirmed, all 4
refutations held, and three predictions turned out too narrow.
- M-BUG-36: `drift --list` reached the command as 0 bytes. drift-cli accepted
--output-file but list mode ignored it, and the listing goes to stderr, which
the command discards per ux-rules rule 2. Fixing the caller alone would not
have helped.
- M-BUG-37: feature-gap's "Create backup" step ran fix-cli without --apply.
Dry-run is the default, so no backup existed (backupId: null) while the
command went on to edit config believing it could roll back.
- M-BUG-38: fix-cli told users to recover with scanners/rollback-cli.mjs, which
does not exist. Dead reference in the one message read after a bad fix.
- M-BUG-21 fourth arm: five templates carried literal [--global]/[--full-machine]
inside executable bash blocks. A bracketed placeholder does not start with a
dash, so every scanner's arg loop takes it as the scan target.
- interview and analyze never said which session they act on; interview could
rewind a finished session; cleanup interpolated an unvalidated id into rm -rf
(an empty id deletes every session); status advertised a `resume` command that
does not exist and documented an `all` argument it never parsed.
TDD: 9 red tests first, including a machine sweep for dead /config-audit
references and for bracketed flags in bash blocks. Suite 1432 -> 1441/0.
Frozen v5.0.0 snapshots untouched; --raw/--json contracts unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGvA1uUQn2hPBPMaCKK6x3
Dogfooding `/config-audit plugin-health` against a fasit registered before the
run: 11 of 12 predictions confirmed, 1 refuted with evidence, 0 deviations.
The command's default path could not produce the report it documents.
M-BUG-21 (third arm): the argument loop ended in
`else if (!args[i].startsWith('-')) targetPath = args[i]` with no unknown-flag
branch, so `--output-file /tmp/x.json` was dropped and its value became the scan
target. Worse than in drift-cli: a non-existent path discovers no plugins, so the
scanner answered "No plugins found" (info) with exit 0 — a reassuring answer, not
an error. Unknown options and a value-less `--output-file` now exit 3.
M-BUG-33: the scanner had no `--output-file` and its default-mode report goes to
stderr, which `commands/plugin-health.md` discards with `2>/dev/null` before
telling the agent to read stdout. Zero bytes captured.
M-BUG-34: per-plugin rows and the grade formula never left `scan()` — the only
grade code, `formatPluginHealthReport`, had no caller — and cross-plugin findings
were flattened behind a `category` they share with per-plugin findings. The
mandated table and Cross-Plugin section were unbuildable, so the command had to
fabricate them. `scanDetailed()` now returns them; `scan()`'s frozen v5.0.0
envelope is unchanged by construction.
M-BUG-35: `.claude-plugin/marketplace.json` was flagged as an unknown file. It is
the documented catalog location, and `"source": "./"` makes the repo root its own
plugin, so one `.claude-plugin/` legitimately holds both.
Also: `commands/posture.md` ran both optional scanners in default mode under
`2>/dev/null` and read stdout — the same class as feature-gap.md:133 in the fix
chunk. A CLI-side flag fix does not close its callers.
Tests 1420 -> 1432, red first. Frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XhhZ8zg1amR7YCAPqiZWdt
Dogfooding `/config-audit fix` against a throwaway repo copy. All eight
predictions registered in the fasit before the run were confirmed, and three
further defects surfaced that were not predicted.
- M-BUG-21, third arm: the argument loop ended in `!arg.startsWith('-') =>
targetPath`, so an unknown flag was dropped and its value became the target.
In `fix` that is the WRITE target under `--apply`. Unknown options and a
value-less `--output-file` now exit 3.
- `--dry-run` was documented in the command's argument-hint and never
implemented; `--output-file` did not exist, so `commands/fix.md` told the
agent to Read a file nothing produced. Both now exist.
- M-BUG-31: `file-rename` was excluded from the backup set, so a renamed rule
file had no backup entry while the command promised one and returned a
backupId that could not restore it.
- M-BUG-32: `verifyFixes` hardcoded `includeGlobal: false`, so after a
`--global` run every untouched user-scope finding was reported as verified.
Reproduced against an unmodified ~/.claude/CLAUDE.md.
- M-BUG-29: a rename was applied before other fixes on the same file, which
then failed with ENOENT while the run still exited 0. Renames sort last.
- M-BUG-30: `severityOrder[s] || 4` maps critical (0) to 4, so critical fixes
sorted last. The old test used the same falsy fallback and agreed with the
bug. Now `?? 4`.
- A failed fix exits 2 instead of 0, matching the other scanners' convention.
Frozen tests/snapshots/v5.0.0/ untouched; --json/--raw stdout byte-identical.
Suite 1420/0 (+10).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ3MCDCnyw7wZSPnUXVhYS
Dogfooding `/config-audit drift` against the machine. Fasit written before the
run; 6/6 predictions plus both F7 arms confirmed, 0 deviations.
M-BUG-21 (both arms):
The arg loop ended in `else if (!arg.startsWith('-')) targetPath = arg` with no
unknown-flag branch, so an unrecognised flag was dropped silently and its VALUE
became the scan target. `--output-file /tmp/x.json` scanned /tmp/x.json — a path
that does not exist — and reported the near-empty scan as drift, forever. The
same silence was destructive for `--save --name` with the value omitted: the
name stayed `default` and an existing baseline was overwritten. And the flag
ux-rules rule 2 requires did not exist at all: commands/drift.md ran the CLI
under `2>/dev/null` while telling the agent to read stdout, but the default
report, the --save confirmation and --list all write to stderr. All three modes
captured nothing.
M-BUG-27 (found during the run, not predicted):
diff-engine never compared the baseline's stored target_path against the current
target. The machine's `default` baseline is anchored to a test fixture, so
`/config-audit drift` diffed two unrelated trees, marked all 20 baseline
findings resolved and all 15 current ones new, and reported trend "improving".
A reassuring, entirely false signal — and the default path.
Root cause is one thing, not three: the CLI validated neither its flags nor its
anchor. Same class as the rollback chunk's "nothing agreed where a backup lives".
Fix: unknown options and value-less --name/--baseline/--output-file exit 3;
--output-file follows the posture.mjs pattern; `_baselineAnchor` rides in the
default-mode payload (stderr alone is invisible under `2>/dev/null`) while
--json/--raw stdout stays v5.0.0-shaped.
Suite 1410/0 (+12). Frozen snapshots untouched; raw/json/default backcompat green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RGrL3noVdFSUhohaKTMSN
Batch release of everything since v5.12.5: one new lens mode and 14 real bugs,
all of them dogfooding finds — either from running the plugin against the
maintainer's real machine, or from walking analyze -> plan -> implement ->
rollback end-to-end on a throwaway repo copy.
Minor, not patch. STATE recorded this batch as "fix: only"; the log says
otherwise — e9921d3 ships `optimize --subtract`, a user-facing opt-in flag, so
semver requires a minor. Verified by reading `git log v5.12.5..HEAD` rather than
trusting the note: 11 fix, 1 feat, 8 docs.
Consequence: the planned v5.13 work (model routing, effort awareness, dead
references) now targets v5.14. docs/v5.13-model-routing-effort-deadref-plan.md
keeps its filename so existing references resolve, and says so at the top —
leaving a doc named for a version that shipped something else is exactly the
dead-reference class that plan is about.
Gates, all re-run against ground truth before writing anything:
- node --test 'tests/**/*.test.mjs' -> 1398 pass / 0 fail
- scanners/self-audit.mjs --check-readme -> passed (tests badge 1344 -> 1398;
the path in STATE said scripts/, which does not exist)
- catalog check-versions.mjs -> 0 ERROR
Counts unchanged: scanners 16, agents 7, commands 21, hooks 4.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AA5LT1UaDzctNkMi414qzA
Pipeline step 4 dogfood. `/config-audit rollback` could not see a single one of
the four real backups on this machine, and reported "Backup not found" for one
that was sitting right there.
Four defects, one root: nothing agreed on where a backup lives or what its
manifest looks like.
- M-BUG-22 `lib/backup.mjs` resolved `~/.config-audit/backups` (pre-v2.2.0)
while every command, agent and doc uses `~/.claude/config-audit/backups`.
The auto-backup hook and fix-cli wrote to the first, implement to the second,
rollback read only the first. Canonical root now, with the legacy root kept
readable so older backups stay listable and restorable (`legacy: true`).
- M-BUG-25 `parseManifest` understood only the engine's quoted `original_path:`
spelling, but implement hand-builds its manifest with `- backup:`/`original:`/
`sha256:`. Every implement-made backup parsed to zero files and restoreBackup
returned `{restored: [], failed: []}` — a success-shaped no-op. Both formats
parse now, and a manifest with unparseable entries throws instead of
pretending to succeed.
- M-BUG-23 both session hooks watched `~/.config-audit/sessions`, which does not
exist; sessions live under `~/.claude/`. "Check for active sessions" had never
fired once. It fires now.
- M-BUG-24 the suite called createBackup() against the developer's real home —
it had left nine stray backups there, and cleanupOldBackups() deletes past ten.
Root is overridable via CONFIG_AUDIT_BACKUP_ROOT; both test files use it.
Rollback still cannot delete files implement CREATED — no backup can hold a file
that never existed. It no longer does so silently: manifests carry a `created:`
list, restoreBackup returns `createdNotRemoved`, and rollback.md requires the
report. Automatic deletion is a destructive action and needs its own design.
Verified against backup 20260717_032636 on a throwaway copy: all three files
restore byte-exact (sha256 match), zero writes outside the copy, backup dir
unmodified. Suite 1382 -> 1398/0; frozen v5.0.0 snapshots untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SejM9RQAa1Hfuq7Ek2WfFr
Two record fixes, no behaviour change.
The README quoted "35 blocks, 13 genuinely ambiguous" for the hand-built ground
truth. Counting its own rows gives 48 classified blocks and 19 ambiguous — the
headline in the fasit disagreed with the table beneath it, apparently by
collapsing letter-suffixed sub-blocks for the summary while listing them
separately. Labels are untouched in both files; only the counts are restated,
and now as row counts, which are reproducible with a grep rather than by
recounting a classification.
The dogfood gate script that proves the blocking §8 criterion lived only in the
session scratchpad, which would have made "zero load-bearing blocks proposed,
11/18 groups, ~756 tok" unverifiable claims the moment the session ended —
precisely the premise-not-fact class this repo's own rules warn about. It now
lives at scripts/dogfood-subtraction-gate.local.mjs, gitignored via a new
*.local.mjs pattern because it indexes the operator's private config by line
number and must never reach the public mirror.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RW2haJXbxZpKivKHseSXNh
Every command so far asked an addition question — what to add, what to move,
what it costs. Nothing asked what is no longer earning its always-loaded rent.
This adds that axis as a fourth lensCheck on the existing hybrid motor rather
than a new scanner or a 22nd command: the measured payoff (~18% of one file)
justifies a mode, not machinery.
It is the only lens that proposes REMOVING config, so it carries a guarantee
the others don't need: a load-bearing block is never a candidate. Precision is
asymmetric — a missed dead line costs a few tokens per turn, a deleted one
costs a wrong remote or a broken script — so the floor is decided in code
(lib/floor-exclusion.mjs) before the opus judge sees anything, never in prose.
Granularity is the leaf block, with two structural exceptions: a paragraph
ending in ':' merges with the list it introduces, and an ordered list is a
contract whose steps inherit floor from any sibling. Unordered lists
deliberately do not inherit — a load-bearing bullet and a disposable one
routinely share a list, and container-reasoning is the error the hand-built
ground truth exists to catch.
Verified against that ground truth (built before any classifier existed), with
the comparison machine-checked rather than read by eye: zero load-bearing
blocks proposed, 11/18 deletable groups surfaced, ~756 tok ~ 18% of a ~4300
token file — inside the pre-registered band. The first run found five floor
violations the synthesized fixture missed; each got a structural rule and a
fixture shape so it cannot regress.
Three real bugs the dogfood run exposed, all now covered:
- JS \b is ASCII-only, so /\bunngå\b/ never matches — every Norwegian keyword
ending in æ/ø/å was silently dead.
- A bare word/word is not a path; "pros/cons" vetoed the largest deletable
block until PATH_RE was tightened to rooted paths and globs.
- "Mid-sentence" must key on a preceding lowercase letter; the loose version
read **bold labels:** and quoted openers as entities, costing 4 of 11 groups.
BP-SUB-001 is grounded entirely in the Anthropic steering blog already cited by
BP-MECH-001..004 and asserts nothing from the talk that motivated the feature —
no "80%", no ablation figure.
Suite 1365 -> 1382/0. Frozen v5.0.0 snapshots untouched; plain optimize output
byte-identical on identical input (--subtract adds keys only when passed).
knowledge-refresh-cli's reference date moved to 2026-08-01: its premise that
every seed entry was verified 2026-06-20 expired when BP-SUB-001 got a genuine
verification date, and backdating the entry to fit the test would have been a
lie about when its source was checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RW2haJXbxZpKivKHseSXNh
Ran the brief's own §8 pre-build checks before writing any code, and two of
them changed the design.
Age signal (§5A, §8): per-line git blame over four real instruction files gives
single-date shares of 88% / 55% / 100% / 58%. Instruction blocks trace back to
bulk commits, and blame reports last-touch rather than vintage — a reformatting
commit (this repo's own 96e32df) makes old instructions look young, so the
signal is biased, not merely sparse. That also kills the mtime fallback. §8
pre-registered this exact outcome and its consequence, so shape A does not ship
as a CA-VIN-* vintage scanner.
Premise correction (§6.1): ~/.claude IS git-tracked as of 2026-07-26 (7 commits,
remote on an external backup volume, 47 files). The rule it justified — mv to
_archive/, never rm — stands on different grounds and is unchanged.
§7 q1: both scopes, user-level mandatory in v1 (the floor test is defined there
and the always-loaded cost sits there).
§7 q2: not deterministically classifiable — the deciding blocks require reading
content against container. Deterministic pre-filter -> precision-gated judge,
with floor-exclusion running BEFORE the judge so a load-bearing block is never a
candidate.
§7 q3: ships as /config-audit optimize --subtract, a fourth lensCheck class
emitting CA-OPT-* with a BP-SUB-001 register rule. No new scanner, no new
command, no badge bump, no snapshot risk. Proportionality decided it: the
hand-built fasit puts the honest payoff at ~850-1400 always-loaded tokens on a
~4300-token file (~20%, not 80%), with 26 of 34 blocks classified floor.
The fasit itself (34 blocks, 13 marked ambiguous per §7.2) is local-only — it
quotes the operator's global CLAUDE.md verbatim and this repo's only remote is
the public open/ mirror.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x2i9NDEtMR9FU7ufaCnBX
§8's floor gate rests on §6.0's classification test, which is a judgment call
rather than a mechanical one. The four named must-survive items ("only Forgejo",
"bash is 3.2", the test command, "~/.claude is not git-tracked") are clear-cut —
any mechanism gets them right, so a fasit built from them proves nothing.
The gate is actually decided by blocks like "Conventional Commits:
type(scope): beskrivelse" (local convention or a nag the model follows anyway?),
"commit ofte med beskrivende meldinger", or the model-routing rubric — local
policy that reads like generic advice. §7.2 now requires 3-5 such blocks in the
fasit deliberately.
Cross-references verified: every section pointer in the brief (§1-§8, §6.0, §5A,
§3.1, §7.2) resolves to an existing heading. Shipping a dead prose reference in
the brief that proposes detecting them would have been an odd artifact — that is
CA-CML's finding class, v5.13 chunk 3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
Operator corrected two things about the brief committed in 3086e8b/3252b51.
Provenance: the operator watched the recording and identifies Boris Cherny on
stage, so the attribution is confirmed by direct observation, not a channel's
claim. The verbatim figures (80 %, "more intelligent without the prompts") still
reach us through the summary's editing and stay at that confidence level. §1 now
carries both levels separately, and the register source string reflects the split
instead of flattening to "unverified".
Design: "start with what it must have" is the constraint the whole feature turns
on, so it is a hard constraint (§6.0), not a candidate-shape detail. Model
capability erodes compensatory instructions ("read the whole file first") and
does nothing to load-bearing local facts ("only Forgejo", "bash is 3.2", the test
command) — the model isn't failing at intelligence there, it cannot know. A tool
that treats them alike deletes the Forgejo constraint because Opus 5 "is smart
enough now". Rebuild is therefore three tiers: floor restored immediately, earned
returns on repeated stumbling, dead never comes back. Policy prohibitions stay in
the floor by decision rather than classification — asymmetric cost, cheap to keep.
Consequences threaded through: §5 disqualifies any shape that cannot express the
distinction, §7.2 becomes the core open question (age and class are independent
signals, so age alone can never carry the call), and §8 gains a blocking floor
test with a hand-built fasit and named must-survive items.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
Two corrections to the brief committed in 3086e8b.
The ordering question (§7.4) still pointed at STATE.md for a decision the
operator had already made, and STATE.md is gitignored — so the tracked artifact
carried a stale queue and deferred to a file a future session cannot read. It
now records the decision inline: delete-and-rebuild goes ahead of pipeline step
4, prior order stands underneath.
§8 gains the premise the brief was quietly resting on: §5A claims project-level
CLAUDE.md/rules yield usable per-block git ages. That is untested. If instruction
blocks trace to one bulk commit, the age signal carries no information and shape
A collapses to the phrasing heuristic — which would answer §7.2 for us. Checked
before any scanner code, not after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
Operator relayed the "delete your CLAUDE.md every six months" idea from a
third-party summary of a Boris Cherny talk. Assessed rather than adopted: the
provenance is secondhand and deliberately kept non-load-bearing (this repo has
one scar from treating a plausible quote as fact), while the feature is argued
from the repo's own logic.
The gap is real and verified, not assumed: feature-gap has no inverse, and grep
over scanners/ confirms nothing measures instruction AGE — 'stale' appears only
for knowledge-register entries and plugin-cache versions. drift's saveBaseline/
diffEnvelopes plus backup.mjs/rollback-engine.mjs are already the undo
machinery that makes deletion a measurement rather than a gamble.
CLI ground truth checked against claude --help: --bare (sets
CLAUDE_CODE_SIMPLE=1), --system-prompt, --setting-sources, --add-dir. So the
env var the video calls undocumented is a documented flag here, which is what
would make an ablation harness buildable.
Brief only — no code, no chunk breakdown, no version committed. Three candidate
shapes with a likely landing (deterministic CA-VIN vintage scanner first),
hard constraints, open questions, and testable verification criteria.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
implement.md spawns implementer agents in parallel batches, all appending to
the same implementation-log.md. Dogfooding showed agents satisfying 'Append
result to:' with a full-file Write — the last writer clobbered 4 of 6 entries.
Pin the mechanism in both contracts: append with Bash >> heredoc, never the
Write/Edit tool on the shared log. Shape tests pin the instruction in both
files (empirically verified: agents given the >> instruction appended safely).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ? -> [^/] replacement ran AFTER the {{GLOBSTAR_SLASH}} placeholder was
restored to '(?:/.+/|/)', corrupting the group opener '(?:' into '([^/]:' —
every rule pattern containing a mid-pattern '/**/' silently matched only the
zero-dir branch and live rules were flagged 'matches no files' (CA-RUL).
Found by dogfooding /config-audit implement on a throwaway repo copy: the
implementer agent's correct 'posts/**/post.md' rule was flagged dead.
Fix: run the ? replacement before placeholder restoration. Fixture outcomes
byte-identical; frozen v5.0.0 baselines untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Claude Code subagent harness instructs spawned agents NOT to write
report/summary/findings/analysis .md files — the parent reads the final
text message. Verified live: analyzer-agent skipped Write entirely and
returned the report inline, so analysis-report.md never landed on disk
and the plan/interview/status phases would find nothing to read.
New contract (orchestrator-writes pattern): analyzer-agent returns the
complete report as its final message; the analyze command saves it
verbatim to the session directory before presenting the summary.
Same class exists in plan/feature-gap/optimize/scanner agent pairs —
deliberately left for their own dogfood chunks (plan is judged as-is
first per the pipeline sequence).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CTontYwY5JGS4nL2AuiASy
Video-derived audit ('The Model Isn't the Moat') cross-checked against
primary sources. Verified: orchestrator+cheap-worker pattern and 5-level
per-agent effort tuning (official docs); rejected: the 'Fable low ≈ Opus
high' chart claim (contradicted by Anthropic's own pages). Five chunks:
register entries BP-MODEL-001/002, fix-engine xhigh hygiene, CA-CML dead
prose references, feature-gap model/effort opportunity, planner-agent
adversarial gate. Sequenced AFTER DEL B pipeline dogfood + batch release.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CTontYwY5JGS4nL2AuiASy
The agent-listing scanner (AGT) emits an always-loaded per-turn token cost
("Agent description is long, re-sent every turn in the always-loaded listing";
scanner category 'token-efficiency', "the dominant single always-loaded
source"). But SCANNER_TO_CATEGORY in humanizer.mjs had no AGT entry, so its
findings fell through to the 'Other' fallback (humanizer.mjs:140) — a bucket
that isn't even in the analyzer-agent's category list. Neither the scanner
prefix nor the per-finding category ('token-efficiency' is not in
CATEGORY_TO_IMPACT) resolved AGT to its true impact.
Same class as M-BUG-16/15: a finding type without its matching humanizer
mapping landing on a default that mismatches its own evidence. The analogous
SKL body finding correctly buckets "Wasted tokens"; AGT (the same always-loaded
token-waste mechanism) silently landed under the meaningless "Other".
Found during analyze-prep premise-verification of the linkedin-posts scan: the
3 AGT findings bucketed "Other" while the analogous SKL findings bucketed
"Wasted tokens".
Fix: add AGT: 'Wasted tokens' to SCANNER_TO_CATEGORY, alongside TOK/CPS/SKL.
RED-first (extended the Wasted-tokens category test to include AGT; the 'Other'
fallback test still uses a synthetic 'XXX' scanner, unaffected). Frozen v5.0.0
untouched (AGT post-dates it; humanizer bypassed for --raw/--json); no
default-output snapshot contains AGT -> 0 regen. Suite 1359/0.
Verified end-to-end on linkedin-posts: 3 AGT findings now "Wasted tokens", 0
"Other" remaining, 28 findings unchanged (category-only). All 16 orchestrator
scanner prefixes now covered by the category map (class closed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01683eAqVecv9VZfQzL8CQ9h
The skill-listing check emits a third finding for an oversized skill BODY
(v5.11 B7, RAW title "Skill body is large (loads on demand when the skill
runs)"). The body is an ON-DEMAND cost — it loads only when the skill is
invoked, not the always-loaded listing Claude reads every turn. The scanner is
careful to distinguish the two (RAW title + comment + evidence note).
But the humanizer-data SKL.static map had no entry for this title, so it fell
through to SKL._default ("A skill is using more of the listing budget than it
should"). The humanized title therefore claimed a listing-budget cost and
directly contradicted the finding's own humanized evidence ("loads ON DEMAND
only ... NOT every turn like the always-loaded listing") — the same internal
contradiction class as M-BUG-15/M-BUG-14, and the same "new finding type added
without a matching humanizer entry" gap the scanner checklist warns about.
Found by finding-granularity premise-verification of the linkedin-posts scan
before feeding it to the analyze pipeline (the prior session's pass focused on
the GAP findings and did not catch the SKL fall-through).
- humanizer-data.mjs: add SKL.static entry for "Skill body is large (loads on
demand when the skill runs)" with on-demand-correct title ("A skill's body is
large (it loads only when that skill runs)"), description, and recommendation.
No listing-budget language; tier1/tier3 forbidden-word checks pass.
- RED-first tests at both layers: humanizer.test.mjs (humanizeFinding path:
title is not the listing-budget _default, conveys on-demand body) and
humanizer-data.test.mjs (static entry exists, on-demand-correct).
RAW envelope unaffected (humanizer bypassed for --raw/--json), frozen v5.0.0
snapshots untouched, default-output fixtures contain no oversized-body skill so
no snapshot regen. Suite 1357->1359/0 (+2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01683eAqVecv9VZfQzL8CQ9h
