Kjell Tore Guttormsen
7df8e0d65b
Anthropic documents that Claude Opus 5 verifies its own work, and that telling it to double-check or to delegate verification to a subagent causes over-verification -- token cost with no quality gain. The general subtraction detector (BP-SUB-001) already surfaces those blocks for every user, with no model-awareness at all. `optimize --subtract --for-model <name>` adds the missing half. It ANNOTATES a subset of the candidates --subtract already produced; it is not a second detector and can never widen the candidate set. A second SUBTRACT_DETECTORS entry would have collided with BP-SUB-001 on de-dup, and a prose-only signal in the agent prompt would have been untestable. There is no auto-detection, by measurement rather than omission: a CLAUDE.md has no frontmatter and no resolvable target model, and this operator's own `route` skill deliberately runs a different model per session -- the same file is read by whichever model comes next. So the model is named, and the citation is reported as conditional everywhere a human sees it (agent report copy, and the Step 7a listing that is the last surface before an approval file). Precision comes from the TARGET, not the verb list. Measured across the 409-file corpus: 392 BP-SUB-001 candidates, 31 (7.9%) carry a verify verb, and 0 also carry a reflexive or delegated target. Two independent raw-text greps found 0 as well, so the zero is the corpus rather than an over-narrow regex. Those 31 verb-only blocks -- "sjekk relevante config-filer", "Type-sjekk: pyright", "To verify plugin functionality" -- are exactly the false positives a verb-only version would have produced, which is BP-JUDG-001's 7/7 failure arriving one lens over. The numbers live in the register entry's note and are pinned by a test, because a session that cannot see the measurement reads the zero as a broken detector and loosens it. `recognized` is reported separately from `matchedCount`: a typo'd model name and a genuinely clean config both yield zero, and without the distinction the CLI would report a silent no-op as good news. Dogfooded on the real machine -- `opus-5` gives recognized:true/matchedCount:0, `oppus5` gives recognized:false. source.published is absent because the guide carries no visible publish date; its absence is asserted so a later session does not invent one to match the other entries' shape. Both quoted sentences were verified verbatim 2026-08-12. The payload stays additive -- forModel and per-candidate modelScope appear only under the flag, so a plain --subtract run is byte-identical to before (asserted on the serialized bytes, since a key set to undefined passes a shallow check). Suite 1724 -> 1752 (+28). The one remaining failure is the pre-existing drift-cli --output-file crash, untouched by this work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BRuXt6tZyowi8QYNKLSHQm
295 lines
14 KiB
Markdown
295 lines
14 KiB
Markdown
---
|
||
name: config-audit:optimize
|
||
description: Optimization lens — config that works but would fit a better mechanism (procedure→skill, lifecycle→hook, path→rule, never→permission)
|
||
argument-hint: "[path]"
|
||
allowed-tools: Read, Write, Glob, Grep, Bash, Agent
|
||
model: opus
|
||
---
|
||
|
||
# Config-Audit: Optimization Lens
|
||
|
||
The "is the config **optimal?**" axis (vs. the health scanners' "is it
|
||
**correct?**"). It finds configuration that *works* but uses a mechanism a
|
||
better one would fit — and frames every one as a *Missed opportunity*, never a
|
||
mistake.
|
||
|
||
Mechanism-fit rules come from the provenance-stamped best-practices register
|
||
(`knowledge/best-practices.json`); only CONFIRMED rules are surfaced. The motor
|
||
is hybrid: a cheap deterministic pre-filter finds candidates, then the opus
|
||
`optimization-lens-agent` judges each in context (precision-gated).
|
||
|
||
## What the user gets
|
||
|
||
- **Procedures → skills** (deterministic, CA-OPT-001)
|
||
- **Lifecycle phrasing → hooks** (BP-MECH-001)
|
||
- **Unscoped path-specific instructions → path-scoped rules** (BP-MECH-002)
|
||
- **Absolute "never" prohibitions → permissions / hooks** (BP-MECH-004)
|
||
- **`--subtract`:** instructions that no longer earn their always-loaded rent (BP-SUB-001)
|
||
- **`--subtract --for-model <name>`:** of those, the ones a named model documents
|
||
as redundant — e.g. self-verification instructions on Claude Opus 5 (BP-PROMPT-001)
|
||
|
||
Each finding cites its register rule + source URL. A clean CLAUDE.md returns "no
|
||
opportunities" — that is a good result, not a failure.
|
||
|
||
## Implementation
|
||
|
||
### Step 1: Determine target
|
||
|
||
Split `$ARGUMENTS` into a path (first non-flag argument; default: current working
|
||
directory) and flags. Recognized flags: `--global` (include the user `~/.claude`
|
||
cascade in discovery), `--subtract` (add the subtraction axis, below),
|
||
`--for-model <name>` (annotate model-scoped candidates, below) and `--apply`
|
||
(execute approved removals — Step 7).
|
||
|
||
`--apply` only means anything alongside `--subtract`. If it is present without
|
||
it, say so and continue with the ordinary lens run:
|
||
|
||
```
|
||
`--apply` executes approved subtraction removals, so it needs `--subtract` too.
|
||
Running the ordinary lens; re-run with `--subtract --apply` to remove anything.
|
||
```
|
||
|
||
`--for-model <name>` has the same dependency. If it is present without
|
||
`--subtract`, say so and continue with the ordinary lens run:
|
||
|
||
```
|
||
`--for-model` annotates subtraction candidates, so it needs `--subtract` too.
|
||
Running the ordinary lens; re-run with `--subtract --for-model <name>`.
|
||
```
|
||
|
||
**`--subtract` — the inverse question.** Every other lens asks what to *add* or
|
||
*move*; this one asks what no longer earns its always-loaded rent. It is opt-in
|
||
because it asks something different, and because deleting is not undoable by
|
||
reading. Pair it with `--global` to reach the user-level CLAUDE.md, where the
|
||
always-loaded cost actually sits (it loads in every repo, every session).
|
||
|
||
**`--for-model <name>` — whose redundancy?** Some instructions are only dead
|
||
weight for a *particular* model. Anthropic documents that Claude Opus 5 verifies
|
||
its own work and over-verifies when told to double-check or to delegate
|
||
verification to a subagent. That claim is model-scoped, so the model must be
|
||
named: a CLAUDE.md carries no frontmatter and no target model, and the same file
|
||
is read by whichever model the next session happens to run. The flag never adds
|
||
candidates — it annotates ones `--subtract` already found. Example:
|
||
`--subtract --for-model opus-5`.
|
||
|
||
If `--subtract` is present, say so up front:
|
||
|
||
```
|
||
Also running the subtraction axis — instructions that cost tokens every turn
|
||
without telling me anything I couldn't work out. Load-bearing local facts
|
||
(remotes, versions, paths, policy) are excluded before anything is judged.
|
||
```
|
||
|
||
Tell the user:
|
||
|
||
```
|
||
## Optimization Lens
|
||
|
||
Looking for configuration that works but would fit a better Claude Code mechanism...
|
||
```
|
||
|
||
### Step 2: Run the lens CLI
|
||
|
||
Generate a session ID (`YYYYMMDD_HHmmss`) if no active session exists.
|
||
|
||
```bash
|
||
mkdir -p ~/.claude/config-audit/sessions/{session-id} 2>/dev/null
|
||
GLOBAL_FLAG=""
|
||
if echo "$ARGUMENTS" | grep -q -- "--global"; then GLOBAL_FLAG="--global"; fi
|
||
SUBTRACT_FLAG=""
|
||
if echo "$ARGUMENTS" | grep -q -- "--subtract"; then SUBTRACT_FLAG="--subtract"; fi
|
||
# --for-model takes a VALUE, so flag and value are two separate variables. Never
|
||
# pack them into one ("--for-model x"): the shell here is zsh, which does not
|
||
# word-split an unquoted expansion, so one variable would reach the CLI as a
|
||
# single malformed argv entry and be silently ignored (M-BUG-45).
|
||
FOR_MODEL_FLAG=""
|
||
FOR_MODEL_VALUE=""
|
||
if echo "$ARGUMENTS" | grep -q -- "--for-model "; then
|
||
FOR_MODEL_FLAG="--for-model"
|
||
FOR_MODEL_VALUE=$(echo "$ARGUMENTS" | sed -n 's/.*--for-model \([^ ]*\).*/\1/p')
|
||
fi
|
||
node ${CLAUDE_PLUGIN_ROOT}/scanners/optimize-lens-cli.mjs "<target-path>" --output-file ~/.claude/config-audit/sessions/{session-id}/optimize-lens.json $GLOBAL_FLAG $SUBTRACT_FLAG $FOR_MODEL_FLAG $FOR_MODEL_VALUE 2>/dev/null; echo $?
|
||
```
|
||
|
||
Exit code 0 is normal. Only exit code 3 is a real error → "The lens couldn't run.
|
||
Check that the path exists and contains a CLAUDE.md."
|
||
|
||
### Step 3: Read the payload
|
||
|
||
Read `~/.claude/config-audit/sessions/{session-id}/optimize-lens.json` with the
|
||
Read tool. It has `deterministic` (already-confirmed OPT findings), `candidates`
|
||
(pre-filter candidates with register provenance), `register`, and `counts`.
|
||
|
||
Under `--subtract` it also has a `subtract` block (`candidates`, `register`) and
|
||
`counts.subtractCandidates`. Each subtraction candidate spans `line`–`endLine`
|
||
(a whole block). Include the whole `subtract` block when spawning the agent.
|
||
|
||
Under `--for-model` the `subtract` block additionally has `forModel`
|
||
(`{ requested, recognized, matchedCount }`), and a subset of its candidates carry
|
||
`modelScope` (`{ registerId, claim, requestedModel }`). If `recognized` is
|
||
`false`, the register does not cover that model name — tell the user before
|
||
showing results, so a zero is never read as "your config is already clean":
|
||
|
||
```
|
||
I don't have a model-specific rule for "{requested}", so nothing was annotated
|
||
for it. The ordinary subtraction results below are unaffected.
|
||
```
|
||
|
||
**Early exit:** if `counts.deterministic === 0` and `counts.candidates === 0`
|
||
(and, under `--subtract`, `counts.subtractCandidates === 0`), skip the agent and
|
||
tell the user plainly:
|
||
|
||
```
|
||
✓ No mechanism-fit opportunities found.
|
||
|
||
Your CLAUDE.md holds facts, not procedures/automation/prohibitions that would be
|
||
better as skills, hooks, rules, or permissions. Nothing to change here.
|
||
```
|
||
|
||
Then go to Step 5.
|
||
|
||
### Step 4: Spawn the precision gate
|
||
|
||
Tell the user what's happening and set expectations:
|
||
|
||
```
|
||
Found {counts.candidates} candidate line(s) + {counts.deterministic} deterministic finding(s).
|
||
Asking the optimization-lens agent to judge each in context (~20-40 seconds)...
|
||
```
|
||
|
||
Spawn the `optimization-lens-agent` (Agent tool) with:
|
||
- the full payload from Step 3 (deterministic + candidates + register),
|
||
- the session directory path so it can write `optimization-lens-report.md`.
|
||
|
||
The agent reads the actual CLAUDE.md, drops low-confidence candidates, and keeps
|
||
only genuine opportunities — each citing its register rule + source.
|
||
|
||
### Step 5: Present results
|
||
|
||
Read the agent's `optimization-lens-report.md` and present it formatted
|
||
(markdown tables / grouped sections). Follow the UX rules: never show raw JSON or
|
||
scanner progress; lead with a one-sentence summary of what was found before the
|
||
detail. Make clear these are LOW-severity *opportunities*.
|
||
|
||
If the agent kept nothing from the candidates (all dropped) but there were
|
||
deterministic findings, show those; if it kept nothing at all, show the clean
|
||
result from Step 3.
|
||
|
||
### Step 7: Apply approved removals (`--subtract --apply` only)
|
||
|
||
Skip this step entirely unless BOTH flags are present and the agent kept at
|
||
least one subtraction finding. Removal is the only thing this plugin does that
|
||
takes configuration away, so nothing here happens without a named choice.
|
||
|
||
**7a — show what is on the table, with honest sizing.** List the kept
|
||
subtraction findings numbered, each with its file, line span and first line of
|
||
text. A finding carrying `modelScope` must show its condition here too — this
|
||
is the last point a human sees it before it reaches an approval file, so the
|
||
citation must not read as unconditional:
|
||
|
||
```
|
||
{n}. {file}:{line}-{endLine} — {first line of text}
|
||
Model-scoped: redundant if you are targeting {requestedModel}. Other
|
||
sessions on this config may run a different model.
|
||
```
|
||
|
||
Do not imply a bigger win than there is:
|
||
|
||
```
|
||
Removing all of these saves roughly {n} tokens per turn — on a typical
|
||
always-loaded CLAUDE.md that is around a fifth of the file, not most of it.
|
||
```
|
||
|
||
Ask which to remove: numbers, `all`, or `none`. `none` ends the command.
|
||
|
||
**7b — write the approval file.** With the **Write** tool, write the operator's
|
||
choice to `~/.claude/config-audit/sessions/{session-id}/subtraction-approved.json`
|
||
(absolute path — a relative one resolves against the user's CWD). Take `file`,
|
||
`line`, `endLine` and `signalText` verbatim from the Step 3 payload; `text` must
|
||
be the `signalText` byte-for-byte, because the engine refuses a removal whose
|
||
text no longer matches the file:
|
||
|
||
```json
|
||
{ "sessionId": "{session-id}",
|
||
"removals": [ { "file": "...", "line": 12, "endLine": 15, "text": "..." } ] }
|
||
```
|
||
|
||
**7c — dry run first.** Always. It costs one call and proves the spans still
|
||
match before anything is written.
|
||
|
||
`--repo` is the **session's own root (`$PWD`), never the scanned path**. It is
|
||
what the target is classified *against*: pass the scan target and
|
||
`~/.claude/CLAUDE.md` classifies as `in-repo`, which drops the gate to `silent`
|
||
on the one target that most needs it (measured — the same silent downgrade as a
|
||
naive `.git`-upward walk, arriving through a different door).
|
||
|
||
```bash
|
||
node ${CLAUDE_PLUGIN_ROOT}/scanners/subtraction-write-cli.mjs --approved ~/.claude/config-audit/sessions/{session-id}/subtraction-approved.json --repo "$PWD" --dry-run --output-file ~/.claude/config-audit/sessions/{session-id}/subtraction-dryrun.json 2>/dev/null; echo $?
|
||
```
|
||
|
||
Read the payload. Exit 3 is a real error (bad or unreadable approval file).
|
||
Report any `refused` entry with its `reason` before going further —
|
||
`block-mismatch` means the file changed since the scan (re-run the lens),
|
||
`floor` means the block is load-bearing and will never be removable.
|
||
|
||
**7d — the scope gate.** If the dry-run payload has `requiresApproval: true`,
|
||
show every line in `disclosures` verbatim and ask for an explicit go-ahead. This
|
||
is the machine-wide case (`~/.claude/CLAUDE.md`): the change costs — and saves —
|
||
in every project, on every turn, so it is not the same decision as editing the
|
||
CLAUDE.md in front of you. Without a clear yes, stop here.
|
||
|
||
**7e — apply.** Same command without `--dry-run`, adding `--approve-scope` only
|
||
if the operator gave that go-ahead in 7d:
|
||
|
||
```bash
|
||
node ${CLAUDE_PLUGIN_ROOT}/scanners/subtraction-write-cli.mjs --approved ~/.claude/config-audit/sessions/{session-id}/subtraction-approved.json --repo "$PWD" --output-file ~/.claude/config-audit/sessions/{session-id}/subtraction-result.json 2>/dev/null; echo $?
|
||
```
|
||
|
||
**7f — report.** From the result payload, tell the user: what was removed (file
|
||
+ line span + the text, from `applied`), what was refused and why (`refused`
|
||
with `reason`), and how to undo it:
|
||
|
||
```
|
||
Backed up as {backupId} — `/config-audit rollback {backupId}` restores every
|
||
file exactly as it was.
|
||
```
|
||
|
||
Never report a run as successful when `counts.applied` is 0.
|
||
|
||
### Step 8: Next steps
|
||
|
||
End with context-sensitive next steps, explaining WHY each is useful:
|
||
|
||
- `/config-audit plan` — turn the kept opportunities into an action plan with
|
||
backups before you change anything.
|
||
- `/config-audit feature-gap` — the complementary lens: features you *don't* use
|
||
yet (this command is about mechanisms you *do* use that could fit better).
|
||
- Re-run `/config-audit optimize` anytime after editing CLAUDE.md.
|
||
|
||
## Notes
|
||
|
||
- This command is **agent-driven and not byte-stable** — its output is a
|
||
human-facing report, deliberately outside the deterministic snapshot suite.
|
||
- `--subtract` **proposes; only `--apply` writes**, and only blocks the operator
|
||
named. Every removal is preceded by a backup whose manifest is verified to
|
||
cover the file being written, and `/config-audit rollback` restores it.
|
||
- **`--for-model` annotates; it never detects.** It tags a subset of the
|
||
candidates `--subtract` already produced, and there is no auto-detection by
|
||
design: a CLAUDE.md has no frontmatter and no resolvable target model, and the
|
||
same file is read by whichever model the next session runs. So the citation is
|
||
always **conditional** — it must be shown that way in the report and in the
|
||
Step 7a approval listing, never as a settled fact about the file.
|
||
- **Removal is not a `fix` action and not a `plan`/`implement` step**, by
|
||
measurement rather than preference: the subtraction axis never enters the
|
||
orchestrated envelope, so `fix`'s re-scan verification would mark every
|
||
removal verified whether or not it happened, and the findings pipeline would
|
||
require a finding code — which names a deterministic check, not a prose
|
||
judgement. `subtraction-write-cli.mjs` owns the execution instead.
|
||
- The subtraction floor is deterministic and runs *before* the agent, so a
|
||
load-bearing block is never a candidate. It errs toward keeping: on a
|
||
well-maintained config this axis is mostly a no-op, and that is a good result.
|
||
- The deterministic half (CA-OPT-001) also rides in the normal orchestrated
|
||
audit; this command adds the prose-judgment half on top.
|
||
- Without `--apply`, no files are modified. To act on a mechanism-fit finding,
|
||
use `/config-audit plan` → `/config-audit implement` (backup + rollback) or
|
||
edit by hand.
|