docs(plan): age signal measured dead; close §7 q1-q3 on optimize --subtract

Ran the brief's own §8 pre-build checks before writing any code, and two of
them changed the design.

Age signal (§5A, §8): per-line git blame over four real instruction files gives
single-date shares of 88% / 55% / 100% / 58%. Instruction blocks trace back to
bulk commits, and blame reports last-touch rather than vintage — a reformatting
commit (this repo's own 96e32df) makes old instructions look young, so the
signal is biased, not merely sparse. That also kills the mtime fallback. §8
pre-registered this exact outcome and its consequence, so shape A does not ship
as a CA-VIN-* vintage scanner.

Premise correction (§6.1): ~/.claude IS git-tracked as of 2026-07-26 (7 commits,
remote on an external backup volume, 47 files). The rule it justified — mv to
_archive/, never rm — stands on different grounds and is unchanged.

§7 q1: both scopes, user-level mandatory in v1 (the floor test is defined there
and the always-loaded cost sits there).
§7 q2: not deterministically classifiable — the deciding blocks require reading
content against container. Deterministic pre-filter -> precision-gated judge,
with floor-exclusion running BEFORE the judge so a load-bearing block is never a
candidate.
§7 q3: ships as /config-audit optimize --subtract, a fourth lensCheck class
emitting CA-OPT-* with a BP-SUB-001 register rule. No new scanner, no new
command, no badge bump, no snapshot risk. Proportionality decided it: the
hand-built fasit puts the honest payoff at ~850-1400 always-loaded tokens on a
~4300-token file (~20%, not 80%), with 26 of 34 blocks classified floor.

The fasit itself (34 blocks, 13 marked ambiguous per §7.2) is local-only — it
quotes the operator's global CLAUDE.md verbatim and this repo's only remote is
the public open/ mirror.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x2i9NDEtMR9FU7ufaCnBX
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 15:43:03 +02:00
commit c0625c568f

View file

@ -107,8 +107,20 @@ rules / skills / hooks: age from git history, plus a compensatory-phrasing signa
whole file first", "don't guess"). Output: ranked deletion candidates with age + token whole file first", "don't guess"). Output: ranked deletion candidates with age + token
cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner
architecture, and composes with `tokens`/`manifest` for the payoff figure. architecture, and composes with `tokens`/`manifest` for the payoff figure.
*Caveat:* `~/.claude` is not git-tracked, so age there falls back to `mtime`, which is
weaker evidence. Project-level CLAUDE.md/rules get real git ages. > **🔴 The age half of this shape is DEAD — measured 2026-07-31 (økt #40), see §8's
> updated checklist.** Per-line `git blame` over four real instruction files gives
> single-date shares of 88 % / 55 % / 100 % / 58 %: instruction blocks trace back to bulk
> commits, so per-block age carries almost no information. Worse, `blame` reports
> *last touch*, not vintage — a reformatting commit (e.g. this repo's own `96e32df`,
> "trim project CLAUDE.md to invariants") makes old instructions look young, which biases
> the signal rather than merely thinning it. That also kills the `mtime` fallback.
> **§8 pre-registered this exact outcome and its consequence: shape A's primary signal
> collapses to the compensatory-phrasing heuristic alone, and §7.2 is answered — the
> classification needs the agent layer.** Do not resurrect age as a "weak secondary
> signal"; that is the drift the pre-registration existed to prevent. What survives of
> shape A is the *deterministic phrasing + token-cost pre-filter*, feeding a
> precision-gated judge (the `optimize` architecture).
**B. Protocol command (`/config-audit rebuild`).** The delete → live with it → earned **B. Protocol command (`/config-audit rebuild`).** The delete → live with it → earned
re-add ledger loop, spanning sessions: archive current config, record what was archived, re-add ledger loop, spanning sessions: archive current config, record what was archived,
@ -124,6 +136,16 @@ A or B rather than its own command.
Likely landing: **A first** (deterministic, testable, immediately useful), with B as the Likely landing: **A first** (deterministic, testable, immediately useful), with B as the
workflow that consumes A's output. C stays a documented idea until A+B exist. workflow that consumes A's output. C stays a documented idea until A+B exist.
> **DECIDED 2026-07-31 (#40), superseding the sketch above — see §7 q2/q3.** Shape A does
> not ship as a standalone `CA-VIN-*` scanner: its age signal is dead, and what remains of
> it (a phrasing + token-cost pre-filter) is precisely the front half of `optimize`'s
> existing hybrid motor. **The landing is `/config-audit optimize --subtract`** — a fourth
> `lensCheck` class emitting `CA-OPT-*` findings, with a deterministic **floor-exclusion**
> step ahead of the judge so a load-bearing block is never a candidate. Shape B (the
> earned-re-add ledger) stays out of v1 precisely because it is what would demand
> cross-session state and therefore a command of its own. Shape C unchanged: documented,
> not built.
## 6. Hard constraints (carry these into any implementation) ## 6. Hard constraints (carry these into any implementation)
### 6.0 The floor: compensatory vs load-bearing instructions ### 6.0 The floor: compensatory vs load-bearing instructions
@ -164,8 +186,18 @@ bash script, or a lost afternoon.
### 6.1 Existing repo constraints ### 6.1 Existing repo constraints
- **`~/.claude` is NOT git-tracked.** A literal `rm` there is unrecoverable. Archive by - **`~/.claude` IS git-tracked as of 2026-07-26 — but archive by `mv` into `_archive/`
`mv` into `_archive/`, never delete. This is already an inviolable rule in STATE.md. anyway, never `rm`.** Premise corrected 2026-07-31 (økt #40); the rule it was used to
justify is unchanged. Ground truth: `/Users/ktg/.claude` is a git repo, 7 commits,
initial commit `13cd708` 2026-07-26, remote `backup
/Volumes/DharmaBackup/claude-config.git` (external volume — not Forgejo, not GitHub),
47 files tracked (`hooks/` 19, `commands/` 14, `scripts/` 9, `CLAUDE.md`,
`settings.json`, `learnings/`, `docs/`). The rule survives on different grounds: only
those 47 files are recoverable, the history is days old, and the remote lives on a
volume that may not be mounted. Two consequences for this feature — (a) `~/.claude`
age evidence is bounded by the repo's own birth date and is therefore worthless, and
(b) any untracked file there (`memory/`, `coord/`, session state) is still
unrecoverable after `rm`.
- **`settings.json` is pathguard-protected** — no Edit/Write. Use `jq` + temp file + - **`settings.json` is pathguard-protected** — no Edit/Write. Use `jq` + temp file +
atomic `mv`, with operator OK ([[settings-json-pathguard-write]]). atomic `mv`, with operator OK ([[settings-json-pathguard-write]]).
- **New scanner ⇒ the 7-step byte-stability checklist** ([[adding-scanner-byte-stability]]): - **New scanner ⇒ the 7-step byte-stability checklist** ([[adding-scanner-byte-stability]]):
@ -178,8 +210,28 @@ bash script, or a lost afternoon.
## 7. Open questions for Thursday ## 7. Open questions for Thursday
> **STATUS 2026-07-31 (#40): q1, q2, q3 are CLOSED below — decided on measured evidence
> (dead age signal) plus the hand-built fasit (`docs/subtraction-fasit.local.md`), under
> the operator's standing delegation of technical/design calls. q4 was already closed
> 2026-07-29. Do not re-litigate without new evidence.**
>
> **The landing: `/config-audit optimize --subtract` — a fourth lens class in the existing
> hybrid motor, not a new command and not a new scanner.** Full rationale in q3.
1. Scope of the deletion candidate set: user-level `~/.claude` only, project only, or both? 1. Scope of the deletion candidate set: user-level `~/.claude` only, project only, or both?
(Age evidence is much stronger for project-level, per §5A.) (Age evidence is much stronger for project-level, per §5A.)
**CLOSED — both, and user-level is mandatory in v1.** The argument that pointed at
project-level was git-age evidence, and that evidence no longer exists (§5A box), so
scope now follows *payoff and testability* instead. Two things force user-level in:
§8's blocking floor test is defined over the operator's global CLAUDE.md, and that file
is where the always-loaded cost actually sits (~4 300 tok every turn, every repo,
every session — vs a project CLAUDE.md that loads only in its own repo). Project-level
comes along because the same motor reads it and because two of §8's four floor anchors
turned out to live there.
*Implementation note, not a reopening:* `optimize` today takes a repo `target`. Reading
`~/.claude/CLAUDE.md` is a target-resolution change, and per §6.1 the `~/.claude` write
rules apply — but this mode proposes, it never writes.
2. **The core question, given §6.0:** can compensatory-vs-load-bearing be classified 2. **The core question, given §6.0:** can compensatory-vs-load-bearing be classified
deterministically with acceptable precision, or does it need the agent layer (like deterministically with acceptable precision, or does it need the agent layer (like
`optimize`'s precision-gated `optimization-lens-agent`, which stays silent when `optimize`'s precision-gated `optimization-lens-agent`, which stays silent when
@ -195,8 +247,69 @@ bash script, or a lost afternoon.
(a table of local policy that reads like advice). Include 35 such blocks (a table of local policy that reads like advice). Include 35 such blocks
deliberately. A fasit built only from obvious cases will pass a tool that fails on deliberately. A fasit built only from obvious cases will pass a tool that fails on
real config. real config.
**CLOSED — it cannot be done deterministically. Deterministic pre-filter → precision-
gated judge, which is the `optimize` architecture verbatim.** Two independent lines of
evidence, both gathered before any code:
- *The age signal is gone* (§5A box). It was the only deterministic input that was
going to do real discriminating work; phrasing heuristics alone are what remain.
- *The fasit shows phrasing alone is not enough.* The blocks that decide the gate all
require reading **content against container** — a load-bearing fact wearing generic
phrasing, or vice versa:
| Fasit block | The trap |
|---|---|
| B-27 (defensive shell-scripting) | Reads as universal advice ("quote your variables"), but each line records a *specific local incident*, and "keep test code ASCII-clean" is inseparable from bash 3.2 (B-26, a §8 floor anchor). **A phrasing regex deletes this and fails the gate.** |
| B-32b ("never work in another repo") | Sits inside a bullet list of compensatory anti-patterns, formatted identically to its neighbours, but encodes the polyrepo structure and names `coord-send`. Container says delete, content says floor. |
| B-05 vs B-06 | Adjacent preference bullets, near-identical form, opposite calls. |
| B-30a / B-30b / B-30c | Three consecutive bullets under one heading, three different calls (Conventional Commits is floor, "lesbarhet > cleverness" is not). |
No regex separates these; a judge reading them in context can.
- *Two failure modes the judge must be prompted against, both found in the fasit:*
**staleness is not a deletion signal** (B-29 pins Opus 4.8 while the session runs
Opus 5 — that is a `drift`/dead-reference finding about a **floor** block), and
**tier-2 is not tier-3** (B-22, premiss-verifisering, is compensatory by class yet
earned itself again during this very session).
**Design consequence — the floor is NOT the judge's call.** Load-bearing exclusion runs
as a deterministic pre-filter step *before* the judge sees anything, so a floor block is
never a candidate at all. §8's gate is blocking and precision is asymmetric; making it
depend on a probabilistic judge would be the wrong guarantee. The judge then decides
only compensatory-tier questions on the remainder, and stays silent when unsure exactly
as `optimization-lens-agent` already does.
3. Does this ship as its own command, or as a `--subtract` mode of `optimize`? 3. Does this ship as its own command, or as a `--subtract` mode of `optimize`?
Both are defensible; command count is already 21. Both are defensible; command count is already 21.
**CLOSED — a `--subtract` mode of `optimize`.** The discriminator is whether v1 needs
cross-session state: it does not. v1 ranks deletion candidates and reports the payoff;
the earned-re-add **ledger** (shape B) is what would require `sessions/` state, and it
is deliberately not in v1. Without the ledger there is nothing a separate command buys.
What the mode inherits by not being new (verified against the code, 2026-07-31):
| Requirement | `optimize` already has it |
|---|---|
| Deterministic pre-filter → opus judge | `optimization-lens-scanner.mjs` (`lens-prefilter`) → `optimization-lens-agent` |
| Precision gate, silent when unsure | Stated in the agent's own prompt |
| "Not a mistake" framing | Every `optimize` finding is a *Missed opportunity* — exactly right for a line that works but no longer earns its rent |
| Register-backed provenance | `knowledge/best-practices.json`, CONFIRMED-only |
| Non-byte-stable by design | Already documented as such in CLAUDE.md |
| Finding IDs | `CA-OPT-*`**no new `CA-VIN-*` scanner** |
And what it avoids: the 7-step byte-stability checklist (§6.1), a scanners badge bump
16 → 17, snapshot risk against frozen `tests/snapshots/v5.0.0/`, and a 22nd command.
**Proportionality is the clinching argument.** The fasit puts the honest ceiling at
~8501 400 always-loaded tokens on a ~4 300-token file — real (~20 %), but nowhere near
the source anecdote's 80 %, and 26 of 34 blocks are floor. On a well-maintained config
the subtraction axis is mostly a no-op. That payoff justifies a mode on an existing
motor; it does not justify a new scanner plus a new command. ("Starte ambisiøse tiltak
når en konfig-justering holder" is a named anti-pattern.)
Registry shape: a 4th `lensCheck` class alongside the three in the agent's table, with
its own `BP-SUB-001` register rule. `--subtract` is the flag because the subtraction
axis should not fire on a plain `/config-audit optimize` run — it asks a different
question and needs the operator to have opted into it.
4. ~~Ordering against the existing queue.~~ **Decided 2026-07-29:** the operator 4. ~~Ordering against the existing queue.~~ **Decided 2026-07-29:** the operator
prioritized this work ahead of pipeline step 4 (`rollback`), to start Thursday prioritized this work ahead of pipeline step 4 (`rollback`), to start Thursday
2026-07-30. Do not re-litigate. The open part is only what follows it — the 2026-07-30. Do not re-litigate. The open part is only what follows it — the
@ -207,27 +320,55 @@ bash script, or a lost afternoon.
**Before building anything:** **Before building anything:**
- [ ] `grep -rniE 'blame|mtime|birthtime' scanners/` returns no instruction-age handling - [x] **DONE 2026-07-31 (#40).** `grep -rniE 'blame|mtime|birthtime' scanners/` returns
→ confirms §3.1 still true at build time. zero hits → §3.1 confirmed still true at build time.
- [ ] `node scanners/drift-cli.mjs <path> --save --name pre-subtraction` succeeds and - [ ] `node scanners/drift-cli.mjs <path> --save --name pre-subtraction` succeeds and
`lib/baseline.mjs` round-trips → confirms §4 reuse is real, not assumed. `lib/baseline.mjs` round-trips → confirms §4 reuse is real, not assumed.
**Still open** — not blocking the fasit, blocking any code.
- [ ] If any claim from §1 is to be written into the register: a primary source exists - [ ] If any claim from §1 is to be written into the register: a primary source exists
(URL to the talk or an Anthropic doc). Otherwise the entry is not written. (URL to the talk or an Anthropic doc). Otherwise the entry is not written.
- [ ] **The age signal actually exists.** §5A's fallback claim — "project-level - [x] **DONE 2026-07-31 (#40) — THE AGE SIGNAL DOES NOT EXIST. Collapse condition met.**
CLAUDE.md/rules get real git ages" — is untested. Run Per-line `git blame` (not `git log`; per-line is the question) over four real
`git log --format=%ad --follow -- CLAUDE.md .claude/rules/` across two or three instruction files:
real repos. If most instruction blocks trace back to a single bulk commit, per-block
age carries no information, shape A's primary signal collapses to the
compensatory-phrasing heuristic alone, and §7.2 (agent layer) is answered for us.
Check this *before* writing any scanner code.
**If shape A is built:** | File | Lines | Largest single-date share |
|---|---|---|
| `~/.claude/CLAUDE.md` | 250 | **88 %** (2026-07-26 = repo init; max age 5 days) |
| `config-audit/CLAUDE.md` | 102 | **55 %** (2026-04-08) |
| `config-audit/.claude/rules/ux-rules.md` | 32 | **100 %** (one commit) |
| `llm-security/CLAUDE.md` | 110 | **58 %** (2026-04-08) |
Blocks trace back to bulk commits exactly as the pre-registration feared, and
`blame` measures last-touch rather than vintage (a reformat resets it), so the
signal is *biased*, not merely sparse. **Consequence, as pre-registered: shape A's
primary signal is gone and §7.2 is answered — deterministic phrasing/cost
pre-filter → precision-gated judge.** See the boxed note in §5A.
- [x] **DONE 2026-07-31 (#40): the §8 floor-test fasit is built, before any classifier
exists.** `docs/subtraction-fasit.local.md` (LOCAL-ONLY — it quotes the operator's
global CLAUDE.md verbatim and this repo's only remote is the public `open/` mirror).
34 blocks over `~/.claude/CLAUDE.md`, each labelled `FLOOR` / `POLICY-FLOOR` /
`DELETABLE`, with 13 marked ⚠ AMBIGUOUS per §7.2. Headline numbers: 26 of 34 blocks
are floor; the deletable set is ~1 400 always-loaded tokens, realistically ~850
after tier-2 earn-backs, against a ~4 300-token file (**~20 %, not 80 %**).
**Correction it forces:** two of §8's four named floor anchors are not in that file
at all — the test command is project-scope (`config-audit/CLAUDE.md`), and
"`~/.claude` is not git-tracked" is now false (§6.1). The floor gate must run over
a *set* of files, not one.
**When `optimize --subtract` is built** (was: "if shape A is built" — retitled 2026-07-31
per §7 q3; the two struck items below were premised on a new standalone scanner, which is
no longer the shape):
- [ ] Red tests exist and fail before implementation. - [ ] Red tests exist and fail before implementation.
- [ ] `node --test 'tests/**/*.test.mjs'` green, count strictly greater than 1365. - [ ] `node --test 'tests/**/*.test.mjs'` green, count strictly greater than 1365.
- [ ] `git diff --stat tests/snapshots/v5.0.0/` is empty (frozen baselines untouched). - [ ] `git diff --stat tests/snapshots/v5.0.0/` is empty (frozen baselines untouched).
- [ ] `node scanners/self-audit.mjs --check-readme` passes (badge counts updated: **Still applies, and matters more as a mode:** the new lens class must not alter any
scanners 16 → 17). existing `optimize` output, so a plain `/config-audit optimize` run has to be diffed
against its current output as well.
- [ ] ~~`node scanners/self-audit.mjs --check-readme` passes (badge counts updated:
scanners 16 → 17).~~ **No badge bump — no new scanner.** `--check-readme` must still
pass, and README/CLAUDE.md still need the docs-gate diffs for a `feat:` commit
([[docs-gate-feat-requires-readme-claudemd]]).
- [ ] Every new finding renders with a non-`Other` `userImpactCategory` and a - [ ] Every new finding renders with a non-`Other` `userImpactCategory` and a
non-`_default` action language → humanizer wiring correct (M-16/M-17). non-`_default` action language → humanizer wiring correct (M-16/M-17).
- [ ] **Floor test (§6.0), the blocking one.** Build a fasit by hand first — per - [ ] **Floor test (§6.0), the blocking one.** Build a fasit by hand first — per