open/config-audit
1
0
Fork 0
Code Issues Pull requests Releases Activity

docs(plan): age signal measured dead; close §7 q1-q3 on optimize --subtract

Browse source 
Ran the brief's own §8 pre-build checks before writing any code, and two of
them changed the design.

Age signal (§5A, §8): per-line git blame over four real instruction files gives
single-date shares of 88% / 55% / 100% / 58%. Instruction blocks trace back to
bulk commits, and blame reports last-touch rather than vintage — a reformatting
commit (this repo's own 96e32df) makes old instructions look young, so the
signal is biased, not merely sparse. That also kills the mtime fallback. §8
pre-registered this exact outcome and its consequence, so shape A does not ship
as a CA-VIN-* vintage scanner.

Premise correction (§6.1): ~/.claude IS git-tracked as of 2026-07-26 (7 commits,
remote on an external backup volume, 47 files). The rule it justified — mv to
_archive/, never rm — stands on different grounds and is unchanged.

§7 q1: both scopes, user-level mandatory in v1 (the floor test is defined there
and the always-loaded cost sits there).
§7 q2: not deterministically classifiable — the deciding blocks require reading
content against container. Deterministic pre-filter -> precision-gated judge,
with floor-exclusion running BEFORE the judge so a load-bearing block is never a
candidate.
§7 q3: ships as /config-audit optimize --subtract, a fourth lensCheck class
emitting CA-OPT-* with a BP-SUB-001 register rule. No new scanner, no new
command, no badge bump, no snapshot risk. Proportionality decided it: the
hand-built fasit puts the honest payoff at ~850-1400 always-loaded tokens on a
~4300-token file (~20%, not 80%), with 26 of 34 blocks classified floor.

The fasit itself (34 blocks, 13 marked ambiguous per §7.2) is local-only — it
quotes the operator's global CLAUDE.md verbatim and this repo's only remote is
the public open/ mirror.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011x2i9NDEtMR9FU7ufaCnBX
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 15:43:03 +02:00
commit c0625c568f
1 changed files with 158 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

173
docs/delete-and-rebuild-brief.md
View file

@ -107,8 +107,20 @@ rules / skills / hooks: age from git history, plus a compensatory-phrasing signa
whole file first", "don't guess"). Output: ranked deletion candidates with age + token
cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner
architecture, and composes with `tokens`/`manifest` for the payoff figure.
*Caveat:* `~/.claude` is not git-tracked, so age there falls back to `mtime`, which is
weaker evidence. Project-level CLAUDE.md/rules get real git ages.
> **🔴 The age half of this shape is DEAD — measured 2026-07-31 (økt #40), see §8's
> updated checklist.** Per-line `git blame` over four real instruction files gives
> single-date shares of 88 % / 55 % / 100 % / 58 %: instruction blocks trace back to bulk
> commits, so per-block age carries almost no information. Worse, `blame` reports
> *last touch*, not vintage — a reformatting commit (e.g. this repo's own `96e32df`,
> "trim project CLAUDE.md to invariants") makes old instructions look young, which biases
> the signal rather than merely thinning it. That also kills the `mtime` fallback.
> **§8 pre-registered this exact outcome and its consequence: shape A's primary signal
> collapses to the compensatory-phrasing heuristic alone, and §7.2 is answered — the
> classification needs the agent layer.** Do not resurrect age as a "weak secondary
> signal"; that is the drift the pre-registration existed to prevent. What survives of
> shape A is the *deterministic phrasing + token-cost pre-filter*, feeding a
> precision-gated judge (the `optimize` architecture).
**B. Protocol command (`/config-audit rebuild`).** The delete → live with it → earned
re-add ledger loop, spanning sessions: archive current config, record what was archived,
@ -124,6 +136,16 @@ A or B rather than its own command.
Likely landing: **A first** (deterministic, testable, immediately useful), with B as the
workflow that consumes A's output. C stays a documented idea until A+B exist.
> **DECIDED 2026-07-31 (#40), superseding the sketch above — see §7 q2/q3.** Shape A does
> not ship as a standalone `CA-VIN-*` scanner: its age signal is dead, and what remains of
> it (a phrasing + token-cost pre-filter) is precisely the front half of `optimize`'s
> existing hybrid motor. **The landing is `/config-audit optimize --subtract`** — a fourth
> `lensCheck` class emitting `CA-OPT-*` findings, with a deterministic **floor-exclusion**
> step ahead of the judge so a load-bearing block is never a candidate. Shape B (the
> earned-re-add ledger) stays out of v1 precisely because it is what would demand
> cross-session state and therefore a command of its own. Shape C unchanged: documented,
> not built.
## 6. Hard constraints (carry these into any implementation)
### 6.0 The floor: compensatory vs load-bearing instructions
@ -164,8 +186,18 @@ bash script, or a lost afternoon.
### 6.1 Existing repo constraints
- **`~/.claude` is NOT git-tracked.** A literal `rm` there is unrecoverable. Archive by
`mv` into `_archive/`, never delete. This is already an inviolable rule in STATE.md.
- **`~/.claude` IS git-tracked as of 2026-07-26 — but archive by `mv` into `_archive/`
anyway, never `rm`.** Premise corrected 2026-07-31 (økt #40); the rule it was used to
justify is unchanged. Ground truth: `/Users/ktg/.claude` is a git repo, 7 commits,
initial commit `13cd708` 2026-07-26, remote `backup
/Volumes/DharmaBackup/claude-config.git` (external volume — not Forgejo, not GitHub),
47 files tracked (`hooks/` 19, `commands/` 14, `scripts/` 9, `CLAUDE.md`,
`settings.json`, `learnings/`, `docs/`). The rule survives on different grounds: only
those 47 files are recoverable, the history is days old, and the remote lives on a
volume that may not be mounted. Two consequences for this feature — (a) `~/.claude`
age evidence is bounded by the repo's own birth date and is therefore worthless, and
(b) any untracked file there (`memory/`, `coord/`, session state) is still
unrecoverable after `rm`.
- **`settings.json` is pathguard-protected** — no Edit/Write. Use `jq` + temp file +
atomic `mv`, with operator OK ([[settings-json-pathguard-write]]).
- **New scanner ⇒ the 7-step byte-stability checklist** ([[adding-scanner-byte-stability]]):
@ -178,8 +210,28 @@ bash script, or a lost afternoon.
## 7. Open questions for Thursday
> **STATUS 2026-07-31 (#40): q1, q2, q3 are CLOSED below — decided on measured evidence
> (dead age signal) plus the hand-built fasit (`docs/subtraction-fasit.local.md`), under
> the operator's standing delegation of technical/design calls. q4 was already closed
> 2026-07-29. Do not re-litigate without new evidence.**
>
> **The landing: `/config-audit optimize --subtract` — a fourth lens class in the existing
> hybrid motor, not a new command and not a new scanner.** Full rationale in q3.
1. Scope of the deletion candidate set: user-level `~/.claude` only, project only, or both?
(Age evidence is much stronger for project-level, per §5A.)
**CLOSED — both, and user-level is mandatory in v1.** The argument that pointed at
project-level was git-age evidence, and that evidence no longer exists (§5A box), so
scope now follows *payoff and testability* instead. Two things force user-level in:
§8's blocking floor test is defined over the operator's global CLAUDE.md, and that file
is where the always-loaded cost actually sits (~4 300 tok every turn, every repo,
every session — vs a project CLAUDE.md that loads only in its own repo). Project-level
comes along because the same motor reads it and because two of §8's four floor anchors
turned out to live there.
*Implementation note, not a reopening:* `optimize` today takes a repo `target`. Reading
`~/.claude/CLAUDE.md` is a target-resolution change, and per §6.1 the `~/.claude` write
rules apply — but this mode proposes, it never writes.
2. **The core question, given §6.0:** can compensatory-vs-load-bearing be classified
deterministically with acceptable precision, or does it need the agent layer (like
`optimize`'s precision-gated `optimization-lens-agent`, which stays silent when
@ -195,8 +247,69 @@ bash script, or a lost afternoon.
(a table of local policy that reads like advice). Include 3–5 such blocks
deliberately. A fasit built only from obvious cases will pass a tool that fails on
real config.
**CLOSED — it cannot be done deterministically. Deterministic pre-filter → precision-
gated judge, which is the `optimize` architecture verbatim.** Two independent lines of
evidence, both gathered before any code:
- *The age signal is gone* (§5A box). It was the only deterministic input that was
going to do real discriminating work; phrasing heuristics alone are what remain.
- *The fasit shows phrasing alone is not enough.* The blocks that decide the gate all
require reading **content against container** — a load-bearing fact wearing generic
phrasing, or vice versa:
| Fasit block | The trap |
|---|---|
| B-27 (defensive shell-scripting) | Reads as universal advice ("quote your variables"), but each line records a *specific local incident*, and "keep test code ASCII-clean" is inseparable from bash 3.2 (B-26, a §8 floor anchor). **A phrasing regex deletes this and fails the gate.** |
| B-32b ("never work in another repo") | Sits inside a bullet list of compensatory anti-patterns, formatted identically to its neighbours, but encodes the polyrepo structure and names `coord-send`. Container says delete, content says floor. |
| B-05 vs B-06 | Adjacent preference bullets, near-identical form, opposite calls. |
| B-30a / B-30b / B-30c | Three consecutive bullets under one heading, three different calls (Conventional Commits is floor, "lesbarhet > cleverness" is not). |
No regex separates these; a judge reading them in context can.
- *Two failure modes the judge must be prompted against, both found in the fasit:*
**staleness is not a deletion signal** (B-29 pins Opus 4.8 while the session runs
Opus 5 — that is a `drift`/dead-reference finding about a **floor** block), and
**tier-2 is not tier-3** (B-22, premiss-verifisering, is compensatory by class yet
earned itself again during this very session).
**Design consequence — the floor is NOT the judge's call.** Load-bearing exclusion runs
as a deterministic pre-filter step *before* the judge sees anything, so a floor block is
never a candidate at all. §8's gate is blocking and precision is asymmetric; making it
depend on a probabilistic judge would be the wrong guarantee. The judge then decides
only compensatory-tier questions on the remainder, and stays silent when unsure exactly
as `optimization-lens-agent` already does.
3. Does this ship as its own command, or as a `--subtract` mode of `optimize`?
Both are defensible; command count is already 21.
**CLOSED — a `--subtract` mode of `optimize`.** The discriminator is whether v1 needs
cross-session state: it does not. v1 ranks deletion candidates and reports the payoff;
the earned-re-add **ledger** (shape B) is what would require `sessions/` state, and it
is deliberately not in v1. Without the ledger there is nothing a separate command buys.
What the mode inherits by not being new (verified against the code, 2026-07-31):
| Requirement | `optimize` already has it |
|---|---|
| Deterministic pre-filter → opus judge | `optimization-lens-scanner.mjs` (`lens-prefilter`) → `optimization-lens-agent` |
| Precision gate, silent when unsure | Stated in the agent's own prompt |
| "Not a mistake" framing | Every `optimize` finding is a *Missed opportunity* — exactly right for a line that works but no longer earns its rent |
| Register-backed provenance | `knowledge/best-practices.json`, CONFIRMED-only |
| Non-byte-stable by design | Already documented as such in CLAUDE.md |
| Finding IDs | `CA-OPT-*` — **no new `CA-VIN-*` scanner** |
And what it avoids: the 7-step byte-stability checklist (§6.1), a scanners badge bump
16 → 17, snapshot risk against frozen `tests/snapshots/v5.0.0/`, and a 22nd command.
**Proportionality is the clinching argument.** The fasit puts the honest ceiling at
~850–1 400 always-loaded tokens on a ~4 300-token file — real (~20 %), but nowhere near
the source anecdote's 80 %, and 26 of 34 blocks are floor. On a well-maintained config
the subtraction axis is mostly a no-op. That payoff justifies a mode on an existing
motor; it does not justify a new scanner plus a new command. ("Starte ambisiøse tiltak
når en konfig-justering holder" is a named anti-pattern.)
Registry shape: a 4th `lensCheck` class alongside the three in the agent's table, with
its own `BP-SUB-001` register rule. `--subtract` is the flag because the subtraction
axis should not fire on a plain `/config-audit optimize` run — it asks a different
question and needs the operator to have opted into it.
4. ~~Ordering against the existing queue.~~ **Decided 2026-07-29:** the operator
prioritized this work ahead of pipeline step 4 (`rollback`), to start Thursday
2026-07-30. Do not re-litigate. The open part is only what follows it — the
@ -207,27 +320,55 @@ bash script, or a lost afternoon.
**Before building anything:**
- [ ] `grep -rniE 'blame|mtime|birthtime' scanners/` returns no instruction-age handling
→ confirms §3.1 still true at build time.
- [x] **DONE 2026-07-31 (#40).** `grep -rniE 'blame|mtime|birthtime' scanners/` returns
zero hits → §3.1 confirmed still true at build time.
- [ ] `node scanners/drift-cli.mjs <path> --save --name pre-subtraction` succeeds and
`lib/baseline.mjs` round-trips → confirms §4 reuse is real, not assumed.
**Still open** — not blocking the fasit, blocking any code.
- [ ] If any claim from §1 is to be written into the register: a primary source exists
(URL to the talk or an Anthropic doc). Otherwise the entry is not written.
- [ ] **The age signal actually exists.** §5A's fallback claim — "project-level
CLAUDE.md/rules get real git ages" — is untested. Run
`git log --format=%ad --follow -- CLAUDE.md .claude/rules/` across two or three
real repos. If most instruction blocks trace back to a single bulk commit, per-block
age carries no information, shape A's primary signal collapses to the
compensatory-phrasing heuristic alone, and §7.2 (agent layer) is answered for us.
Check this *before* writing any scanner code.
- [x] **DONE 2026-07-31 (#40) — THE AGE SIGNAL DOES NOT EXIST. Collapse condition met.**
Per-line `git blame` (not `git log`; per-line is the question) over four real
instruction files:
**If shape A is built:**
| File | Lines | Largest single-date share |
|---|---|---|
| `~/.claude/CLAUDE.md` | 250 | **88 %** (2026-07-26 = repo init; max age 5 days) |
| `config-audit/CLAUDE.md` | 102 | **55 %** (2026-04-08) |
| `config-audit/.claude/rules/ux-rules.md` | 32 | **100 %** (one commit) |
| `llm-security/CLAUDE.md` | 110 | **58 %** (2026-04-08) |
Blocks trace back to bulk commits exactly as the pre-registration feared, and
`blame` measures last-touch rather than vintage (a reformat resets it), so the
signal is *biased*, not merely sparse. **Consequence, as pre-registered: shape A's
primary signal is gone and §7.2 is answered — deterministic phrasing/cost
pre-filter → precision-gated judge.** See the boxed note in §5A.
- [x] **DONE 2026-07-31 (#40): the §8 floor-test fasit is built, before any classifier
exists.** `docs/subtraction-fasit.local.md` (LOCAL-ONLY — it quotes the operator's
global CLAUDE.md verbatim and this repo's only remote is the public `open/` mirror).
34 blocks over `~/.claude/CLAUDE.md`, each labelled `FLOOR` / `POLICY-FLOOR` /
`DELETABLE`, with 13 marked ⚠ AMBIGUOUS per §7.2. Headline numbers: 26 of 34 blocks
are floor; the deletable set is ~1 400 always-loaded tokens, realistically ~850
after tier-2 earn-backs, against a ~4 300-token file (**~20 %, not 80 %**).
**Correction it forces:** two of §8's four named floor anchors are not in that file
at all — the test command is project-scope (`config-audit/CLAUDE.md`), and
"`~/.claude` is not git-tracked" is now false (§6.1). The floor gate must run over
a *set* of files, not one.
**When `optimize --subtract` is built** (was: "if shape A is built" — retitled 2026-07-31
per §7 q3; the two struck items below were premised on a new standalone scanner, which is
no longer the shape):
- [ ] Red tests exist and fail before implementation.
- [ ] `node --test 'tests/**/*.test.mjs'` green, count strictly greater than 1365.
- [ ] `git diff --stat tests/snapshots/v5.0.0/` is empty (frozen baselines untouched).
- [ ] `node scanners/self-audit.mjs --check-readme` passes (badge counts updated:
scanners 16 → 17).
**Still applies, and matters more as a mode:** the new lens class must not alter any
existing `optimize` output, so a plain `/config-audit optimize` run has to be diffed
against its current output as well.
- [ ] ~~`node scanners/self-audit.mjs --check-readme` passes (badge counts updated:
scanners 16 → 17).~~ **No badge bump — no new scanner.** `--check-readme` must still
pass, and README/CLAUDE.md still need the docs-gate diffs for a `feat:` commit
([[docs-gate-feat-requires-readme-claudemd]]).
- [ ] Every new finding renders with a non-`Other` `userImpactCategory` and a
non-`_default` action language → humanizer wiring correct (M-16/M-17).
- [ ] **Floor test (§6.0), the blocking one.** Build a fasit by hand first — per