config-audit/docs/delete-and-rebuild-brief.md
Kjell Tore Guttormsen 31073c2178 docs(plan): fasit must cover the ambiguous middle, not the poles (§7.2)
§8's floor gate rests on §6.0's classification test, which is a judgment call
rather than a mechanical one. The four named must-survive items ("only Forgejo",
"bash is 3.2", the test command, "~/.claude is not git-tracked") are clear-cut —
any mechanism gets them right, so a fasit built from them proves nothing.

The gate is actually decided by blocks like "Conventional Commits:
type(scope): beskrivelse" (local convention or a nag the model follows anyway?),
"commit ofte med beskrivende meldinger", or the model-routing rubric — local
policy that reads like generic advice. §7.2 now requires 3-5 such blocks in the
fasit deliberately.

Cross-references verified: every section pointer in the brief (§1-§8, §6.0, §5A,
§3.1, §7.2) resolves to an existing heading. Shipping a dead prose reference in
the brief that proposes detecting them would have been an odd artifact — that is
CA-CML's finding class, v5.13 chunk 3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
2026-07-29 09:57:07 +02:00

242 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Brief — Delete-and-Rebuild (config subtraction)
**Status:** BRIEF, not a plan. No code, no chunk breakdown, no version number committed.
Written 2026-07-29 so a session starting Thursday evening has a durable starting point.
STATE.md is gitignored in this repo, so this file — not STATE — is the record.
---
## 1. Trigger and its provenance
The operator relayed a third-party YouTube summary (Hyper Automation Labs) of a talk
Boris Cherny reportedly gave at Y Combinator Startup School, one day after Opus 5
shipped. Claims attributed to him in that summary:
- Anthropic deleted ~80 % of Claude Code's own system prompt when Opus 5 landed.
- Advice to users: every six months, delete your CLAUDE.md, your skills, your hooks —
see what the model does.
- Rebuild method: delete everything, use it, add one line back only when the model
stumbles on the same thing repeatedly.
- The model measured *slightly more intelligent* with the built-in prompts stripped.
**Two levels of confidence here, and they must not be collapsed** (updated 2026-07-29
after the operator corrected the first draft):
- **Attribution — confirmed.** The operator watched the recording and identifies Boris
Cherny on stage. This is not a channel's claim about who spoke; it is direct
observation by the operator. The talk happened and it is him.
- **The verbatim figures — still summary-level.** "80 % of the system prompt", "the
model measured slightly more intelligent without the prompts", the Bun numbers: these
reach us through the channel's editing, not through a primary transcript. They are
plausible and consistent with §2, and they are not quoted as fact anywhere below.
So if this becomes a `BP-*` entry in the best-practices register, the source field reads
*"Boris Cherny, YC Startup School talk (attribution confirmed by operator); figures via
third-party summary, not primary-verified"* — not "unverified", and not a bare citation
either. Getting a primary transcript for the figures is a nice-to-have, never a
prerequisite: the feature is argued from this repo's own logic (§3), and this repo has
one scar from treating a plausible quote as load-bearing fact («Fable low ≈ Opus high»,
fabricated, rejected 2026-07-14).
**What the operator has affirmed as in scope:** delete CLAUDE.md, then rebuild by adding
back what the model needs help with — *starting with what it must have*. That last clause
is not a detail; it is the design constraint in §6.0.
## 2. Verified ground truth (checked against the local CLI, 2026-07-29)
These *are* facts, and they are what make an empirical variant buildable:
| Fact | How verified |
|---|---|
| `claude --bare` exists — "Minimal mode: skip hooks, LSP, plugin sync, attribution, auto-memory, background prefetches, keychain reads, and CLAUDE.md auto-discovery. Sets `CLAUDE_CODE_SIMPLE=1`." | `claude --help` |
| `claude --system-prompt <prompt>` — replaces the session system prompt | `claude --help` |
| `--append-system-prompt`, `--add-dir`, `--setting-sources <user,project,local>`, `--plugin-dir`, `--settings`, `--agents` — all present and all usable to compose an isolated config for a controlled run | `claude --help` |
So the `CLAUDE_CODE_SIMPLE=1` env var the video calls "undocumented" is, in this CLI
version, a documented flag (`--bare`). That matters: an A/B ablation harness would not
need an undocumented hook.
## 3. Why this fits this repo (the argument that does *not* depend on §1)
The plugin has three pillars — Health, Opportunities, Action. Every existing command
answers a question on the **addition** axis:
- `feature-gap` — what could you add?
- `optimize` — what would fit a better mechanism?
- `posture` / `tokens` / `manifest` — how good/expensive is what you have?
- `fix` / `implement` — apply changes.
**Nothing answers the subtraction question: what is no longer earning its rent?**
`feature-gap` has no inverse. That is a real hole, independent of who said what on
a stage.
Two further reasons this belongs *here* specifically:
1. **Nothing in the repo measures instruction age.** Verified by grep over `scanners/`:
`stale` appears only for knowledge-register entry age (`lib/knowledge-refresh.mjs`)
and for stale plugin-cache versions (`scan-orchestrator.mjs`, M-BUG-11). No scanner
touches `git blame`, `mtime`, or the vintage of a CLAUDE.md block. An instruction
written for Sonnet 3.5 and an instruction written last week are indistinguishable to
every current scanner.
2. **Deleting bravely is only sane if you can undo it.** That is already pillar three:
`lib/backup.mjs`, `rollback-engine.mjs`, `auto-backup-config.mjs` (PreToolUse), and
`drift`'s `saveBaseline` / `loadBaseline` / `diffEnvelopes`. The safety net exists;
the feature that would use it does not. This is arguably the strongest framing:
*config-audit is already the infrastructure that makes "delete it and see" a
measurement rather than a gamble.*
## 4. Reusable machinery (do not rebuild these)
| Need | Already exists |
|---|---|
| Snapshot config before deleting | `scanners/lib/baseline.mjs` (`saveBaseline`), used by `drift` |
| Diff before/after | `diffEnvelopes` in the same module |
| Backup + restore individual files w/ sha256 manifest | `scanners/lib/backup.mjs`, `scanners/rollback-engine.mjs` |
| Cost of each source, always-loaded subtotal | `scanners/manifest.mjs`, `token-hotspots.mjs` |
| Human-approved-writes command pattern | `knowledge-refresh`, `campaign` (both non-byte-stable by design) |
## 5. Candidate shapes (sketches — pick on Thursday, do not pre-commit)
All three inherit the floor constraint in §6.0: whatever the shape, load-bearing local
facts are never deletion candidates, and a rebuild restores them first. A shape that
cannot express that distinction is disqualified regardless of how cheap it is.
**A. Deterministic vintage scanner (`CA-VIN-*`).** Per instruction block in CLAUDE.md /
rules / skills / hooks: age from git history, plus a compensatory-phrasing signal
(blocks that exist to correct model behaviour — "ALWAYS", "never forget", "read the
whole file first", "don't guess"). Output: ranked deletion candidates with age + token
cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner
architecture, and composes with `tokens`/`manifest` for the payoff figure.
*Caveat:* `~/.claude` is not git-tracked, so age there falls back to `mtime`, which is
weaker evidence. Project-level CLAUDE.md/rules get real git ages.
**B. Protocol command (`/config-audit rebuild`).** The delete → live with it → earned
re-add ledger loop, spanning sessions: archive current config, record what was archived,
and maintain a ledger where a line only returns when the operator records that the model
actually stumbled on it. Highest fidelity to the source idea; needs cross-session state
(the `sessions/` machinery already exists) and is inherently not byte-stable.
**C. Measured ablation harness.** Use `--bare` + `--system-prompt`/`--add-dir` to run the
same prompt with and without a block and compare. Closest to a real verifier, most
expensive in tokens, weakest determinism. Probably a later `--experimental` sub-mode of
A or B rather than its own command.
Likely landing: **A first** (deterministic, testable, immediately useful), with B as the
workflow that consumes A's output. C stays a documented idea until A+B exist.
## 6. Hard constraints (carry these into any implementation)
### 6.0 The floor: compensatory vs load-bearing instructions
**This is the invariant all three shapes in §5 must respect, and it is what makes the
feature safe to ship at all.** Not every line in a CLAUDE.md is the same kind of thing:
| Class | What it is | Test | Disposition |
|---|---|---|---|
| **Compensatory** | An instruction correcting model *behaviour* — "read the whole file first", "don't guess", "ALWAYS verify", "think before you code" | A smarter model would do this unprompted | **Deletion candidate.** Returns only when earned. |
| **Load-bearing** | A *local fact* the model cannot derive at any capability level — "never GitHub, only Forgejo at git.fromaitochitta.com", "system bash is 3.2, no `declare -A`", "test with `node --test 'tests/**/*.test.mjs'`", "`~/.claude` is not git-tracked" | No amount of intelligence produces this from the codebase alone | **Floor. Never a deletion candidate.** |
Model capability erodes the first class and does nothing to the second. That is the whole
mechanism behind the source idea — and it means "delete your CLAUDE.md" is only correct
for one of the two classes. A tool that treats them alike would delete the operator's
Forgejo constraint because Opus 5 "is smart enough now", which is a category error: the
model isn't failing at intelligence there, it simply cannot know.
**Consequence for the rebuild ordering.** A rebuild is not one undifferentiated
add-back-on-stumble loop. It is three tiers:
1. **Floor — goes back immediately, no trial period.** Load-bearing facts. The config is
never in a state where these are absent; a "delete everything" that drops them is a
broken experiment, not a brave one.
2. **Earned — out, returns only on repeated observed stumbling.** Compensatory
instructions that turn out to still be needed by *this* model.
3. **Dead — out and never missed.** The payoff, measurable in tokens via `manifest`.
A third class sits deliberately outside the axis: **policy prohibitions** ("never commit
secrets"). These may well be things the model would honour unprompted, but their cost of
being wrong is asymmetric and they are cheap. They stay in the floor by decision, not by
classification. Do not let a "the model knows this now" argument reach them.
The hard part is tier 1 vs tier 2, and that classification — not the deletion mechanics —
is the real engineering problem in this brief. Precision is asymmetric: a missed dead line
costs a few tokens per turn; a deleted load-bearing line costs a wrong remote, a broken
bash script, or a lost afternoon.
### 6.1 Existing repo constraints
- **`~/.claude` is NOT git-tracked.** A literal `rm` there is unrecoverable. Archive by
`mv` into `_archive/`, never delete. This is already an inviolable rule in STATE.md.
- **`settings.json` is pathguard-protected** — no Edit/Write. Use `jq` + temp file +
atomic `mv`, with operator OK ([[settings-json-pathguard-write]]).
- **New scanner ⇒ the 7-step byte-stability checklist** ([[adding-scanner-byte-stability]]):
scanner + orchestrator + scoring area-map + strip-added-scanner + humanizer count +
SC-5 + humanizer wiring (`SCANNER_TO_CATEGORY` entry and `TRANSLATIONS.static` per RAW
title — M-16/M-17). Frozen `tests/snapshots/v5.0.0/` must stay untouched.
- **TDD is inviolable** — red tests before implementation, including for .md contracts.
- **`feat:` commits require non-trivial diffs in both README.md and CLAUDE.md**
([[docs-gate-feat-requires-readme-claudemd]]).
## 7. Open questions for Thursday
1. Scope of the deletion candidate set: user-level `~/.claude` only, project only, or both?
(Age evidence is much stronger for project-level, per §5A.)
2. **The core question, given §6.0:** can compensatory-vs-load-bearing be classified
deterministically with acceptable precision, or does it need the agent layer (like
`optimize`'s precision-gated `optimization-lens-agent`, which stays silent when
unsure)? Note the two signals are independent — a load-bearing fact can be old, and a
compensatory instruction can be new — so age alone can never carry this call. Prior:
a deterministic pre-filter feeding a precision-gated judge, which is exactly the
`optimize` architecture already in the repo.
**Design the §8 fasit around the ambiguous middle, not the poles.** The four named
must-survive items are clear-cut and any mechanism will get them right; the gate is
really decided by blocks like "Conventional Commits: `type(scope): beskrivelse`"
(local convention, or a nag the model would follow anyway?), "commit ofte med
beskrivende meldinger" (pure behaviour correction?), or the model-routing rubric
(a table of local policy that reads like advice). Include 35 such blocks
deliberately. A fasit built only from obvious cases will pass a tool that fails on
real config.
3. Does this ship as its own command, or as a `--subtract` mode of `optimize`?
Both are defensible; command count is already 21.
4. ~~Ordering against the existing queue.~~ **Decided 2026-07-29:** the operator
prioritized this work ahead of pipeline step 4 (`rollback`), to start Thursday
2026-07-30. Do not re-litigate. The open part is only what follows it — the
prior order stands underneath: step 4 `rollback`, then the M-11→M-20 batch
release, then the v5.13 plan.
## 8. Verifisering (testable criteria)
**Before building anything:**
- [ ] `grep -rniE 'blame|mtime|birthtime' scanners/` returns no instruction-age handling
→ confirms §3.1 still true at build time.
- [ ] `node scanners/drift-cli.mjs <path> --save --name pre-subtraction` succeeds and
`lib/baseline.mjs` round-trips → confirms §4 reuse is real, not assumed.
- [ ] If any claim from §1 is to be written into the register: a primary source exists
(URL to the talk or an Anthropic doc). Otherwise the entry is not written.
- [ ] **The age signal actually exists.** §5A's fallback claim — "project-level
CLAUDE.md/rules get real git ages" — is untested. Run
`git log --format=%ad --follow -- CLAUDE.md .claude/rules/` across two or three
real repos. If most instruction blocks trace back to a single bulk commit, per-block
age carries no information, shape A's primary signal collapses to the
compensatory-phrasing heuristic alone, and §7.2 (agent layer) is answered for us.
Check this *before* writing any scanner code.
**If shape A is built:**
- [ ] Red tests exist and fail before implementation.
- [ ] `node --test 'tests/**/*.test.mjs'` green, count strictly greater than 1365.
- [ ] `git diff --stat tests/snapshots/v5.0.0/` is empty (frozen baselines untouched).
- [ ] `node scanners/self-audit.mjs --check-readme` passes (badge counts updated:
scanners 16 → 17).
- [ ] Every new finding renders with a non-`Other` `userImpactCategory` and a
non-`_default` action language → humanizer wiring correct (M-16/M-17).
- [ ] **Floor test (§6.0), the blocking one.** Build a fasit by hand first — per
[[judge-the-judge-build-fasit-first]] — over the operator's own global CLAUDE.md,
labelling each block compensatory or load-bearing. Then run the tool. Required
result: **zero load-bearing blocks proposed for deletion.** Non-negotiable; the
known-correct floor items to check for explicitly are "never GitHub, only Forgejo",
"system bash is 3.2", the test command, and "`~/.claude` is not git-tracked". One
of those in the delete list fails the gate outright, whatever the recall.
- [ ] Recall, secondary: at least one candidate the operator agrees is genuinely dead
weight. Precision over recall throughout — a missed dead line costs a few tokens
per turn, a deleted load-bearing line costs a wrong remote or a broken script.