config-audit/docs/delete-and-rebuild-brief.md
Kjell Tore Guttormsen 31073c2178 docs(plan): fasit must cover the ambiguous middle, not the poles (§7.2)
§8's floor gate rests on §6.0's classification test, which is a judgment call
rather than a mechanical one. The four named must-survive items ("only Forgejo",
"bash is 3.2", the test command, "~/.claude is not git-tracked") are clear-cut —
any mechanism gets them right, so a fasit built from them proves nothing.

The gate is actually decided by blocks like "Conventional Commits:
type(scope): beskrivelse" (local convention or a nag the model follows anyway?),
"commit ofte med beskrivende meldinger", or the model-routing rubric — local
policy that reads like generic advice. §7.2 now requires 3-5 such blocks in the
fasit deliberately.

Cross-references verified: every section pointer in the brief (§1-§8, §6.0, §5A,
§3.1, §7.2) resolves to an existing heading. Shipping a dead prose reference in
the brief that proposes detecting them would have been an odd artifact — that is
CA-CML's finding class, v5.13 chunk 3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
2026-07-29 09:57:07 +02:00

15 KiB
Raw Blame History

Brief — Delete-and-Rebuild (config subtraction)

Status: BRIEF, not a plan. No code, no chunk breakdown, no version number committed. Written 2026-07-29 so a session starting Thursday evening has a durable starting point. STATE.md is gitignored in this repo, so this file — not STATE — is the record.


1. Trigger and its provenance

The operator relayed a third-party YouTube summary (Hyper Automation Labs) of a talk Boris Cherny reportedly gave at Y Combinator Startup School, one day after Opus 5 shipped. Claims attributed to him in that summary:

  • Anthropic deleted ~80 % of Claude Code's own system prompt when Opus 5 landed.
  • Advice to users: every six months, delete your CLAUDE.md, your skills, your hooks — see what the model does.
  • Rebuild method: delete everything, use it, add one line back only when the model stumbles on the same thing repeatedly.
  • The model measured slightly more intelligent with the built-in prompts stripped.

Two levels of confidence here, and they must not be collapsed (updated 2026-07-29 after the operator corrected the first draft):

  • Attribution — confirmed. The operator watched the recording and identifies Boris Cherny on stage. This is not a channel's claim about who spoke; it is direct observation by the operator. The talk happened and it is him.
  • The verbatim figures — still summary-level. "80 % of the system prompt", "the model measured slightly more intelligent without the prompts", the Bun numbers: these reach us through the channel's editing, not through a primary transcript. They are plausible and consistent with §2, and they are not quoted as fact anywhere below.

So if this becomes a BP-* entry in the best-practices register, the source field reads "Boris Cherny, YC Startup School talk (attribution confirmed by operator); figures via third-party summary, not primary-verified" — not "unverified", and not a bare citation either. Getting a primary transcript for the figures is a nice-to-have, never a prerequisite: the feature is argued from this repo's own logic (§3), and this repo has one scar from treating a plausible quote as load-bearing fact («Fable low ≈ Opus high», fabricated, rejected 2026-07-14).

What the operator has affirmed as in scope: delete CLAUDE.md, then rebuild by adding back what the model needs help with — starting with what it must have. That last clause is not a detail; it is the design constraint in §6.0.

2. Verified ground truth (checked against the local CLI, 2026-07-29)

These are facts, and they are what make an empirical variant buildable:

Fact How verified
claude --bare exists — "Minimal mode: skip hooks, LSP, plugin sync, attribution, auto-memory, background prefetches, keychain reads, and CLAUDE.md auto-discovery. Sets CLAUDE_CODE_SIMPLE=1." claude --help
claude --system-prompt <prompt> — replaces the session system prompt claude --help
--append-system-prompt, --add-dir, --setting-sources <user,project,local>, --plugin-dir, --settings, --agents — all present and all usable to compose an isolated config for a controlled run claude --help

So the CLAUDE_CODE_SIMPLE=1 env var the video calls "undocumented" is, in this CLI version, a documented flag (--bare). That matters: an A/B ablation harness would not need an undocumented hook.

3. Why this fits this repo (the argument that does not depend on §1)

The plugin has three pillars — Health, Opportunities, Action. Every existing command answers a question on the addition axis:

  • feature-gap — what could you add?
  • optimize — what would fit a better mechanism?
  • posture / tokens / manifest — how good/expensive is what you have?
  • fix / implement — apply changes.

Nothing answers the subtraction question: what is no longer earning its rent? feature-gap has no inverse. That is a real hole, independent of who said what on a stage.

Two further reasons this belongs here specifically:

  1. Nothing in the repo measures instruction age. Verified by grep over scanners/: stale appears only for knowledge-register entry age (lib/knowledge-refresh.mjs) and for stale plugin-cache versions (scan-orchestrator.mjs, M-BUG-11). No scanner touches git blame, mtime, or the vintage of a CLAUDE.md block. An instruction written for Sonnet 3.5 and an instruction written last week are indistinguishable to every current scanner.
  2. Deleting bravely is only sane if you can undo it. That is already pillar three: lib/backup.mjs, rollback-engine.mjs, auto-backup-config.mjs (PreToolUse), and drift's saveBaseline / loadBaseline / diffEnvelopes. The safety net exists; the feature that would use it does not. This is arguably the strongest framing: config-audit is already the infrastructure that makes "delete it and see" a measurement rather than a gamble.

4. Reusable machinery (do not rebuild these)

Need Already exists
Snapshot config before deleting scanners/lib/baseline.mjs (saveBaseline), used by drift
Diff before/after diffEnvelopes in the same module
Backup + restore individual files w/ sha256 manifest scanners/lib/backup.mjs, scanners/rollback-engine.mjs
Cost of each source, always-loaded subtotal scanners/manifest.mjs, token-hotspots.mjs
Human-approved-writes command pattern knowledge-refresh, campaign (both non-byte-stable by design)

5. Candidate shapes (sketches — pick on Thursday, do not pre-commit)

All three inherit the floor constraint in §6.0: whatever the shape, load-bearing local facts are never deletion candidates, and a rebuild restores them first. A shape that cannot express that distinction is disqualified regardless of how cheap it is.

A. Deterministic vintage scanner (CA-VIN-*). Per instruction block in CLAUDE.md / rules / skills / hooks: age from git history, plus a compensatory-phrasing signal (blocks that exist to correct model behaviour — "ALWAYS", "never forget", "read the whole file first", "don't guess"). Output: ranked deletion candidates with age + token cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner architecture, and composes with tokens/manifest for the payoff figure. Caveat: ~/.claude is not git-tracked, so age there falls back to mtime, which is weaker evidence. Project-level CLAUDE.md/rules get real git ages.

B. Protocol command (/config-audit rebuild). The delete → live with it → earned re-add ledger loop, spanning sessions: archive current config, record what was archived, and maintain a ledger where a line only returns when the operator records that the model actually stumbled on it. Highest fidelity to the source idea; needs cross-session state (the sessions/ machinery already exists) and is inherently not byte-stable.

C. Measured ablation harness. Use --bare + --system-prompt/--add-dir to run the same prompt with and without a block and compare. Closest to a real verifier, most expensive in tokens, weakest determinism. Probably a later --experimental sub-mode of A or B rather than its own command.

Likely landing: A first (deterministic, testable, immediately useful), with B as the workflow that consumes A's output. C stays a documented idea until A+B exist.

6. Hard constraints (carry these into any implementation)

6.0 The floor: compensatory vs load-bearing instructions

This is the invariant all three shapes in §5 must respect, and it is what makes the feature safe to ship at all. Not every line in a CLAUDE.md is the same kind of thing:

Class What it is Test Disposition
Compensatory An instruction correcting model behaviour — "read the whole file first", "don't guess", "ALWAYS verify", "think before you code" A smarter model would do this unprompted Deletion candidate. Returns only when earned.
Load-bearing A local fact the model cannot derive at any capability level — "never GitHub, only Forgejo at git.fromaitochitta.com", "system bash is 3.2, no declare -A", "test with node --test 'tests/**/*.test.mjs'", "~/.claude is not git-tracked" No amount of intelligence produces this from the codebase alone Floor. Never a deletion candidate.

Model capability erodes the first class and does nothing to the second. That is the whole mechanism behind the source idea — and it means "delete your CLAUDE.md" is only correct for one of the two classes. A tool that treats them alike would delete the operator's Forgejo constraint because Opus 5 "is smart enough now", which is a category error: the model isn't failing at intelligence there, it simply cannot know.

Consequence for the rebuild ordering. A rebuild is not one undifferentiated add-back-on-stumble loop. It is three tiers:

  1. Floor — goes back immediately, no trial period. Load-bearing facts. The config is never in a state where these are absent; a "delete everything" that drops them is a broken experiment, not a brave one.
  2. Earned — out, returns only on repeated observed stumbling. Compensatory instructions that turn out to still be needed by this model.
  3. Dead — out and never missed. The payoff, measurable in tokens via manifest.

A third class sits deliberately outside the axis: policy prohibitions ("never commit secrets"). These may well be things the model would honour unprompted, but their cost of being wrong is asymmetric and they are cheap. They stay in the floor by decision, not by classification. Do not let a "the model knows this now" argument reach them.

The hard part is tier 1 vs tier 2, and that classification — not the deletion mechanics — is the real engineering problem in this brief. Precision is asymmetric: a missed dead line costs a few tokens per turn; a deleted load-bearing line costs a wrong remote, a broken bash script, or a lost afternoon.

6.1 Existing repo constraints

  • ~/.claude is NOT git-tracked. A literal rm there is unrecoverable. Archive by mv into _archive/, never delete. This is already an inviolable rule in STATE.md.
  • settings.json is pathguard-protected — no Edit/Write. Use jq + temp file + atomic mv, with operator OK (settings-json-pathguard-write).
  • New scanner ⇒ the 7-step byte-stability checklist (adding-scanner-byte-stability): scanner + orchestrator + scoring area-map + strip-added-scanner + humanizer count + SC-5 + humanizer wiring (SCANNER_TO_CATEGORY entry and TRANSLATIONS.static per RAW title — M-16/M-17). Frozen tests/snapshots/v5.0.0/ must stay untouched.
  • TDD is inviolable — red tests before implementation, including for .md contracts.
  • feat: commits require non-trivial diffs in both README.md and CLAUDE.md (docs-gate-feat-requires-readme-claudemd).

7. Open questions for Thursday

  1. Scope of the deletion candidate set: user-level ~/.claude only, project only, or both? (Age evidence is much stronger for project-level, per §5A.)
  2. The core question, given §6.0: can compensatory-vs-load-bearing be classified deterministically with acceptable precision, or does it need the agent layer (like optimize's precision-gated optimization-lens-agent, which stays silent when unsure)? Note the two signals are independent — a load-bearing fact can be old, and a compensatory instruction can be new — so age alone can never carry this call. Prior: a deterministic pre-filter feeding a precision-gated judge, which is exactly the optimize architecture already in the repo. Design the §8 fasit around the ambiguous middle, not the poles. The four named must-survive items are clear-cut and any mechanism will get them right; the gate is really decided by blocks like "Conventional Commits: type(scope): beskrivelse" (local convention, or a nag the model would follow anyway?), "commit ofte med beskrivende meldinger" (pure behaviour correction?), or the model-routing rubric (a table of local policy that reads like advice). Include 35 such blocks deliberately. A fasit built only from obvious cases will pass a tool that fails on real config.
  3. Does this ship as its own command, or as a --subtract mode of optimize? Both are defensible; command count is already 21.
  4. Ordering against the existing queue. Decided 2026-07-29: the operator prioritized this work ahead of pipeline step 4 (rollback), to start Thursday 2026-07-30. Do not re-litigate. The open part is only what follows it — the prior order stands underneath: step 4 rollback, then the M-11→M-20 batch release, then the v5.13 plan.

8. Verifisering (testable criteria)

Before building anything:

  • grep -rniE 'blame|mtime|birthtime' scanners/ returns no instruction-age handling → confirms §3.1 still true at build time.
  • node scanners/drift-cli.mjs <path> --save --name pre-subtraction succeeds and lib/baseline.mjs round-trips → confirms §4 reuse is real, not assumed.
  • If any claim from §1 is to be written into the register: a primary source exists (URL to the talk or an Anthropic doc). Otherwise the entry is not written.
  • The age signal actually exists. §5A's fallback claim — "project-level CLAUDE.md/rules get real git ages" — is untested. Run git log --format=%ad --follow -- CLAUDE.md .claude/rules/ across two or three real repos. If most instruction blocks trace back to a single bulk commit, per-block age carries no information, shape A's primary signal collapses to the compensatory-phrasing heuristic alone, and §7.2 (agent layer) is answered for us. Check this before writing any scanner code.

If shape A is built:

  • Red tests exist and fail before implementation.
  • node --test 'tests/**/*.test.mjs' green, count strictly greater than 1365.
  • git diff --stat tests/snapshots/v5.0.0/ is empty (frozen baselines untouched).
  • node scanners/self-audit.mjs --check-readme passes (badge counts updated: scanners 16 → 17).
  • Every new finding renders with a non-Other userImpactCategory and a non-_default action language → humanizer wiring correct (M-16/M-17).
  • Floor test (§6.0), the blocking one. Build a fasit by hand first — per judge-the-judge-build-fasit-first — over the operator's own global CLAUDE.md, labelling each block compensatory or load-bearing. Then run the tool. Required result: zero load-bearing blocks proposed for deletion. Non-negotiable; the known-correct floor items to check for explicitly are "never GitHub, only Forgejo", "system bash is 3.2", the test command, and "~/.claude is not git-tracked". One of those in the delete list fails the gate outright, whatever the recall.
  • Recall, secondary: at least one candidate the operator agrees is genuinely dead weight. Precision over recall throughout — a missed dead line costs a few tokens per turn, a deleted load-bearing line costs a wrong remote or a broken script.