Operator corrected two things about the brief committed in 3086e8b/3252b51.
Provenance: the operator watched the recording and identifies Boris Cherny on
stage, so the attribution is confirmed by direct observation, not a channel's
claim. The verbatim figures (80 %, "more intelligent without the prompts") still
reach us through the summary's editing and stay at that confidence level. §1 now
carries both levels separately, and the register source string reflects the split
instead of flattening to "unverified".
Design: "start with what it must have" is the constraint the whole feature turns
on, so it is a hard constraint (§6.0), not a candidate-shape detail. Model
capability erodes compensatory instructions ("read the whole file first") and
does nothing to load-bearing local facts ("only Forgejo", "bash is 3.2", the test
command) — the model isn't failing at intelligence there, it cannot know. A tool
that treats them alike deletes the Forgejo constraint because Opus 5 "is smart
enough now". Rebuild is therefore three tiers: floor restored immediately, earned
returns on repeated stumbling, dead never comes back. Policy prohibitions stay in
the floor by decision rather than classification — asymmetric cost, cheap to keep.
Consequences threaded through: §5 disqualifies any shape that cannot express the
distinction, §7.2 becomes the core open question (age and class are independent
signals, so age alone can never carry the call), and §8 gains a blocking floor
test with a hand-built fasit and named must-survive items.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JNrtHo9hKSLKNyMS6b4Zuy
234 lines
14 KiB
Markdown
234 lines
14 KiB
Markdown
# Brief — Delete-and-Rebuild (config subtraction)
|
|
|
|
**Status:** BRIEF, not a plan. No code, no chunk breakdown, no version number committed.
|
|
Written 2026-07-29 so a session starting Thursday evening has a durable starting point.
|
|
STATE.md is gitignored in this repo, so this file — not STATE — is the record.
|
|
|
|
---
|
|
|
|
## 1. Trigger and its provenance
|
|
|
|
The operator relayed a third-party YouTube summary (Hyper Automation Labs) of a talk
|
|
Boris Cherny reportedly gave at Y Combinator Startup School, one day after Opus 5
|
|
shipped. Claims attributed to him in that summary:
|
|
|
|
- Anthropic deleted ~80 % of Claude Code's own system prompt when Opus 5 landed.
|
|
- Advice to users: every six months, delete your CLAUDE.md, your skills, your hooks —
|
|
see what the model does.
|
|
- Rebuild method: delete everything, use it, add one line back only when the model
|
|
stumbles on the same thing repeatedly.
|
|
- The model measured *slightly more intelligent* with the built-in prompts stripped.
|
|
|
|
**Two levels of confidence here, and they must not be collapsed** (updated 2026-07-29
|
|
after the operator corrected the first draft):
|
|
|
|
- **Attribution — confirmed.** The operator watched the recording and identifies Boris
|
|
Cherny on stage. This is not a channel's claim about who spoke; it is direct
|
|
observation by the operator. The talk happened and it is him.
|
|
- **The verbatim figures — still summary-level.** "80 % of the system prompt", "the
|
|
model measured slightly more intelligent without the prompts", the Bun numbers: these
|
|
reach us through the channel's editing, not through a primary transcript. They are
|
|
plausible and consistent with §2, and they are not quoted as fact anywhere below.
|
|
|
|
So if this becomes a `BP-*` entry in the best-practices register, the source field reads
|
|
*"Boris Cherny, YC Startup School talk (attribution confirmed by operator); figures via
|
|
third-party summary, not primary-verified"* — not "unverified", and not a bare citation
|
|
either. Getting a primary transcript for the figures is a nice-to-have, never a
|
|
prerequisite: the feature is argued from this repo's own logic (§3), and this repo has
|
|
one scar from treating a plausible quote as load-bearing fact («Fable low ≈ Opus high»,
|
|
fabricated, rejected 2026-07-14).
|
|
|
|
**What the operator has affirmed as in scope:** delete CLAUDE.md, then rebuild by adding
|
|
back what the model needs help with — *starting with what it must have*. That last clause
|
|
is not a detail; it is the design constraint in §6.0.
|
|
|
|
## 2. Verified ground truth (checked against the local CLI, 2026-07-29)
|
|
|
|
These *are* facts, and they are what make an empirical variant buildable:
|
|
|
|
| Fact | How verified |
|
|
|---|---|
|
|
| `claude --bare` exists — "Minimal mode: skip hooks, LSP, plugin sync, attribution, auto-memory, background prefetches, keychain reads, and CLAUDE.md auto-discovery. Sets `CLAUDE_CODE_SIMPLE=1`." | `claude --help` |
|
|
| `claude --system-prompt <prompt>` — replaces the session system prompt | `claude --help` |
|
|
| `--append-system-prompt`, `--add-dir`, `--setting-sources <user,project,local>`, `--plugin-dir`, `--settings`, `--agents` — all present and all usable to compose an isolated config for a controlled run | `claude --help` |
|
|
|
|
So the `CLAUDE_CODE_SIMPLE=1` env var the video calls "undocumented" is, in this CLI
|
|
version, a documented flag (`--bare`). That matters: an A/B ablation harness would not
|
|
need an undocumented hook.
|
|
|
|
## 3. Why this fits this repo (the argument that does *not* depend on §1)
|
|
|
|
The plugin has three pillars — Health, Opportunities, Action. Every existing command
|
|
answers a question on the **addition** axis:
|
|
|
|
- `feature-gap` — what could you add?
|
|
- `optimize` — what would fit a better mechanism?
|
|
- `posture` / `tokens` / `manifest` — how good/expensive is what you have?
|
|
- `fix` / `implement` — apply changes.
|
|
|
|
**Nothing answers the subtraction question: what is no longer earning its rent?**
|
|
`feature-gap` has no inverse. That is a real hole, independent of who said what on
|
|
a stage.
|
|
|
|
Two further reasons this belongs *here* specifically:
|
|
|
|
1. **Nothing in the repo measures instruction age.** Verified by grep over `scanners/`:
|
|
`stale` appears only for knowledge-register entry age (`lib/knowledge-refresh.mjs`)
|
|
and for stale plugin-cache versions (`scan-orchestrator.mjs`, M-BUG-11). No scanner
|
|
touches `git blame`, `mtime`, or the vintage of a CLAUDE.md block. An instruction
|
|
written for Sonnet 3.5 and an instruction written last week are indistinguishable to
|
|
every current scanner.
|
|
2. **Deleting bravely is only sane if you can undo it.** That is already pillar three:
|
|
`lib/backup.mjs`, `rollback-engine.mjs`, `auto-backup-config.mjs` (PreToolUse), and
|
|
`drift`'s `saveBaseline` / `loadBaseline` / `diffEnvelopes`. The safety net exists;
|
|
the feature that would use it does not. This is arguably the strongest framing:
|
|
*config-audit is already the infrastructure that makes "delete it and see" a
|
|
measurement rather than a gamble.*
|
|
|
|
## 4. Reusable machinery (do not rebuild these)
|
|
|
|
| Need | Already exists |
|
|
|---|---|
|
|
| Snapshot config before deleting | `scanners/lib/baseline.mjs` (`saveBaseline`), used by `drift` |
|
|
| Diff before/after | `diffEnvelopes` in the same module |
|
|
| Backup + restore individual files w/ sha256 manifest | `scanners/lib/backup.mjs`, `scanners/rollback-engine.mjs` |
|
|
| Cost of each source, always-loaded subtotal | `scanners/manifest.mjs`, `token-hotspots.mjs` |
|
|
| Human-approved-writes command pattern | `knowledge-refresh`, `campaign` (both non-byte-stable by design) |
|
|
|
|
## 5. Candidate shapes (sketches — pick on Thursday, do not pre-commit)
|
|
|
|
All three inherit the floor constraint in §6.0: whatever the shape, load-bearing local
|
|
facts are never deletion candidates, and a rebuild restores them first. A shape that
|
|
cannot express that distinction is disqualified regardless of how cheap it is.
|
|
|
|
**A. Deterministic vintage scanner (`CA-VIN-*`).** Per instruction block in CLAUDE.md /
|
|
rules / skills / hooks: age from git history, plus a compensatory-phrasing signal
|
|
(blocks that exist to correct model behaviour — "ALWAYS", "never forget", "read the
|
|
whole file first", "don't guess"). Output: ranked deletion candidates with age + token
|
|
cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner
|
|
architecture, and composes with `tokens`/`manifest` for the payoff figure.
|
|
*Caveat:* `~/.claude` is not git-tracked, so age there falls back to `mtime`, which is
|
|
weaker evidence. Project-level CLAUDE.md/rules get real git ages.
|
|
|
|
**B. Protocol command (`/config-audit rebuild`).** The delete → live with it → earned
|
|
re-add ledger loop, spanning sessions: archive current config, record what was archived,
|
|
and maintain a ledger where a line only returns when the operator records that the model
|
|
actually stumbled on it. Highest fidelity to the source idea; needs cross-session state
|
|
(the `sessions/` machinery already exists) and is inherently not byte-stable.
|
|
|
|
**C. Measured ablation harness.** Use `--bare` + `--system-prompt`/`--add-dir` to run the
|
|
same prompt with and without a block and compare. Closest to a real verifier, most
|
|
expensive in tokens, weakest determinism. Probably a later `--experimental` sub-mode of
|
|
A or B rather than its own command.
|
|
|
|
Likely landing: **A first** (deterministic, testable, immediately useful), with B as the
|
|
workflow that consumes A's output. C stays a documented idea until A+B exist.
|
|
|
|
## 6. Hard constraints (carry these into any implementation)
|
|
|
|
### 6.0 The floor: compensatory vs load-bearing instructions
|
|
|
|
**This is the invariant all three shapes in §5 must respect, and it is what makes the
|
|
feature safe to ship at all.** Not every line in a CLAUDE.md is the same kind of thing:
|
|
|
|
| Class | What it is | Test | Disposition |
|
|
|---|---|---|---|
|
|
| **Compensatory** | An instruction correcting model *behaviour* — "read the whole file first", "don't guess", "ALWAYS verify", "think before you code" | A smarter model would do this unprompted | **Deletion candidate.** Returns only when earned. |
|
|
| **Load-bearing** | A *local fact* the model cannot derive at any capability level — "never GitHub, only Forgejo at git.fromaitochitta.com", "system bash is 3.2, no `declare -A`", "test with `node --test 'tests/**/*.test.mjs'`", "`~/.claude` is not git-tracked" | No amount of intelligence produces this from the codebase alone | **Floor. Never a deletion candidate.** |
|
|
|
|
Model capability erodes the first class and does nothing to the second. That is the whole
|
|
mechanism behind the source idea — and it means "delete your CLAUDE.md" is only correct
|
|
for one of the two classes. A tool that treats them alike would delete the operator's
|
|
Forgejo constraint because Opus 5 "is smart enough now", which is a category error: the
|
|
model isn't failing at intelligence there, it simply cannot know.
|
|
|
|
**Consequence for the rebuild ordering.** A rebuild is not one undifferentiated
|
|
add-back-on-stumble loop. It is three tiers:
|
|
|
|
1. **Floor — goes back immediately, no trial period.** Load-bearing facts. The config is
|
|
never in a state where these are absent; a "delete everything" that drops them is a
|
|
broken experiment, not a brave one.
|
|
2. **Earned — out, returns only on repeated observed stumbling.** Compensatory
|
|
instructions that turn out to still be needed by *this* model.
|
|
3. **Dead — out and never missed.** The payoff, measurable in tokens via `manifest`.
|
|
|
|
A third class sits deliberately outside the axis: **policy prohibitions** ("never commit
|
|
secrets"). These may well be things the model would honour unprompted, but their cost of
|
|
being wrong is asymmetric and they are cheap. They stay in the floor by decision, not by
|
|
classification. Do not let a "the model knows this now" argument reach them.
|
|
|
|
The hard part is tier 1 vs tier 2, and that classification — not the deletion mechanics —
|
|
is the real engineering problem in this brief. Precision is asymmetric: a missed dead line
|
|
costs a few tokens per turn; a deleted load-bearing line costs a wrong remote, a broken
|
|
bash script, or a lost afternoon.
|
|
|
|
### 6.1 Existing repo constraints
|
|
|
|
- **`~/.claude` is NOT git-tracked.** A literal `rm` there is unrecoverable. Archive by
|
|
`mv` into `_archive/`, never delete. This is already an inviolable rule in STATE.md.
|
|
- **`settings.json` is pathguard-protected** — no Edit/Write. Use `jq` + temp file +
|
|
atomic `mv`, with operator OK ([[settings-json-pathguard-write]]).
|
|
- **New scanner ⇒ the 7-step byte-stability checklist** ([[adding-scanner-byte-stability]]):
|
|
scanner + orchestrator + scoring area-map + strip-added-scanner + humanizer count +
|
|
SC-5 + humanizer wiring (`SCANNER_TO_CATEGORY` entry and `TRANSLATIONS.static` per RAW
|
|
title — M-16/M-17). Frozen `tests/snapshots/v5.0.0/` must stay untouched.
|
|
- **TDD is inviolable** — red tests before implementation, including for .md contracts.
|
|
- **`feat:` commits require non-trivial diffs in both README.md and CLAUDE.md**
|
|
([[docs-gate-feat-requires-readme-claudemd]]).
|
|
|
|
## 7. Open questions for Thursday
|
|
|
|
1. Scope of the deletion candidate set: user-level `~/.claude` only, project only, or both?
|
|
(Age evidence is much stronger for project-level, per §5A.)
|
|
2. **The core question, given §6.0:** can compensatory-vs-load-bearing be classified
|
|
deterministically with acceptable precision, or does it need the agent layer (like
|
|
`optimize`'s precision-gated `optimization-lens-agent`, which stays silent when
|
|
unsure)? Note the two signals are independent — a load-bearing fact can be old, and a
|
|
compensatory instruction can be new — so age alone can never carry this call. Prior:
|
|
a deterministic pre-filter feeding a precision-gated judge, which is exactly the
|
|
`optimize` architecture already in the repo.
|
|
3. Does this ship as its own command, or as a `--subtract` mode of `optimize`?
|
|
Both are defensible; command count is already 21.
|
|
4. ~~Ordering against the existing queue.~~ **Decided 2026-07-29:** the operator
|
|
prioritized this work ahead of pipeline step 4 (`rollback`), to start Thursday
|
|
2026-07-30. Do not re-litigate. The open part is only what follows it — the
|
|
prior order stands underneath: step 4 `rollback`, then the M-11→M-20 batch
|
|
release, then the v5.13 plan.
|
|
|
|
## 8. Verifisering (testable criteria)
|
|
|
|
**Before building anything:**
|
|
|
|
- [ ] `grep -rniE 'blame|mtime|birthtime' scanners/` returns no instruction-age handling
|
|
→ confirms §3.1 still true at build time.
|
|
- [ ] `node scanners/drift-cli.mjs <path> --save --name pre-subtraction` succeeds and
|
|
`lib/baseline.mjs` round-trips → confirms §4 reuse is real, not assumed.
|
|
- [ ] If any claim from §1 is to be written into the register: a primary source exists
|
|
(URL to the talk or an Anthropic doc). Otherwise the entry is not written.
|
|
- [ ] **The age signal actually exists.** §5A's fallback claim — "project-level
|
|
CLAUDE.md/rules get real git ages" — is untested. Run
|
|
`git log --format=%ad --follow -- CLAUDE.md .claude/rules/` across two or three
|
|
real repos. If most instruction blocks trace back to a single bulk commit, per-block
|
|
age carries no information, shape A's primary signal collapses to the
|
|
compensatory-phrasing heuristic alone, and §7.2 (agent layer) is answered for us.
|
|
Check this *before* writing any scanner code.
|
|
|
|
**If shape A is built:**
|
|
|
|
- [ ] Red tests exist and fail before implementation.
|
|
- [ ] `node --test 'tests/**/*.test.mjs'` green, count strictly greater than 1365.
|
|
- [ ] `git diff --stat tests/snapshots/v5.0.0/` is empty (frozen baselines untouched).
|
|
- [ ] `node scanners/self-audit.mjs --check-readme` passes (badge counts updated:
|
|
scanners 16 → 17).
|
|
- [ ] Every new finding renders with a non-`Other` `userImpactCategory` and a
|
|
non-`_default` action language → humanizer wiring correct (M-16/M-17).
|
|
- [ ] **Floor test (§6.0), the blocking one.** Build a fasit by hand first — per
|
|
[[judge-the-judge-build-fasit-first]] — over the operator's own global CLAUDE.md,
|
|
labelling each block compensatory or load-bearing. Then run the tool. Required
|
|
result: **zero load-bearing blocks proposed for deletion.** Non-negotiable; the
|
|
known-correct floor items to check for explicitly are "never GitHub, only Forgejo",
|
|
"system bash is 3.2", the test command, and "`~/.claude` is not git-tracked". One
|
|
of those in the delete list fails the gate outright, whatever the recall.
|
|
- [ ] Recall, secondary: at least one candidate the operator agrees is genuinely dead
|
|
weight. Precision over recall throughout — a missed dead line costs a few tokens
|
|
per turn, a deleted load-bearing line costs a wrong remote or a broken script.
|