feat(optimize): add --subtract, the subtraction axis, behind a deterministic floor

Every command so far asked an addition question — what to add, what to move,
what it costs. Nothing asked what is no longer earning its always-loaded rent.
This adds that axis as a fourth lensCheck on the existing hybrid motor rather
than a new scanner or a 22nd command: the measured payoff (~18% of one file)
justifies a mode, not machinery.

It is the only lens that proposes REMOVING config, so it carries a guarantee
the others don't need: a load-bearing block is never a candidate. Precision is
asymmetric — a missed dead line costs a few tokens per turn, a deleted one
costs a wrong remote or a broken script — so the floor is decided in code
(lib/floor-exclusion.mjs) before the opus judge sees anything, never in prose.

Granularity is the leaf block, with two structural exceptions: a paragraph
ending in ':' merges with the list it introduces, and an ordered list is a
contract whose steps inherit floor from any sibling. Unordered lists
deliberately do not inherit — a load-bearing bullet and a disposable one
routinely share a list, and container-reasoning is the error the hand-built
ground truth exists to catch.

Verified against that ground truth (built before any classifier existed), with
the comparison machine-checked rather than read by eye: zero load-bearing
blocks proposed, 11/18 deletable groups surfaced, ~756 tok ~ 18% of a ~4300
token file — inside the pre-registered band. The first run found five floor
violations the synthesized fixture missed; each got a structural rule and a
fixture shape so it cannot regress.

Three real bugs the dogfood run exposed, all now covered:
- JS \b is ASCII-only, so /\bunngå\b/ never matches — every Norwegian keyword
  ending in æ/ø/å was silently dead.
- A bare word/word is not a path; "pros/cons" vetoed the largest deletable
  block until PATH_RE was tightened to rooted paths and globs.
- "Mid-sentence" must key on a preceding lowercase letter; the loose version
  read **bold labels:** and quoted openers as entities, costing 4 of 11 groups.

BP-SUB-001 is grounded entirely in the Anthropic steering blog already cited by
BP-MECH-001..004 and asserts nothing from the talk that motivated the feature —
no "80%", no ablation figure.

Suite 1365 -> 1382/0. Frozen v5.0.0 snapshots untouched; plain optimize output
byte-identical on identical input (--subtract adds keys only when passed).
knowledge-refresh-cli's reference date moved to 2026-08-01: its premise that
every seed entry was verified 2026-06-20 expired when BP-SUB-001 got a genuine
verification date, and backdating the entry to fit the test would have been a
lie about when its source was checked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RW2haJXbxZpKivKHseSXNh
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 16:24:51 +02:00
commit e9921d3c9d
13 changed files with 1080 additions and 32 deletions

View file

@ -499,6 +499,56 @@ orchestrates pre-filter→agent→report. **Agent-driven → deliberately NOT by
outside the snapshot suite); the pre-filter lib *is* unit-tested (13 tests). No new orchestrated
scanner → scanner count stays 15; agents 6→7, commands 18→19, suite 1055→1068.
### Subtraction lens (`optimize --subtract`, `BP-SUB-001`) — the inverse axis
**Why a mode, not a command or scanner.** Every other command asks an addition question; nothing
asked what is no longer earning its rent. A hand-built ground truth over a real 250-line global
CLAUDE.md put the honest payoff at ~8501400 always-loaded tokens of ~4300 (~20 %, not the source
anecdote's 80 %), with 26 of 34 blocks load-bearing. That proportion justifies a fourth `lensCheck`
on the existing hybrid motor — not a new scanner (no badge bump 16→17, no frozen-snapshot risk) and
not a 22nd command. Cross-session state is what would have forced a command, and the earned-re-add
*ledger* is deliberately out of v1.
**Polarity is flipped from `lens-prefilter`.** That module is recall-first because a false candidate
only costs the judge a moment. Here a false candidate is a proposal to *delete*, so
`subtraction-prefilter.mjs` is precision-first and carries a blocking deterministic guarantee.
**Floor-exclusion is a separate module on purpose** (`lib/floor-exclusion.mjs`). §6.0's asymmetry —
a missed dead line costs a few tokens per turn, a deleted load-bearing line costs a wrong remote —
means the guarantee must not rest on a probabilistic judge, so the floor runs *before* the agent
and is legible as its own unit. Markers: code span, URL/host, filename, rooted path, version pin,
policy invariant, and `unresolved-entity` (a mixed-case capitalized word mid-sentence). The last is
a deliberate conservative default — resolving "Forgejo" from an ordinary capitalized word needs a
dictionary, so the mechanism declines and keeps the block.
**Granularity: leaf block + two structural exceptions.** (1) A paragraph ending in `:` merges with
the list it introduces — a stem often carries no literal of its own, and deleting it without its
list is meaningless. (2) An *ordered* list is a contract: steps inherit floor from any sibling,
because deleting step 2 of a five-step protocol is not like dropping one platitude. Unordered lists
do **not** inherit — a load-bearing bullet and a disposable one routinely share a list, and
container-reasoning is exactly the error the ground truth was built to catch.
**Three lessons from the dogfood run**, all invisible to the synthesized fixture and worth keeping:
- **JS `\b` is ASCII-only.** `/\bunngå\b/` never matches — the trailing `å` is not a word character,
so there is no boundary after it. Every Norwegian keyword ending in æ/ø/å was silently dead. Use
the `LB`/`RB` lookaround constants, never `\b`, around that vocabulary.
- **A bare `word/word` is not a path.** `pros/cons` vetoed the single largest deletable block until
`PATH_RE` was tightened to rooted paths and globs; real filenames are `FILENAME_RE`'s job.
- **"Mid-sentence" must key on a preceding lowercase letter**, not on "anything that is not a full
stop". The loose version read `**Bold labels:**` and quoted openers (`"Som AI kan jeg ikke…"`) as
entities and cost 4 of 11 deletable groups.
**Measured against the ground truth:** zero load-bearing blocks proposed (the blocking §8 gate),
11/18 deletable groups surfaced, ≈756 tok ≈ 18 % of the file — inside the pre-registered band. The
misses are all the conservative default working as designed (entity names, code spans, and
declarative/infinitive phrasing that carries no imperative). No new scanner: scanners stay 16,
commands 21, agents 7; suite 1365→1382.
**Note for a future narrowing of the veto:** the two failure modes the agent prompt hardens against
— staleness-is-not-deletion and tier-2-is-not-tier-3 — are currently *also* covered by exclusion
(both example blocks carry code spans/version pins and never reach the judge). The prompt language
is the only protection if those markers are ever loosened.
**Test-isolation fix (this session):** `token-hotspots.test.mjs` `runScanner` now wraps `scan()` in
the shared `withHermeticHome` helper — the suite is green on BOTH a real and a clean `HOME` (the OPT
section's old «run with clean HOME» caveat is resolved). Snapshot/byte tests were already hermetic.