open/config-audit
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

feat(tokens): MCP tool-schema deferral check + CLI-over-MCP lever (v5.10 B4)

Browse source 
By default Claude Code defers MCP tool schemas (names-only, ~120 tok; full
schemas on demand via tool search). CA-TOK-006 detects config-file signals that
force the FULL schemas into the always-loaded prefix every turn:
  - settings.json env.ENABLE_TOOL_SEARCH="false"          (high)
  - "ToolSearch" in permissions.deny                       (high)
  - configured model is a Haiku model                      (medium)
  - per-server .mcp.json alwaysLoad:true (CC v2.1.121+)    (high)
auto[:N] is threshold mode (info, not a trigger).

New engine lib/mcp-deferral.mjs: pure assessMcpDeferral({settings,mcpServers})
(unit-tested, no IO) + thin IO wrapper assessMcpDeferralForRepo shared by TOK and
GAP. Severity scales with aggregate forced-upfront tokens (medium-confidence
reasons cap at medium). feature-gap cliOverMcpLeverFinding fires only as a
companion to CA-TOK-006 (prefer gh/aws/gcloud over MCP for common ops).

Honest scoping (Verifiseringsplikt): triggers on config files ONLY — never
process.env shell vars. Vertex / custom ANTHROPIC_BASE_URL / runtime /model
switch are launch state (would flap snapshots machine-dependently), so they are
DISCLOSED in every finding, not triggered. Tool-level anthropic/alwaysLoad and
claude.ai connectors likewise disclosed. Mechanism verified 2026-06-23 against
code.claude.com/docs (context-window.md, mcp.md#configure-tool-search +
#exempt-a-server-from-deferral, costs.md); the prefix-cache invalidation claim
was NOT-CONFIRMED in docs and is not asserted.

alwaysLoad added to CA-MCP VALID_SERVER_FIELDS (no longer flagged as unknown).
active-config-reader surfaces per-server alwaysLoad. Byte-stable: CA-TOK-006
fires only on new conditions; frozen v5.0.0 + SC-5 snapshots untouched.
Tests 1215 -> 1239 (engine 16, integration 5, lever 2, mcp-field guard 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-06-23 19:59:14 +02:00
commit 8f7e196046
13 changed files with 692 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

41
docs/scanner-internals.md
View file

@ -18,7 +18,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
| `import-resolver.mjs` | IMP | Broken @imports, circular refs, deep chains, tilde paths |
| `conflict-detector.mjs` | CNF | Settings conflicts, permission contradictions, hook duplicates |
| `feature-gap-scanner.mjs` | GAP | 25 feature checks across 4 tiers — shown as opportunities, not grades |
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascade, bloated SKILL.md descriptions, MCP tool-schema budget (prompt-cache patterns) |
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascade, bloated SKILL.md descriptions, MCP tool-schema budget, MCP tool-schema deferral (CA-TOK-006), stale plugin-cache disk-cleanup (prompt-cache patterns) |
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of CLAUDE.md cascade (beyond Pattern A's top-30 window) |
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries (low). (1) Tools in BOTH `permissions.deny` AND `permissions.allow` — deny wins; dominance is param-aware and treats the `Tool(*)` deny-all glob as equivalent to a bare deny (covers a bare allow). (2) Unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — CC accepts allow globs only after a literal glob-free `mcp__<server>__` prefix. Predicates shared with CNF live in `lib/permission-rules.mjs` |
| `collision-scanner.mjs` | COL | Cross-plugin skill name collisions (low); user-vs-plugin overlaps (medium); `details.namespaces` payload |
@ -146,6 +146,45 @@ compare (proves the original schema is byte-identical), and the **SC-5 default-o
track current output — diff reviewed as additive-only. **Lesson for any future hotspot/scanner-output
field:** grep every frozen-v5.0.0 comparator (it is 5 files, not 2) before assuming the blast radius.
### token-hotspots — MCP tool-schema deferral (v5.10 B4, CA-TOK-006)
By default Claude Code **defers** MCP tool schemas: only tool *names* enter the always-loaded prefix
(~120 tokens total) and full schemas load on demand via tool search. Several signals force the FULL
schemas into the prefix every turn instead. CA-TOK-006 detects them from **config files only**, so the
finding is deterministic and hermetic-safe (mirrors Pattern G's project-local scoping):
| Signal | Source | Confidence |
|--------|--------|------------|
| `ENABLE_TOOL_SEARCH: "false"` | merged project+local settings.json `env` block | high |
| `"ToolSearch"` in `permissions.deny` | settings.json | high |
| configured `model` matches `/haiku/` | settings.json (Haiku lacks `tool_reference` support) | medium |
| per-server `alwaysLoad: true` | project `.mcp.json` (CC v2.1.121+) | high |
| `ENABLE_TOOL_SEARCH: "auto[:N]"` | settings `env` | threshold mode — **info, not a trigger** |
The engine (`lib/mcp-deferral.mjs`) splits a **pure** `assessMcpDeferral({settings, mcpServers})`
(fully unit-tested, no IO) from a thin IO wrapper `assessMcpDeferralForRepo(repoPath, {mcpServers})`
shared by TOK and GAP. Severity scales with the aggregate forced-upfront token cost
(`severityForForcedSchemas`: ≥5000→high, ≥1500→medium; **medium-confidence reasons cap at medium**).
**Honest scoping decision (Verifiseringsplikt).** The detector deliberately does **NOT** read
`process.env` shell vars. Tool search is also disabled on **Vertex AI**, with a custom
**`ANTHROPIC_BASE_URL`** (non-first-party host), or after a runtime **`/model`** switch to Haiku — but
those are launch/runtime state, not config files, so triggering on them would make the finding
machine-dependent (the marketplace-medium snapshot has MCP servers; an ambient `ANTHROPIC_BASE_URL`
would flap it). They are **disclosed** in every finding (`DEFERRAL_DISCLOSURE`), never triggered.
Tool-level `anthropic/alwaysLoad` (set server-side in the `tools/list` `_meta`) and claude.ai
connectors are likewise invisible to a static scan and disclosed. Mechanism verified 2026-06-23
against `code.claude.com/docs`: `context-window.md` (MCP deferred, ~120 tok),
`mcp.md#configure-tool-search` + `#exempt-a-server-from-deferral`, `costs.md`. The prefix-cache
connect/disconnect-invalidation claim from the raw research was **`[NOT CONFIRMED]`** in docs and is
NOT asserted by this finding.
**feature-gap companion.** `cliOverMcpLeverFinding` (GAP) fires **only** when CA-TOK-006's assessment
shows schemas forced upfront — recommends preferring CLI (`gh`/`aws`/`gcloud`) over MCP for common
operations (CLI adds zero context tokens until invoked). Deferred MCP is effectively free, so the
lever stays silent in the default case (opportunity, not noise — mirrors the bundledSkills lever).
`alwaysLoad` was added to CA-MCP's `VALID_SERVER_FIELDS` so it is never flagged as an unknown field.
### CML scanner — context-window-scaled char budget
Beyond the line-count checks (200/500 lines, both MEDIUM), the CML scanner mirrors