open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

feat(tokens): MCP tool-schema deferral check + CLI-over-MCP lever (v5.10 B4)

Browse source 
By default Claude Code defers MCP tool schemas (names-only, ~120 tok; full
schemas on demand via tool search). CA-TOK-006 detects config-file signals that
force the FULL schemas into the always-loaded prefix every turn:
  - settings.json env.ENABLE_TOOL_SEARCH="false"          (high)
  - "ToolSearch" in permissions.deny                       (high)
  - configured model is a Haiku model                      (medium)
  - per-server .mcp.json alwaysLoad:true (CC v2.1.121+)    (high)
auto[:N] is threshold mode (info, not a trigger).

New engine lib/mcp-deferral.mjs: pure assessMcpDeferral({settings,mcpServers})
(unit-tested, no IO) + thin IO wrapper assessMcpDeferralForRepo shared by TOK and
GAP. Severity scales with aggregate forced-upfront tokens (medium-confidence
reasons cap at medium). feature-gap cliOverMcpLeverFinding fires only as a
companion to CA-TOK-006 (prefer gh/aws/gcloud over MCP for common ops).

Honest scoping (Verifiseringsplikt): triggers on config files ONLY — never
process.env shell vars. Vertex / custom ANTHROPIC_BASE_URL / runtime /model
switch are launch state (would flap snapshots machine-dependently), so they are
DISCLOSED in every finding, not triggered. Tool-level anthropic/alwaysLoad and
claude.ai connectors likewise disclosed. Mechanism verified 2026-06-23 against
code.claude.com/docs (context-window.md, mcp.md#configure-tool-search +
#exempt-a-server-from-deferral, costs.md); the prefix-cache invalidation claim
was NOT-CONFIRMED in docs and is not asserted.

alwaysLoad added to CA-MCP VALID_SERVER_FIELDS (no longer flagged as unknown).
active-config-reader surfaces per-server alwaysLoad. Byte-stable: CA-TOK-006
fires only on new conditions; frozen v5.0.0 + SC-5 snapshots untouched.
Tests 1215 -> 1239 (engine 16, integration 5, lever 2, mcp-field guard 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-06-23 19:59:14 +02:00
commit 8f7e196046
13 changed files with 692 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

26
tests/scanners/feature-gap-scanner.test.mjs
View file

@ -5,7 +5,7 @@ import { fileURLToPath } from 'node:url';
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { resetCounter } from '../../scanners/lib/output.mjs';
import { scan, opportunitySummary, bundledSkillsLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
@ -340,3 +340,27 @@ describe('GAP scanner — disableBundledSkills lever wiring (HOME-scoped)', () =
}
});
});
describe('cliOverMcpLeverFinding (CLI-over-MCP lever, v5.10 B4)', () => {
it('returns null when nothing is forced upfront', () => {
assert.equal(cliOverMcpLeverFinding({ assessment: { forcedUpfront: false } }), null);
assert.equal(cliOverMcpLeverFinding({ assessment: null }), null);
assert.equal(cliOverMcpLeverFinding({}), null);
});
it('fires a low-severity opportunity when MCP schemas are forced upfront', () => {
resetCounter();
const f = cliOverMcpLeverFinding({
assessment: {
forcedUpfront: true,
aggregateTokens: 2500,
affectedServers: [{ name: 'srv-a' }],
reason: 'enable-tool-search-false',
},
});
assert.ok(f, 'expected a lever finding');
assert.equal(f.severity, 'low');
assert.equal(f.category, 'token-efficiency');
assert.match(f.recommendation || '', /\bgh\b|\baws\b|\bgcloud\b/);
});
});