Kjell Tore Guttormsen
8f7e196046
By default Claude Code defers MCP tool schemas (names-only, ~120 tok; full
schemas on demand via tool search). CA-TOK-006 detects config-file signals that
force the FULL schemas into the always-loaded prefix every turn:
- settings.json env.ENABLE_TOOL_SEARCH="false" (high)
- "ToolSearch" in permissions.deny (high)
- configured model is a Haiku model (medium)
- per-server .mcp.json alwaysLoad:true (CC v2.1.121+) (high)
auto[:N] is threshold mode (info, not a trigger).
New engine lib/mcp-deferral.mjs: pure assessMcpDeferral({settings,mcpServers})
(unit-tested, no IO) + thin IO wrapper assessMcpDeferralForRepo shared by TOK and
GAP. Severity scales with aggregate forced-upfront tokens (medium-confidence
reasons cap at medium). feature-gap cliOverMcpLeverFinding fires only as a
companion to CA-TOK-006 (prefer gh/aws/gcloud over MCP for common ops).
Honest scoping (Verifiseringsplikt): triggers on config files ONLY — never
process.env shell vars. Vertex / custom ANTHROPIC_BASE_URL / runtime /model
switch are launch state (would flap snapshots machine-dependently), so they are
DISCLOSED in every finding, not triggered. Tool-level anthropic/alwaysLoad and
claude.ai connectors likewise disclosed. Mechanism verified 2026-06-23 against
code.claude.com/docs (context-window.md, mcp.md#configure-tool-search +
#exempt-a-server-from-deferral, costs.md); the prefix-cache invalidation claim
was NOT-CONFIRMED in docs and is not asserted.
alwaysLoad added to CA-MCP VALID_SERVER_FIELDS (no longer flagged as unknown).
active-config-reader surfaces per-server alwaysLoad. Byte-stable: CA-TOK-006
fires only on new conditions; frozen v5.0.0 + SC-5 snapshots untouched.
Tests 1215 -> 1239 (engine 16, integration 5, lever 2, mcp-field guard 1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
70 lines
3.4 KiB
JavaScript
70 lines
3.4 KiB
JavaScript
import { describe, it } from 'node:test';
|
|
import assert from 'node:assert/strict';
|
|
import { resolve } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
import { resetCounter } from '../../scanners/lib/output.mjs';
|
|
import { scan } from '../../scanners/token-hotspots.mjs';
|
|
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
|
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
|
|
|
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
|
const FIXTURES = resolve(__dirname, '../fixtures');
|
|
|
|
// Hermetic HOME so readActiveConfig does not leak the developer's real
|
|
// ~/.claude.json plugin MCP servers into the fixture result (mirrors the other
|
|
// TOK tests). The deferral check reads project+local settings + project .mcp.json.
|
|
async function runScanner(fixtureName) {
|
|
resetCounter();
|
|
const path = resolve(FIXTURES, fixtureName);
|
|
const discovery = await discoverConfigFiles(path);
|
|
return withHermeticHome(() => scan(path, discovery));
|
|
}
|
|
|
|
const TITLE = 'MCP tool schemas forced into the always-loaded prefix';
|
|
const findDeferral = (result) => result.findings.find(f => f.title === TITLE);
|
|
|
|
describe('TOK scanner — MCP deferral (v5.10 B4, CA-TOK-006)', () => {
|
|
it('deferred (default) project → NO deferral finding', async () => {
|
|
const result = await runScanner('mcp-deferral/deferred');
|
|
assert.equal(findDeferral(result), undefined,
|
|
'a plain server with tool search on must not be flagged as forced-upfront');
|
|
});
|
|
|
|
it('alwaysLoad:true server → deferral finding fires (medium severity)', async () => {
|
|
const result = await runScanner('mcp-deferral/alwaysload');
|
|
const f = findDeferral(result);
|
|
assert.ok(f, `expected a deferral finding; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
|
assert.equal(f.severity, 'medium', `expected medium for ~2500 forced tokens, got ${f.severity}`);
|
|
assert.equal(f.category, 'token-efficiency');
|
|
// Only the alwaysLoad server is named, not the deferred sibling.
|
|
assert.match(String(f.evidence || ''), /always-srv/);
|
|
assert.doesNotMatch(String(f.description || ''), /deferred-srv/);
|
|
assert.match(String(f.evidence || ''), /alwaysLoad/);
|
|
});
|
|
|
|
it('ENABLE_TOOL_SEARCH="false" in settings → deferral finding (all servers affected)', async () => {
|
|
const result = await runScanner('mcp-deferral/disabled-settings');
|
|
const f = findDeferral(result);
|
|
assert.ok(f, `expected a deferral finding for tool-search-disabled fixture`);
|
|
assert.match(String(f.evidence || ''), /reason=enable-tool-search-false/);
|
|
assert.match(String(f.description || ''), /Tool search is disabled/);
|
|
});
|
|
|
|
it('every deferral finding discloses runtime conditions + calibration', async () => {
|
|
const result = await runScanner('mcp-deferral/alwaysload');
|
|
const f = findDeferral(result);
|
|
assert.ok(f);
|
|
// DEFERRAL_DISCLOSURE names the launch/runtime conditions a static scan can't see.
|
|
assert.match(String(f.evidence || ''), /Vertex AI/);
|
|
assert.match(String(f.evidence || ''), /ANTHROPIC_BASE_URL/);
|
|
// CALIBRATION_NOTE shared by all TOK pattern findings.
|
|
assert.match(String(f.evidence || ''), /severity reflects estimated tokens\/turn/i);
|
|
});
|
|
|
|
it('finding ID matches CA-TOK-NNN format', async () => {
|
|
const result = await runScanner('mcp-deferral/alwaysload');
|
|
const f = findDeferral(result);
|
|
assert.ok(f);
|
|
assert.match(f.id, /^CA-TOK-\d{3}$/);
|
|
});
|
|
});
|