feat(feature-gap): recommend disableBundledSkills under skill-listing pressure
Chunk 2 of the disableBundledSkills GAP feature. Adds a conditional GAP check
that prescribes the `disableBundledSkills` lever — but only when the active
skill listing is measurably over budget (SKL's CA-SKL-002 overflow signal) and
the lever is un-pulled. It stays an opportunity, not noise.
Bundled skills (/code-review, /batch, /debug, /loop, /claude-api, …) live in the
CC binary, not on disk, so their exact cost is unmeasurable here — the finding
says so plainly, and frames the lever as zero-cost budget reclaim that leaves
the user's own skills untouched. CC 2.1.169+.
- Pure, exported bundledSkillsLeverFinding({leverPulled, aggregate}) → finding|null
(severity low, category token-efficiency, CA-GAP-NNN), wired into scan() via the
shared measureActiveSkillListing().
- Lever resolved via new isBundledSkillsDisabled(): env var + settings cascade
read directly, because discovery does NOT tag ~/.claude/settings.json (its
relPath lacks ".claude" when walked from the .claude root) — the dominant
user-scope location for this global preference would otherwise be missed.
- GAP scan() now reads HOME → existing GAP tests retrofitted to withHermeticHome
per the hermetic rule. Snapshots unchanged, contamination grep clean.
- 16 new tests (9 GAP, 7 lib). Suite 887 -> 903. README/CLAUDE.md document the
cross-scanner remediation; test counts synced. self-audit: PASS, configGrade
A 96, pluginGrade A 100, readme gate passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ter3E2JSi1Khgmuf2kady8
This commit is contained in:
parent
0a631e3061
commit
dfe9049b55
6 changed files with 338 additions and 12 deletions
|
|
@ -19,7 +19,7 @@ Analyzes and optimizes Claude Code configuration across three pillars:
|
|||
| `/config-audit posture` | Quick health scorecard (A-F grades, 10 quality areas incl. Token Efficiency, Plugin Hygiene) |
|
||||
| `/config-audit tokens` | prompt-cache-aware token hotspots (6 patterns: cache-breaking, redundant perms, deep imports, oversized cascade, bloated SKILL.md desc, MCP tool-schema budget) — optional `--accurate-tokens` API calibration, `--with-telemetry-recipe` cache-hit recipe pointer |
|
||||
| `/config-audit manifest` | Ranked table of every system-prompt token source (CLAUDE.md, plugins, skills, MCP, hooks) sorted by estimated tokens |
|
||||
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
|
||||
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact (incl. a conditional `disableBundledSkills` lever when the active skill listing is over budget — remediation companion to SKL `CA-SKL-002`) |
|
||||
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
||||
| `/config-audit rollback` | Restore configuration from backup |
|
||||
| `/config-audit plan` | Create action plan from audit findings |
|
||||
|
|
@ -109,7 +109,7 @@ Default: auto-detects scope from git context. Override with `/config-audit full|
|
|||
node --test 'tests/**/*.test.mjs'
|
||||
```
|
||||
|
||||
887 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`.
|
||||
903 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`.
|
||||
|
||||
## Gotchas
|
||||
|
||||
|
|
|
|||
13
README.md
13
README.md
|
|
@ -12,7 +12,7 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 13 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, and cross-plugin collision detection. Zero external dependencies.
|
||||
|
|
@ -306,12 +306,21 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
| `mcp-config-validator.mjs` | MCP | Invalid server types, exposed env vars, unknown fields |
|
||||
| `import-resolver.mjs` | IMP | Broken @imports, circular references, deep chains, tilde path issues |
|
||||
| `conflict-detector.mjs` | CNF | Settings contradictions across scopes, permission conflicts, hook duplicates |
|
||||
| `feature-gap-scanner.mjs` | GAP | 25 feature checks — shown as opportunities, not grades |
|
||||
| `feature-gap-scanner.mjs` | GAP | 25 feature checks shown as opportunities, not grades — plus a conditional `disableBundledSkills` recommendation when the active skill listing is over budget |
|
||||
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascades, bloated skill descriptions, MCP tool-schema budget |
|
||||
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of the CLAUDE.md cascade — beyond the cache-prefix window but still re-loaded every turn |
|
||||
| `disabled-in-schema-scanner.mjs` | DIS | Tools listed in BOTH `permissions.deny` and `permissions.allow` — deny wins, allow entries are dead config |
|
||||
| `collision-scanner.mjs` | COL | Cross-plugin skill name collisions; user-vs-plugin overlaps |
|
||||
|
||||
> **Cross-scanner remediation — diagnosis meets the fix.** SKL diagnoses an over-budget
|
||||
> skill listing (`CA-SKL-002`); GAP prescribes the remedy. When the active skill listing
|
||||
> exceeds its ~2%-of-context budget and `disableBundledSkills` is not already set (in the
|
||||
> env var or the settings cascade), the feature-gap scanner recommends that lever — hiding
|
||||
> Claude Code's bundled skills (`/code-review`, `/batch`, `/debug`, `/loop`, `/claude-api`, …)
|
||||
> from the model to reclaim listing budget without touching your own skills (CC 2.1.169+).
|
||||
> It fires only under measured pressure, so it stays an opportunity rather than noise. Both
|
||||
> scanners share one budget definition (`scanners/lib/skill-listing-budget.mjs`).
|
||||
|
||||
### CLI Tools
|
||||
|
||||
All tools work standalone — no Claude Code session needed:
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
/**
|
||||
* GAP Scanner — Feature Gap Scanner
|
||||
* Compares actual configuration against complete Claude Code feature register.
|
||||
* 25 gap dimensions across 4 tiers. Always runs with includeGlobal: true.
|
||||
* 25 gap dimensions across 4 tiers, plus a conditional disableBundledSkills
|
||||
* budget-lever check (remediation companion to SKL CA-SKL-002, fires only under
|
||||
* measured skill-listing pressure). Always runs with includeGlobal: true.
|
||||
* Finding IDs: CA-GAP-NNN
|
||||
*/
|
||||
|
||||
|
|
@ -10,6 +12,7 @@ import { readTextFile, discoverConfigFiles } from './lib/file-discovery.mjs';
|
|||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { findImports, parseJson, parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { measureActiveSkillListing, isBundledSkillsDisabled, BUDGET_CALIBRATION_NOTE } from './lib/skill-listing-budget.mjs';
|
||||
|
||||
const SCANNER = 'GAP';
|
||||
|
||||
|
|
@ -87,6 +90,50 @@ function getSettingsValue(ctx, key) {
|
|||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remediation companion to SKL CA-SKL-002: when the active skill listing is over
|
||||
* its budget and the `disableBundledSkills` lever is un-pulled, recommend it.
|
||||
*
|
||||
* Bundled (built-in) skills — /code-review, /batch, /debug, /loop, /claude-api
|
||||
* and more — live in the Claude Code binary, not on disk, so their exact listing
|
||||
* cost cannot be measured here. But they draw on the SAME budget the SKL scanner
|
||||
* measures; when that budget is already exceeded, dropping them is a zero-cost
|
||||
* lever that does not touch the user's own skills. We fire ONLY under measured
|
||||
* pressure (SKL's overflow signal) so this stays an opportunity, not noise.
|
||||
*
|
||||
* Pure and exported for unit testing.
|
||||
*
|
||||
* @param {{ leverPulled: boolean, aggregate: (import('./lib/skill-listing-budget.mjs').BudgetAssessment|null) }} args
|
||||
* @returns {object|null} a GAP finding, or null when the lever is pulled or the listing is within budget
|
||||
*/
|
||||
export function bundledSkillsLeverFinding({ leverPulled, aggregate }) {
|
||||
if (leverPulled) return null;
|
||||
if (!aggregate || !aggregate.overBudget) return null;
|
||||
|
||||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
title: 'Bundled skills add to an over-budget skill listing',
|
||||
description:
|
||||
`Your ${aggregate.scanned} active skills already carry ~${aggregate.aggregateTokens} tokens of ` +
|
||||
`description text, over the ${aggregate.budgetTokens}-token listing budget Claude Code allots the ` +
|
||||
'skill listing on a 200k context window (~2% of context, CC 2.1.32). Claude Code also loads its ' +
|
||||
'bundled (built-in) skills — /code-review, /batch, /debug, /loop, /claude-api and more — into that ' +
|
||||
'same listing. They are not on disk, so their exact cost cannot be measured here, but they draw on ' +
|
||||
'the same budget. `disableBundledSkills: true` drops them from the listing, reclaiming space without ' +
|
||||
'touching your own skills.',
|
||||
evidence:
|
||||
`description_tokens~${aggregate.aggregateTokens}; budget@200k=${aggregate.budgetTokens} tok; over_by~` +
|
||||
`${aggregate.overBy} tok; lever=disableBundledSkills (unset) - ${BUDGET_CALIBRATION_NOTE}`,
|
||||
recommendation:
|
||||
'Set `disableBundledSkills: true` in settings.json (or the CLAUDE_CODE_DISABLE_BUNDLED_SKILLS env var) ' +
|
||||
'to hide built-in skills and slash commands from the model and reclaim skill-listing budget (CC 2.1.169+). ' +
|
||||
'Keep it off if you rely on bundled skills like /code-review — in that case trim your own skill ' +
|
||||
'descriptions or use `skillOverrides` instead.',
|
||||
category: 'token-efficiency',
|
||||
});
|
||||
}
|
||||
|
||||
/** @type {GapCheck[]} */
|
||||
const GAP_CHECKS = [
|
||||
// --- Tier 1: Foundation ---
|
||||
|
|
@ -386,6 +433,14 @@ export async function scan(targetPath, sharedDiscovery) {
|
|||
}
|
||||
}
|
||||
|
||||
// disableBundledSkills lever — fires only under measured skill-listing pressure
|
||||
// (SKL's CA-SKL-002 overflow signal). HOME-scoped: the listing and the lever
|
||||
// cascade both resolve via process.env.HOME, independent of project discovery.
|
||||
const leverPulled = await isBundledSkillsDisabled(ctx.targetPath);
|
||||
const { aggregate } = await measureActiveSkillListing();
|
||||
const leverFinding = bundledSkillsLeverFinding({ leverPulled, aggregate });
|
||||
if (leverFinding) findings.push(leverFinding);
|
||||
|
||||
const filesScanned = discovery.files.length;
|
||||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -19,9 +19,10 @@
|
|||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { join } from 'node:path';
|
||||
import { estimateTokens, enumeratePlugins, enumerateSkills } from './active-config-reader.mjs';
|
||||
import { readTextFile } from './file-discovery.mjs';
|
||||
import { parseFrontmatter } from './yaml-parser.mjs';
|
||||
import { parseFrontmatter, parseJson } from './yaml-parser.mjs';
|
||||
|
||||
// Verified per-description skill-listing cap (CC 2.1.105, changelog L1502).
|
||||
// Descriptions longer than this are truncated in the listing the model sees.
|
||||
|
|
@ -123,3 +124,49 @@ export async function measureActiveSkillListing() {
|
|||
const aggregate = assessSkillListingBudget(skills.map((s) => s.descLength));
|
||||
return { skills, aggregate };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read an env flag, treating null, "", "0", "false", "no", "off" as un-set.
|
||||
* @param {string|undefined} v
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function envFlag(v) {
|
||||
if (v == null) return false;
|
||||
const s = String(v).trim().toLowerCase();
|
||||
return s !== '' && s !== '0' && s !== 'false' && s !== 'no' && s !== 'off';
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve whether the `disableBundledSkills` lever is effectively ON, reading the
|
||||
* env var and the settings cascade directly (user ~/.claude, then project, then
|
||||
* project-local).
|
||||
*
|
||||
* Reads the files directly rather than relying on config-discovery
|
||||
* classification: when discovery walks ~/.claude from the .claude root, the
|
||||
* user settings.json has a relPath of "settings.json" (no ".claude" segment)
|
||||
* and is NOT tagged as settings-json — so the dominant user-scope location for
|
||||
* this global preference would otherwise be missed. HOME-scoped via
|
||||
* process.env.HOME.
|
||||
*
|
||||
* @param {string} [projectPath] - project root, to also read project + local settings
|
||||
* @returns {Promise<boolean>}
|
||||
*/
|
||||
export async function isBundledSkillsDisabled(projectPath) {
|
||||
if (envFlag(process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS)) return true;
|
||||
|
||||
const home = process.env.HOME || process.env.USERPROFILE || '';
|
||||
const candidates = [];
|
||||
if (home) candidates.push(join(home, '.claude', 'settings.json'));
|
||||
if (projectPath) {
|
||||
candidates.push(join(projectPath, '.claude', 'settings.json'));
|
||||
candidates.push(join(projectPath, '.claude', 'settings.local.json'));
|
||||
}
|
||||
|
||||
for (const p of candidates) {
|
||||
const content = await readTextFile(p);
|
||||
if (!content) continue;
|
||||
const parsed = parseJson(content);
|
||||
if (parsed && parsed.disableBundledSkills === true) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,12 @@
|
|||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { join } from 'node:path';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import {
|
||||
assessSkillListingBudget,
|
||||
envFlag,
|
||||
isBundledSkillsDisabled,
|
||||
DESCRIPTION_CAP,
|
||||
AGGREGATE_BUDGET_TOKENS,
|
||||
CONTEXT_WINDOW_ANCHOR,
|
||||
|
|
@ -98,3 +103,79 @@ describe('assessSkillListingBudget — aggregate math', () => {
|
|||
assert.equal(r.scanned, 3);
|
||||
});
|
||||
});
|
||||
|
||||
describe('envFlag', () => {
|
||||
it('treats 1/true/yes/on as set', () => {
|
||||
for (const v of ['1', 'true', 'TRUE', 'yes', 'on', ' 1 ']) {
|
||||
assert.equal(envFlag(v), true, `expected ${JSON.stringify(v)} → true`);
|
||||
}
|
||||
});
|
||||
|
||||
it('treats null/empty/0/false/no/off as un-set', () => {
|
||||
for (const v of [undefined, null, '', '0', 'false', 'no', 'off', ' ']) {
|
||||
assert.equal(envFlag(v), false, `expected ${JSON.stringify(v)} → false`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('isBundledSkillsDisabled — lever cascade', () => {
|
||||
async function withHome(fn) {
|
||||
const home = await mkdtemp(join(tmpdir(), 'config-audit-lever-home-'));
|
||||
const originalHome = process.env.HOME;
|
||||
const originalEnv = process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
process.env.HOME = home;
|
||||
delete process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
try {
|
||||
return await fn(home);
|
||||
} finally {
|
||||
process.env.HOME = originalHome;
|
||||
if (originalEnv === undefined) delete process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
else process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS = originalEnv;
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function writeSettings(dir, obj) {
|
||||
await mkdir(dir, { recursive: true });
|
||||
await writeFile(join(dir, 'settings.json'), JSON.stringify(obj));
|
||||
}
|
||||
|
||||
it('is false on a clean HOME with no env and no settings', async () => {
|
||||
await withHome(async () => {
|
||||
assert.equal(await isBundledSkillsDisabled(), false);
|
||||
});
|
||||
});
|
||||
|
||||
it('is true when the env var is set', async () => {
|
||||
await withHome(async () => {
|
||||
process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS = '1';
|
||||
assert.equal(await isBundledSkillsDisabled(), true);
|
||||
});
|
||||
});
|
||||
|
||||
it('is true when user ~/.claude/settings.json sets it (the location discovery misses)', async () => {
|
||||
await withHome(async (home) => {
|
||||
await writeSettings(join(home, '.claude'), { disableBundledSkills: true });
|
||||
assert.equal(await isBundledSkillsDisabled(), true);
|
||||
});
|
||||
});
|
||||
|
||||
it('is true when project .claude/settings.json sets it', async () => {
|
||||
await withHome(async () => {
|
||||
const project = await mkdtemp(join(tmpdir(), 'config-audit-lever-proj-'));
|
||||
try {
|
||||
await writeSettings(join(project, '.claude'), { disableBundledSkills: true });
|
||||
assert.equal(await isBundledSkillsDisabled(project), true);
|
||||
} finally {
|
||||
await rm(project, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
it('is false when the setting is present but not strictly true', async () => {
|
||||
await withHome(async (home) => {
|
||||
await writeSettings(join(home, '.claude'), { disableBundledSkills: false });
|
||||
assert.equal(await isBundledSkillsDisabled(), false);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,10 +1,13 @@
|
|||
import { describe, it, beforeEach } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { resolve } from 'node:path';
|
||||
import { resolve, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan, opportunitySummary } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
const FIXTURES = resolve(__dirname, '../fixtures');
|
||||
|
|
@ -20,7 +23,10 @@ describe('GAP scanner — healthy project', () => {
|
|||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('healthy-project');
|
||||
result = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
// Hermetic HOME: scan() now enumerates active skills via process.env.HOME
|
||||
// (the disableBundledSkills lever check). An empty HOME keeps these fixture
|
||||
// tests environment-independent — no skills, no budget pressure, no lever finding.
|
||||
result = await withHermeticHome(() => scan(resolve(FIXTURES, 'healthy-project'), discovery));
|
||||
});
|
||||
|
||||
it('returns status ok', () => {
|
||||
|
|
@ -91,7 +97,7 @@ describe('GAP scanner — minimal project', () => {
|
|||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('minimal-project');
|
||||
result = await scan(resolve(FIXTURES, 'minimal-project'), discovery);
|
||||
result = await withHermeticHome(() => scan(resolve(FIXTURES, 'minimal-project'), discovery));
|
||||
});
|
||||
|
||||
it('returns status ok', () => {
|
||||
|
|
@ -125,7 +131,7 @@ describe('GAP scanner — minimal project', () => {
|
|||
it('has more findings than healthy project', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('healthy-project');
|
||||
const healthyResult = await scan(resolve(FIXTURES, 'healthy-project'), discovery);
|
||||
const healthyResult = await withHermeticHome(() => scan(resolve(FIXTURES, 'healthy-project'), discovery));
|
||||
assert.ok(result.findings.length > healthyResult.findings.length);
|
||||
});
|
||||
});
|
||||
|
|
@ -135,7 +141,7 @@ describe('GAP scanner — empty project', () => {
|
|||
beforeEach(async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('empty-project');
|
||||
result = await scan(resolve(FIXTURES, 'empty-project'), discovery);
|
||||
result = await withHermeticHome(() => scan(resolve(FIXTURES, 'empty-project'), discovery));
|
||||
});
|
||||
|
||||
it('returns status ok (never skips)', () => {
|
||||
|
|
@ -206,3 +212,131 @@ describe('opportunitySummary', () => {
|
|||
assert.equal(result.explore.length, 2);
|
||||
});
|
||||
});
|
||||
|
||||
// ── disableBundledSkills lever (CA-GAP) — remediation companion to SKL CA-SKL-002 ──
|
||||
|
||||
describe('bundledSkillsLeverFinding — pure decision', () => {
|
||||
const overBudget = { scanned: 17, aggregateChars: 17000, aggregateTokens: 4250, budgetTokens: 4000, overBudget: true, overBy: 250 };
|
||||
const underBudget = { scanned: 3, aggregateChars: 3000, aggregateTokens: 750, budgetTokens: 4000, overBudget: false, overBy: 0 };
|
||||
|
||||
it('returns null when the lever is already pulled, even over budget', () => {
|
||||
assert.equal(bundledSkillsLeverFinding({ leverPulled: true, aggregate: overBudget }), null);
|
||||
});
|
||||
|
||||
it('returns null when the listing is under budget', () => {
|
||||
assert.equal(bundledSkillsLeverFinding({ leverPulled: false, aggregate: underBudget }), null);
|
||||
});
|
||||
|
||||
it('returns a finding when the lever is un-pulled AND the listing is over budget', () => {
|
||||
resetCounter();
|
||||
const f = bundledSkillsLeverFinding({ leverPulled: false, aggregate: overBudget });
|
||||
assert.ok(f, 'expected a finding');
|
||||
assert.match(f.id, /^CA-GAP-\d{3}$/);
|
||||
assert.equal(f.scanner, 'GAP');
|
||||
assert.equal(f.severity, 'low');
|
||||
assert.equal(f.category, 'token-efficiency');
|
||||
assert.match(f.recommendation, /disableBundledSkills/);
|
||||
assert.match(`${f.description} ${f.recommendation}`, /2\.1\.169/);
|
||||
});
|
||||
|
||||
it('handles a null/garbage aggregate without throwing', () => {
|
||||
assert.equal(bundledSkillsLeverFinding({ leverPulled: false, aggregate: null }), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GAP scanner — disableBundledSkills lever wiring (HOME-scoped)', () => {
|
||||
async function buildHome(count, descLen, settings) {
|
||||
const home = await mkdtemp(join(tmpdir(), 'config-audit-gap-home-'));
|
||||
for (let i = 0; i < count; i++) {
|
||||
const dir = join(home, '.claude', 'skills', `s${i}`);
|
||||
await mkdir(dir, { recursive: true });
|
||||
await writeFile(join(dir, 'SKILL.md'), `---\nname: s${i}\ndescription: ${'a'.repeat(descLen)}\n---\nBody.\n`);
|
||||
}
|
||||
if (settings) {
|
||||
await mkdir(join(home, '.claude'), { recursive: true });
|
||||
await writeFile(join(home, '.claude', 'settings.json'), JSON.stringify(settings));
|
||||
}
|
||||
return home;
|
||||
}
|
||||
|
||||
/** Run the GAP scanner with HOME pointed at `home`; an empty throwaway project. */
|
||||
async function runGapWithHome(home) {
|
||||
const project = await mkdtemp(join(tmpdir(), 'config-audit-gap-proj-'));
|
||||
const original = process.env.HOME;
|
||||
process.env.HOME = home;
|
||||
try {
|
||||
resetCounter();
|
||||
const discovery = await discoverConfigFiles(project, { includeGlobal: true });
|
||||
const result = await scan(project, discovery);
|
||||
return result;
|
||||
} finally {
|
||||
process.env.HOME = original;
|
||||
await rm(project, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
const hasLever = (result) =>
|
||||
result.findings.some(f => f.scanner === 'GAP' && /disableBundledSkills/.test(f.recommendation || ''));
|
||||
|
||||
it('fires when the listing is over budget and the lever is un-pulled', async () => {
|
||||
const home = await buildHome(17, 1000); // 17000 chars -> 4250 tok > 4000 budget
|
||||
try {
|
||||
const result = await runGapWithHome(home);
|
||||
const f = result.findings.find(x => x.scanner === 'GAP' && /disableBundledSkills/.test(x.recommendation || ''));
|
||||
assert.ok(f, `expected a disableBundledSkills finding; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
assert.equal(f.severity, 'low');
|
||||
assert.equal(f.category, 'token-efficiency');
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('does NOT fire when the listing is under budget', async () => {
|
||||
const home = await buildHome(3, 1000); // 3000 chars -> 750 tok < 4000
|
||||
try {
|
||||
const result = await runGapWithHome(home);
|
||||
assert.equal(hasLever(result), false);
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('does NOT fire when disableBundledSkills is already true in settings', async () => {
|
||||
const home = await buildHome(17, 1000, { disableBundledSkills: true });
|
||||
try {
|
||||
const result = await runGapWithHome(home);
|
||||
assert.equal(hasLever(result), false,
|
||||
'lever already pulled in settings — should not recommend it again');
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('does NOT fire when the CLAUDE_CODE_DISABLE_BUNDLED_SKILLS env var is set', async () => {
|
||||
const home = await buildHome(17, 1000);
|
||||
const originalEnv = process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS = '1';
|
||||
try {
|
||||
const result = await runGapWithHome(home);
|
||||
assert.equal(hasLever(result), false, 'env lever should suppress the recommendation');
|
||||
} finally {
|
||||
if (originalEnv === undefined) delete process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
else process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS = originalEnv;
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('treats CLAUDE_CODE_DISABLE_BUNDLED_SKILLS=0 as un-pulled (still fires)', async () => {
|
||||
const home = await buildHome(17, 1000);
|
||||
const originalEnv = process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS = '0';
|
||||
try {
|
||||
const result = await runGapWithHome(home);
|
||||
assert.equal(hasLever(result), true, '"0" means off — the lever is NOT pulled, so the finding should fire');
|
||||
} finally {
|
||||
if (originalEnv === undefined) delete process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS;
|
||||
else process.env.CLAUDE_CODE_DISABLE_BUNDLED_SKILLS = originalEnv;
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue