fix(scanners): retire the autoMode GAP dimension, a /doctor duplicate (D1)
CC 2.1.226's /doctor Check 8 covers auto mode with usage-weighted judgement. The binding positioning forbids carrying a feature whose whole value is duplicating a /doctor check, so the "adopt this feature" nudge goes. The deterministic side stays: SET still validates autoMode structure and still flags it as dead config in shared project settings. GAP dimensions 25 -> 24. The title lived in FOUR tables, not the two the removal was scoped against: the dimension list, scoring TITLE_TO_ID, the humanizer's static translations, and the scoring denominators (TIER_COUNTS t3 8->7, TOTAL_DIMENSIONS 25->24, MAX_WEIGHTED 42->41) -- the one that moves a user-visible number. findGapId falls back to 'unknown' silently, so a partial removal would have degraded without failing. A blanket sync invariant now asserts all four against GAP_CHECKS instead of comparing occurrences pairwise; each arm was verified red against its own defect (denominator drift, orphaned humanizer entry, resurrected dimension). Frozen tests/snapshots/v5.0.0/ stays untouched. strip-retired-gap.mjs is the removal twin of strip-added-scanner.mjs: it strips the retired dimension from whichever side still carries it and re-derives GAP IDs, since retiring a dimension from mid-list shifts every later ID by one. Derived utilization figures are dropped from comparison rather than recomputed -- recomputing them in a test helper would assert the new arithmetic against itself, and scoring.test.mjs already pins them exactly. Re-seeding was rejected: it would silently bake in any other drift across every scanner those four files cover. risk_score, risk_band, verdict, overallGrade, maturity and segment are byte-identical across the change (severity info carries zero risk weight; GAP is excluded from the overall grade). Utilization shifts 43 -> 44 on the fixture. D2 (CA-SKL-002) is NOT removed. Verified against the primary source first: the CC changelog carries exactly one budget-fraction statement (L3786, 2.1.32) and nothing supersedes it, so our 2% is current and 002 is not a duplicate with a stale figure. /doctor's ~1% could not be reconciled from the changelog and it discloses its own numbers as disk estimates, so it is recorded, not adopted. Left explicitly unverified in a code note: L3786 says "character budget" while we express tokens -- a 4x difference nobody can settle from the wording. Suite 1531 -> 1535, all green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsfPGxgwbR3MY54wDC6hat
This commit is contained in:
parent
182a37c1af
commit
4027cdcf54
17 changed files with 368 additions and 75 deletions
203
tests/helpers/strip-retired-gap.mjs
Normal file
203
tests/helpers/strip-retired-gap.mjs
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
/**
|
||||
* The REMOVAL twin of strip-added-scanner.mjs.
|
||||
*
|
||||
* The frozen v5.0.0 baselines predate v5.14's retirement of the GAP dimension
|
||||
* `No autoMode classifier` (a pure /doctor Check 8 duplicate under the binding
|
||||
* positioning). Where strip-added-scanner.mjs removes post-v5.0.0 ADDITIONS
|
||||
* from live output so it matches the frozen expectation, this removes a
|
||||
* RETIRED dimension from whichever side still carries it — the frozen
|
||||
* expectation. Both sides run through the same normalizer, so every operation
|
||||
* here must be a no-op on live output; that constraint is what dictates the
|
||||
* three shapes below.
|
||||
*
|
||||
* Why strip rather than re-seed: re-seeding these four baselines would bake in
|
||||
* whatever OTHER drift has accumulated since v5.0.0 across every scanner they
|
||||
* cover, silently. Stripping keeps the strongest invariant — every byte the
|
||||
* original scanners emit is still compared — and confines the exception to the
|
||||
* one dimension we deliberately deleted.
|
||||
*
|
||||
* Three payload shapes carry GAP findings, all derived from one scan and all
|
||||
* therefore in emission order:
|
||||
* - envelopes (scan-orchestrator, posture) — `scanners[GAP].findings`
|
||||
* - drift — flat `newFindings`/`resolvedFindings`/`unchangedFindings`/`movedFindings`
|
||||
* - fix-cli — `manual[]` entries keyed `findingId`
|
||||
* A single traversal handles all three rather than three special cases.
|
||||
*
|
||||
* What is adjusted, and what is deliberately NOT:
|
||||
* - the retired finding is dropped, and GAP IDs are re-derived densely in
|
||||
* traversal order. GAP IDs are already position-derived (`output.mjs`
|
||||
* assigns them from an emission counter), so this re-derivation applies the
|
||||
* scanner's own rule, not a new one. Retiring a dimension from the middle
|
||||
* of the list shifts every later ID by one.
|
||||
* - counter bookkeeping is decremented mechanically, exactly as
|
||||
* strip-added-scanner.mjs decrements `scanners_ok`.
|
||||
* - the DERIVED utilization figures (`utilization.score`/`overhang` and the
|
||||
* feature_coverage area `score`) are dropped from comparison instead of
|
||||
* recomputed. Recomputing them inside a test helper would assert the new
|
||||
* arithmetic against itself. They are covered exhaustively by dedicated
|
||||
* unit tests in tests/lib/scoring.test.mjs (exact expected values per
|
||||
* tier), which is the right place for arithmetic.
|
||||
* - risk_score, risk_band, verdict, overallGrade, maturity, segment and
|
||||
* drift's scoreChange are left alone: the retired dimension was
|
||||
* severity `info`, which carries zero risk weight, and GAP is excluded from
|
||||
* the overall grade (scoring.mjs). Measured 2026-08-09: all byte-identical
|
||||
* across the change.
|
||||
*
|
||||
* Mutates in place and returns the payload for chaining inside a normalizer.
|
||||
*/
|
||||
|
||||
const RETIRED_GAP_TITLES = new Set(['No autoMode classifier']);
|
||||
|
||||
const GAP_ID_RE = /^CA-GAP-\d+$/;
|
||||
|
||||
/** Decrement a numeric field by n when it exists. */
|
||||
function decrement(obj, key, n) {
|
||||
if (obj && typeof obj[key] === 'number') obj[key] -= n;
|
||||
}
|
||||
|
||||
/**
|
||||
* True for a FINDING-shaped object belonging to the GAP scanner.
|
||||
*
|
||||
* The `title` requirement is load-bearing: the scanner envelope entry also
|
||||
* carries `scanner: 'GAP'`, and counting it as a finding shifts every
|
||||
* re-derived ID by one.
|
||||
*/
|
||||
function isGapEntry(o) {
|
||||
if (!o || typeof o !== 'object' || typeof o.title !== 'string') return false;
|
||||
if (o.scanner === 'GAP') return true;
|
||||
return GAP_ID_RE.test(o.id || '') || GAP_ID_RE.test(o.findingId || '');
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk the payload, removing retired GAP entries from every array and
|
||||
* collecting the survivors in traversal (= emission) order.
|
||||
*/
|
||||
function walk(node, state) {
|
||||
if (Array.isArray(node)) {
|
||||
for (let i = node.length - 1; i >= 0; i--) {
|
||||
const item = node[i];
|
||||
if (isGapEntry(item) && RETIRED_GAP_TITLES.has(item.title)) {
|
||||
state.removed++;
|
||||
if (item.severity) {
|
||||
state.bySeverity[item.severity] = (state.bySeverity[item.severity] || 0) + 1;
|
||||
}
|
||||
node.splice(i, 1);
|
||||
}
|
||||
}
|
||||
for (const item of node) {
|
||||
if (isGapEntry(item)) state.survivors.push(item);
|
||||
walk(item, state);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (node && typeof node === 'object') {
|
||||
for (const v of Object.values(node)) walk(v, state);
|
||||
}
|
||||
}
|
||||
|
||||
/** Decrement every counter location a retired GAP finding contributes to. */
|
||||
function adjustCounters(payload, state) {
|
||||
const { removed, bySeverity } = state;
|
||||
|
||||
const envelopes = [payload, payload.scannerEnvelope].filter(
|
||||
e => e && typeof e === 'object',
|
||||
);
|
||||
for (const env of envelopes) {
|
||||
if (Array.isArray(env.scanners)) {
|
||||
for (const s of env.scanners) {
|
||||
if (s && s.scanner === 'GAP') {
|
||||
for (const [sev, n] of Object.entries(bySeverity)) decrement(s.counts, sev, n);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (env.aggregate) {
|
||||
decrement(env.aggregate, 'total_findings', removed);
|
||||
for (const [sev, n] of Object.entries(bySeverity)) decrement(env.aggregate.counts, sev, n);
|
||||
}
|
||||
}
|
||||
|
||||
// Posture-shaped roll-ups above the envelope.
|
||||
decrement(payload, 'opportunityCount', removed);
|
||||
if (Array.isArray(payload.areas)) {
|
||||
for (const a of payload.areas) {
|
||||
if (a && a.id === 'feature_coverage') decrement(a, 'findingCount', removed);
|
||||
}
|
||||
}
|
||||
|
||||
// Drift totals.
|
||||
if (payload.summary) {
|
||||
decrement(payload.summary, 'totalBefore', removed);
|
||||
decrement(payload.summary, 'totalAfter', removed);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip retired GAP dimensions from any CLI payload and drop the derived
|
||||
* utilization figures from comparison.
|
||||
* @template T
|
||||
* @param {T} payload
|
||||
* @returns {T}
|
||||
*/
|
||||
export function stripRetiredGap(payload) {
|
||||
if (!payload || typeof payload !== 'object') return payload;
|
||||
|
||||
const state = { removed: 0, bySeverity: {}, survivors: [] };
|
||||
walk(payload, state);
|
||||
|
||||
if (state.removed > 0) {
|
||||
// Re-derive GAP IDs densely — the same position rule output.mjs applies.
|
||||
state.survivors.forEach((f, i) => {
|
||||
const id = `CA-GAP-${String(i + 1).padStart(3, '0')}`;
|
||||
if ('id' in f) f.id = id;
|
||||
if ('findingId' in f) f.findingId = id;
|
||||
});
|
||||
adjustCounters(payload, state);
|
||||
}
|
||||
|
||||
// Derived scores: dropped on BOTH sides unconditionally, so live and frozen
|
||||
// stay symmetric. See the header note.
|
||||
if (payload.utilization && typeof payload.utilization === 'object') {
|
||||
delete payload.utilization.score;
|
||||
delete payload.utilization.overhang;
|
||||
}
|
||||
if (Array.isArray(payload.areas)) {
|
||||
for (const a of payload.areas) {
|
||||
if (a && a.id === 'feature_coverage') delete a.score;
|
||||
}
|
||||
}
|
||||
// Drift reports the same derived score under a different key shape.
|
||||
if (Array.isArray(payload.areaChanges)) {
|
||||
for (const a of payload.areaChanges) {
|
||||
if (a && a.name === 'Feature Coverage') {
|
||||
delete a.before?.score;
|
||||
delete a.after?.score;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mask the GAP finding tally in a captured stderr scorecard.
|
||||
*
|
||||
* The scorecard prints only a count, with nothing in the text identifying WHICH
|
||||
* dimensions it covers — so unlike the line-removal in stripAddedScannerStderr,
|
||||
* a decrement here could not be a no-op on the live side. Both sides go through
|
||||
* the same normalizer, so the tally is masked rather than adjusted; the exact
|
||||
* GAP count is asserted on the structured payload, where the findings
|
||||
* themselves are compared.
|
||||
* @param {string} text
|
||||
* @returns {string}
|
||||
*/
|
||||
export function maskGapTallyStderr(text) {
|
||||
if (typeof text !== 'string') return text;
|
||||
return text
|
||||
.replace(
|
||||
/^(\s*`?\[GAP\][^\n]*?)\d+ finding\(s\)/gm,
|
||||
(_m, head) => `${head}<GAPCOUNT> finding(s)`,
|
||||
)
|
||||
// The scorecard footer repeats the tally, in a raw and a humanized wording.
|
||||
.replace(/^(\s*)\d+( opportunities available)/gm, '$1<GAPCOUNT>$2')
|
||||
.replace(/^(\s*)\d+( ways you could get more out of)/gm, '$1<GAPCOUNT>$2');
|
||||
}
|
||||
|
|
@ -31,6 +31,7 @@ import { homedir } from 'node:os';
|
|||
import { hermeticEnv, HERMETIC_HOME } from './helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from './helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from './helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap } from './helpers/strip-retired-gap.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -104,7 +105,7 @@ function normalizeScanOrchestrator(env) {
|
|||
}
|
||||
}
|
||||
stripAncestorDerived(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
function normalizePosture(p) {
|
||||
|
|
@ -121,7 +122,7 @@ function normalizePosture(p) {
|
|||
}
|
||||
stripAncestorDerived(out.scannerEnvelope);
|
||||
}
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
function normalizeTokenHotspots(p) {
|
||||
|
|
@ -132,12 +133,12 @@ function normalizeTokenHotspots(p) {
|
|||
|
||||
function normalizeDrift(p) {
|
||||
// Drift result has no time fields — round-trip through JSON for safety.
|
||||
return JSON.parse(JSON.stringify(p));
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
}
|
||||
|
||||
function normalizeFix(p) {
|
||||
// Fix-cli stdout is the planFixes result with no time fields.
|
||||
return JSON.parse(JSON.stringify(p));
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
}
|
||||
|
||||
function normalizePluginHealth(p) {
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ describe('calculateUtilization', () => {
|
|||
assert.equal(result.overhang, 0);
|
||||
});
|
||||
|
||||
it('returns 0% when all 25 dimensions are gaps', () => {
|
||||
it('returns 0% when all 24 dimensions are gaps', () => {
|
||||
const result = calculateUtilization(allGapFindings());
|
||||
assert.equal(result.score, 0);
|
||||
assert.equal(result.overhang, 100);
|
||||
|
|
@ -101,24 +101,24 @@ describe('calculateUtilization', () => {
|
|||
it('weighs T1 gaps heavier (3x)', () => {
|
||||
const onlyT1 = t1GapFindings(); // 5 T1 gaps = 15 weight lost
|
||||
const result = calculateUtilization(onlyT1);
|
||||
// Lost: 5 × 3 = 15 out of 42. Present: 27/42 = 64%
|
||||
assert.equal(result.score, 64);
|
||||
// Lost: 5 × 3 = 15 out of 41. Present: 26/41 = 63%
|
||||
assert.equal(result.score, 63);
|
||||
});
|
||||
|
||||
it('weighs T4 gaps lighter (1x)', () => {
|
||||
const onlyT4 = t4GapFindings(); // 5 T4 gaps = 5 weight lost
|
||||
const result = calculateUtilization(onlyT4);
|
||||
// Lost: 5 × 1 = 5 out of 42. Present: 37/42 = 88%
|
||||
// Lost: 5 × 1 = 5 out of 41. Present: 36/41 = 88%
|
||||
assert.equal(result.score, 88);
|
||||
});
|
||||
|
||||
it('T1+T2 present but no T3+T4 scores ~69%', () => {
|
||||
// T3: 8 dims × 1 = 8, T4: 5 dims × 1 = 5. Lost = 13 out of 42. Present = 29/42 = 69%
|
||||
it('T1+T2 present but no T3+T4 scores ~71%', () => {
|
||||
// T3: 7 dims × 1 = 7, T4: 5 dims × 1 = 5. Lost = 12 out of 41. Present = 29/41 = 71%
|
||||
const t3t4Gaps = Object.entries(TITLE_TO_ID)
|
||||
.filter(([, id]) => id.startsWith('t3') || id.startsWith('t4'))
|
||||
.map(([title, id]) => makeGapFinding(title, id.split('_')[0]));
|
||||
const result = calculateUtilization(t3t4Gaps);
|
||||
assert.equal(result.score, 69);
|
||||
assert.equal(result.score, 71);
|
||||
});
|
||||
|
||||
it('score + overhang = 100', () => {
|
||||
|
|
@ -581,17 +581,17 @@ describe('generateHealthScorecard', () => {
|
|||
// Constants and exports
|
||||
// ========================================
|
||||
describe('scoring constants', () => {
|
||||
it('TITLE_TO_ID has 25 entries', () => {
|
||||
assert.equal(Object.keys(TITLE_TO_ID).length, 25);
|
||||
it('TITLE_TO_ID has 24 entries', () => {
|
||||
assert.equal(Object.keys(TITLE_TO_ID).length, 24);
|
||||
});
|
||||
|
||||
it('TIER_COUNTS sum to 25', () => {
|
||||
it('TIER_COUNTS sum to 24', () => {
|
||||
const sum = Object.values(TIER_COUNTS).reduce((a, b) => a + b, 0);
|
||||
assert.equal(sum, 25);
|
||||
assert.equal(sum, 24);
|
||||
});
|
||||
|
||||
it('MAX_WEIGHTED is 42', () => {
|
||||
assert.equal(MAX_WEIGHTED, 42);
|
||||
it('MAX_WEIGHTED is 41', () => {
|
||||
assert.equal(MAX_WEIGHTED, 41);
|
||||
});
|
||||
|
||||
it('TIER_WEIGHTS match spec', () => {
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ import { homedir } from 'node:os';
|
|||
import { hermeticEnv, HERMETIC_HOME } from './helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from './helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from './helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap, maskGapTallyStderr } from './helpers/strip-retired-gap.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -99,7 +100,7 @@ function normalizeScanOrchestrator(env) {
|
|||
}
|
||||
}
|
||||
stripAncestorDerived(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
function normalizePosture(p) {
|
||||
|
|
@ -116,7 +117,7 @@ function normalizePosture(p) {
|
|||
}
|
||||
stripAncestorDerived(out.scannerEnvelope);
|
||||
}
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
function normalizeTokenHotspots(p) {
|
||||
|
|
@ -126,11 +127,11 @@ function normalizeTokenHotspots(p) {
|
|||
}
|
||||
|
||||
function normalizeDrift(p) {
|
||||
return JSON.parse(JSON.stringify(p));
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
}
|
||||
|
||||
function normalizeFix(p) {
|
||||
return JSON.parse(JSON.stringify(p));
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(p)));
|
||||
}
|
||||
|
||||
function normalizePluginHealth(p) {
|
||||
|
|
@ -276,8 +277,8 @@ describe('SC-7 --raw posture stderr scorecard verbatim', () => {
|
|||
}
|
||||
const expected = await readFile(stderrSnapshotPath, 'utf8');
|
||||
assert.equal(
|
||||
normalizeStderrDurations(stripAddedScannerStderr(stderr.trim())),
|
||||
normalizeStderrDurations(stripAddedScannerStderr(expected.trim())),
|
||||
normalizeStderrDurations(maskGapTallyStderr(stripAddedScannerStderr(stderr.trim()))),
|
||||
normalizeStderrDurations(maskGapTallyStderr(stripAddedScannerStderr(expected.trim()))),
|
||||
'posture --raw stderr must reproduce the v5.0.0 scorecard verbatim (apart from durations)',
|
||||
);
|
||||
});
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ import { promisify } from 'node:util';
|
|||
import { readFile, writeFile, unlink, mkdir, access } from 'node:fs/promises';
|
||||
import { hermeticEnv, HERMETIC_HOME } from '../helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripRetiredGap } from '../helpers/strip-retired-gap.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -43,7 +44,7 @@ async function runCli(cliPath, args, env = {}) {
|
|||
function normalizeTokenHotspotsPayload(p) {
|
||||
const out = JSON.parse(JSON.stringify(p));
|
||||
out.duration_ms = 0;
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripRetiredGap(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizeManifestOutput(o) {
|
||||
|
|
@ -76,7 +77,7 @@ function normalizePluginHealthOutput(o) {
|
|||
|
||||
function normalizeDriftOutput(o) {
|
||||
// Drift result has no time fields; just round-trip through JSON.
|
||||
return JSON.parse(JSON.stringify(o));
|
||||
return stripRetiredGap(JSON.parse(JSON.stringify(o)));
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
|
|
@ -318,14 +319,14 @@ describe('fix-cli humanizer (Step 7)', () => {
|
|||
const { stdout } = await runCli(CLI, [FIXTURE, '--json']);
|
||||
const actual = JSON.parse(stdout);
|
||||
const expected = JSON.parse(await readFile(SNAPSHOT, 'utf-8'));
|
||||
assert.deepStrictEqual(actual, expected);
|
||||
assert.deepStrictEqual(stripRetiredGap(actual), stripRetiredGap(expected));
|
||||
});
|
||||
|
||||
it('--raw: stdout JSON byte-equal v5.0.0 snapshot', async () => {
|
||||
const { stdout } = await runCli(CLI, [FIXTURE, '--raw']);
|
||||
const actual = JSON.parse(stdout);
|
||||
const expected = JSON.parse(await readFile(SNAPSHOT, 'utf-8'));
|
||||
assert.deepStrictEqual(actual, expected);
|
||||
assert.deepStrictEqual(stripRetiredGap(actual), stripRetiredGap(expected));
|
||||
});
|
||||
|
||||
it('default mode stderr differs from --raw stderr when findings have humanizer translations', async () => {
|
||||
|
|
|
|||
|
|
@ -5,7 +5,9 @@ import { fileURLToPath } from 'node:url';
|
|||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding, filterHookLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding, filterHookLeverFinding, GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { TITLE_TO_ID as GAP_TITLE_TO_ID, TIER_COUNTS, TOTAL_DIMENSIONS } from '../../scanners/lib/scoring.mjs';
|
||||
import { TRANSLATIONS } from '../../scanners/lib/humanizer-data.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
|
|
@ -506,3 +508,63 @@ describe('GAP scanner — test/demo data must not mask real gaps (M-BUG-13)', ()
|
|||
);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* D1 — retired GAP dimensions.
|
||||
*
|
||||
* The binding /doctor positioning (README «config-audit vs. the built-in
|
||||
* /doctor») forbids carrying a dimension whose whole value is duplicating a
|
||||
* /doctor check. `No autoMode classifier` was retired when CC 2.1.226's
|
||||
* /doctor Check 8 was measured to cover auto mode with usage-weighted
|
||||
* judgement (re-measured 2026-08-09; the deterministic SET `autoMode`
|
||||
* structure/dead-config validation is NOT a duplicate and stays).
|
||||
*
|
||||
* The title lived in THREE tables, not two: the dimension list, the scoring
|
||||
* title→id map, and the humanizer's static translations. `findGapId` falls
|
||||
* back to 'unknown' silently, so a partial removal degrades without failing —
|
||||
* hence the blanket sync invariant below rather than three separate absence
|
||||
* checks.
|
||||
*/
|
||||
describe('GAP scanner — retired dimensions (D1)', () => {
|
||||
const RETIRED_TITLES = ['No autoMode classifier'];
|
||||
|
||||
it('emits no retired dimension on a fixture that lacks the feature', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('healthy-project');
|
||||
const result = await withHermeticHome(
|
||||
() => scan(resolve(FIXTURES, 'healthy-project'), discovery),
|
||||
);
|
||||
const titles = result.findings.map(f => f.title);
|
||||
for (const t of RETIRED_TITLES) {
|
||||
assert.ok(!titles.includes(t), `retired dimension still emitted: ${t}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('leaves no orphaned entry in any of the three tables', () => {
|
||||
for (const t of RETIRED_TITLES) {
|
||||
assert.ok(!GAP_CHECKS.some(g => g.title === t), `still in GAP_CHECKS: ${t}`);
|
||||
assert.ok(!(t in GAP_TITLE_TO_ID), `orphan in scoring TITLE_TO_ID: ${t}`);
|
||||
assert.ok(!(t in TRANSLATIONS.GAP.static), `orphan in humanizer TRANSLATIONS.GAP: ${t}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('keeps the three title tables carrying exactly the same titles', () => {
|
||||
const checks = GAP_CHECKS.map(g => g.title).sort();
|
||||
const scoring = Object.keys(GAP_TITLE_TO_ID).sort();
|
||||
const humanizer = Object.keys(TRANSLATIONS.GAP.static).sort();
|
||||
assert.deepEqual(scoring, checks, 'scoring TITLE_TO_ID drifted from GAP_CHECKS');
|
||||
assert.deepEqual(humanizer, checks, 'humanizer TRANSLATIONS.GAP drifted from GAP_CHECKS');
|
||||
});
|
||||
|
||||
// The scoring denominators are a FOURTH copy of the dimension inventory, and
|
||||
// the one that silently moves every user's utilization score: the score is
|
||||
// presentWeight / MAX_WEIGHTED, derived from TIER_COUNTS. Adding or retiring a
|
||||
// dimension without updating these skews the score instead of failing.
|
||||
it('keeps the scoring denominators in step with the dimension inventory', () => {
|
||||
const perTier = { t1: 0, t2: 0, t3: 0, t4: 0 };
|
||||
for (const g of GAP_CHECKS) perTier[g.tier]++;
|
||||
assert.deepEqual(TIER_COUNTS, perTier, 'TIER_COUNTS drifted from GAP_CHECKS tiers');
|
||||
assert.equal(TOTAL_DIMENSIONS, GAP_CHECKS.length,
|
||||
'TOTAL_DIMENSIONS drifted from GAP_CHECKS length');
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import { readFile, unlink } from 'node:fs/promises';
|
|||
import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from '../helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap, maskGapTallyStderr } from '../helpers/strip-retired-gap.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -41,7 +42,7 @@ function normalizePosture(p) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -50,7 +51,7 @@ function normalizePosture(p) {
|
|||
* run matches the frozen 13-scanner v5.0.0 stderr scorecard.
|
||||
*/
|
||||
function normalizeStderr(s) {
|
||||
return stripAddedScannerStderr(s).replace(/\(\d+ms\)/g, '(0ms)');
|
||||
return maskGapTallyStderr(stripAddedScannerStderr(s)).replace(/\(\d+ms\)/g, '(0ms)');
|
||||
}
|
||||
|
||||
async function runPosture(flags) {
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import { readFile, unlink } from 'node:fs/promises';
|
|||
import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners } from '../helpers/strip-added-scanner.mjs';
|
||||
import { stripRetiredGap } from '../helpers/strip-retired-gap.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -38,7 +39,7 @@ function normalizeEnvelope(env) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
return stripRetiredGap(stripAddedScanners(stripHotspotLoadPattern(out)));
|
||||
}
|
||||
|
||||
async function runOrchestrator(flags) {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{
|
||||
"kind": "text",
|
||||
"payload": "`[CML] CLAUDE.md Linter`: 1 finding(s) (0ms)\n `[SET] Settings Validator`: 0 finding(s) (0ms)\n `[HKV] Hook Validator`: 0 finding(s) (0ms)\n `[RUL] Rules Validator`: 0 finding(s) (0ms)\n `[MCP] MCP Config Validator`: 0 finding(s) (0ms)\n `[IMP] Import Resolver`: 0 finding(s) (0ms)\n `[CNF] Conflict Detector`: 0 finding(s) (0ms)\n `[GAP] Feature Gap Scanner`: 17 finding(s) (0ms)\n `[TOK] Token Hotspots`: 1 finding(s) (0ms)\n `[CPS] Cache-Prefix Stability`: 0 finding(s) (0ms)\n `[DIS] Disabled-In-Schema`: 0 finding(s) (0ms)\n `[COL] Plugin Skill Collision`: 0 finding(s) (0ms)\n `[SKL] Skill-Listing Budget`: 0 finding(s) (0ms)\n `[AGT] Agent-Listing Budget`: 0 finding(s) (0ms)\n `[OST] Output-Style Validation`: 0 finding(s) (0ms)\n `[OPT] Optimization Lens`: 0 finding(s) (0ms)\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Configuration health\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n Health: A (98/100) — Healthy setup, only minor polish needed\n 9 areas reviewed\n\n Area scores\n ───────────\n `CLAUDE.md` ........... A (90) `Settings` ............ A (100)\n `Hooks` ............... A (100) `Rules` ............... A (100)\n `MCP` ................. A (100) `Imports` ............. A (100)\n `Conflicts` ........... A (100) `Token Efficiency` .... A (90)\n `Plugin Hygiene` ...... A (100)\n\n 17 ways you could get more out of Claude Code — see /config-audit feature-gap\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
"payload": "`[CML] CLAUDE.md Linter`: 1 finding(s) (0ms)\n `[SET] Settings Validator`: 0 finding(s) (0ms)\n `[HKV] Hook Validator`: 0 finding(s) (0ms)\n `[RUL] Rules Validator`: 0 finding(s) (0ms)\n `[MCP] MCP Config Validator`: 0 finding(s) (0ms)\n `[IMP] Import Resolver`: 0 finding(s) (0ms)\n `[CNF] Conflict Detector`: 0 finding(s) (0ms)\n `[GAP] Feature Gap Scanner`: 16 finding(s) (0ms)\n `[TOK] Token Hotspots`: 1 finding(s) (0ms)\n `[CPS] Cache-Prefix Stability`: 0 finding(s) (0ms)\n `[DIS] Disabled-In-Schema`: 0 finding(s) (0ms)\n `[COL] Plugin Skill Collision`: 0 finding(s) (0ms)\n `[SKL] Skill-Listing Budget`: 0 finding(s) (0ms)\n `[AGT] Agent-Listing Budget`: 0 finding(s) (0ms)\n `[OST] Output-Style Validation`: 0 finding(s) (0ms)\n `[OPT] Optimization Lens`: 0 finding(s) (0ms)\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Configuration health\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n Health: A (98/100) — Healthy setup, only minor polish needed\n 9 areas reviewed\n\n Area scores\n ───────────\n `CLAUDE.md` ........... A (90) `Settings` ............ A (100)\n `Hooks` ............... A (100) `Rules` ............... A (100)\n `MCP` ................. A (100) `Imports` ............. A (100)\n `Conflicts` ........... A (100) `Token Efficiency` .... A (90)\n `Plugin Hygiene` ...... A (100)\n\n 16 ways you could get more out of Claude Code — see /config-audit feature-gap\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -341,22 +341,6 @@
|
|||
"id": "CA-GAP-014",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't set up auto-mode classification",
|
||||
"description": "Auto-mode classification helps Claude decide when to act on its own vs. ask you, based on the kind of task.",
|
||||
"file": null,
|
||||
"line": null,
|
||||
"evidence": null,
|
||||
"category": "t3",
|
||||
"recommendation": "Add an auto-mode classifier in your settings if you want this nuance.",
|
||||
"autoFixable": false,
|
||||
"userImpactCategory": "Missed opportunity",
|
||||
"userActionLanguage": "FYI",
|
||||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-015",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't built a custom plugin yet",
|
||||
"description": "Plugins let you bundle skills, automations, and connected services that you want available across many projects.",
|
||||
"file": null,
|
||||
|
|
@ -370,7 +354,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-016",
|
||||
"id": "CA-GAP-015",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "Your project has no settings managed by your organization",
|
||||
|
|
@ -386,7 +370,7 @@
|
|||
"relevanceContext": "affects-everyone"
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-017",
|
||||
"id": "CA-GAP-016",
|
||||
"scanner": "GAP",
|
||||
"severity": "info",
|
||||
"title": "You haven't connected Claude to your editor's language servers",
|
||||
|
|
@ -407,7 +391,7 @@
|
|||
"high": 0,
|
||||
"medium": 1,
|
||||
"low": 5,
|
||||
"info": 11
|
||||
"info": 10
|
||||
}
|
||||
},
|
||||
{
|
||||
|
|
@ -613,13 +597,13 @@
|
|||
}
|
||||
],
|
||||
"aggregate": {
|
||||
"total_findings": 19,
|
||||
"total_findings": 18,
|
||||
"counts": {
|
||||
"critical": 0,
|
||||
"high": 0,
|
||||
"medium": 1,
|
||||
"low": 7,
|
||||
"info": 11
|
||||
"info": 10
|
||||
},
|
||||
"risk_score": 11,
|
||||
"risk_band": "Medium",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue