fix(scanners): retire the autoMode GAP dimension, a /doctor duplicate (D1)
CC 2.1.226's /doctor Check 8 covers auto mode with usage-weighted judgement. The binding positioning forbids carrying a feature whose whole value is duplicating a /doctor check, so the "adopt this feature" nudge goes. The deterministic side stays: SET still validates autoMode structure and still flags it as dead config in shared project settings. GAP dimensions 25 -> 24. The title lived in FOUR tables, not the two the removal was scoped against: the dimension list, scoring TITLE_TO_ID, the humanizer's static translations, and the scoring denominators (TIER_COUNTS t3 8->7, TOTAL_DIMENSIONS 25->24, MAX_WEIGHTED 42->41) -- the one that moves a user-visible number. findGapId falls back to 'unknown' silently, so a partial removal would have degraded without failing. A blanket sync invariant now asserts all four against GAP_CHECKS instead of comparing occurrences pairwise; each arm was verified red against its own defect (denominator drift, orphaned humanizer entry, resurrected dimension). Frozen tests/snapshots/v5.0.0/ stays untouched. strip-retired-gap.mjs is the removal twin of strip-added-scanner.mjs: it strips the retired dimension from whichever side still carries it and re-derives GAP IDs, since retiring a dimension from mid-list shifts every later ID by one. Derived utilization figures are dropped from comparison rather than recomputed -- recomputing them in a test helper would assert the new arithmetic against itself, and scoring.test.mjs already pins them exactly. Re-seeding was rejected: it would silently bake in any other drift across every scanner those four files cover. risk_score, risk_band, verdict, overallGrade, maturity and segment are byte-identical across the change (severity info carries zero risk weight; GAP is excluded from the overall grade). Utilization shifts 43 -> 44 on the fixture. D2 (CA-SKL-002) is NOT removed. Verified against the primary source first: the CC changelog carries exactly one budget-fraction statement (L3786, 2.1.32) and nothing supersedes it, so our 2% is current and 002 is not a duplicate with a stale figure. /doctor's ~1% could not be reconciled from the changelog and it discloses its own numbers as disk estimates, so it is recorded, not adopted. Left explicitly unverified in a code note: L3786 says "character budget" while we express tokens -- a 4x difference nobody can settle from the wording. Suite 1531 -> 1535, all green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RsfPGxgwbR3MY54wDC6hat
This commit is contained in:
parent
182a37c1af
commit
4027cdcf54
17 changed files with 368 additions and 75 deletions
|
|
@ -274,7 +274,7 @@ export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
|||
}
|
||||
|
||||
/** @type {GapCheck[]} */
|
||||
const GAP_CHECKS = [
|
||||
export const GAP_CHECKS = [
|
||||
// --- Tier 1: Foundation ---
|
||||
{
|
||||
id: 't1_1', tier: 't1',
|
||||
|
|
@ -480,12 +480,11 @@ const GAP_CHECKS = [
|
|||
return false;
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 't3_8', tier: 't3',
|
||||
title: 'No autoMode classifier',
|
||||
recommendation: 'Configure autoMode in user/local settings with environment context and allow/deny rules.',
|
||||
check: async (ctx) => anySettingsHas(ctx, 'autoMode'),
|
||||
},
|
||||
// t3_8 ('No autoMode classifier') retired in v5.14: CC 2.1.226's /doctor
|
||||
// Check 8 covers auto mode with usage-weighted judgement, so an "adopt this
|
||||
// feature" nudge is a pure duplicate under the binding /doctor positioning.
|
||||
// The DETERMINISTIC side stays: SET still validates autoMode structure and
|
||||
// flags it as dead config in shared project settings.
|
||||
|
||||
// --- Tier 4: Team/Enterprise ---
|
||||
{
|
||||
|
|
|
|||
|
|
@ -499,11 +499,6 @@ export const TRANSLATIONS = {
|
|||
description: 'Dynamic context lets a skill see fresh information (file contents, command output) at the moment it runs, not at the time it was written.',
|
||||
recommendation: 'Use the dynamic-context block in skills that need up-to-date information.',
|
||||
},
|
||||
'No autoMode classifier': {
|
||||
title: 'You haven\'t set up auto-mode classification',
|
||||
description: 'Auto-mode classification helps Claude decide when to act on its own vs. ask you, based on the kind of task.',
|
||||
recommendation: 'Add an auto-mode classifier in your settings if you want this nuance.',
|
||||
},
|
||||
'No project .mcp.json in git': {
|
||||
title: 'Your team has no shared list of connected services',
|
||||
description: 'Without a project-level connected-services file, every teammate has to set up their own connections.',
|
||||
|
|
|
|||
|
|
@ -20,17 +20,17 @@ const GRADE_CONTEXT = {
|
|||
|
||||
// --- Tier weights for utilization calculation ---
|
||||
const TIER_WEIGHTS = { t1: 3, t2: 2, t3: 1, t4: 1 };
|
||||
const TIER_COUNTS = { t1: 5, t2: 7, t3: 8, t4: 5 };
|
||||
const TOTAL_DIMENSIONS = 25;
|
||||
const TIER_COUNTS = { t1: 5, t2: 7, t3: 7, t4: 5 };
|
||||
const TOTAL_DIMENSIONS = 24;
|
||||
const MAX_WEIGHTED = Object.entries(TIER_COUNTS).reduce(
|
||||
(sum, [tier, count]) => sum + count * TIER_WEIGHTS[tier],
|
||||
0,
|
||||
); // 5*3 + 7*2 + 8*1 + 5*1 = 42
|
||||
); // 5*3 + 7*2 + 7*1 + 5*1 = 41
|
||||
|
||||
/**
|
||||
* Calculate weighted utilization from GAP scanner findings.
|
||||
* @param {object[]} gapFindings - Array of GAP scanner findings (each has .category = t1|t2|t3|t4)
|
||||
* @param {number} [totalDimensions=25]
|
||||
* @param {number} [totalDimensions=24]
|
||||
* @returns {{ score: number, overhang: number }}
|
||||
*/
|
||||
export function calculateUtilization(gapFindings, totalDimensions = TOTAL_DIMENSIONS) {
|
||||
|
|
@ -102,7 +102,7 @@ function findGapId(finding) {
|
|||
return TITLE_TO_ID[finding.title] || 'unknown';
|
||||
}
|
||||
|
||||
/** Title→ID mapping for all 25 gap checks */
|
||||
/** Title→ID mapping for all 24 gap checks */
|
||||
const TITLE_TO_ID = {
|
||||
'No CLAUDE.md file': 't1_1',
|
||||
'No permissions configured': 't1_2',
|
||||
|
|
@ -123,7 +123,6 @@ const TITLE_TO_ID = {
|
|||
'No advanced skill frontmatter': 't3_5',
|
||||
'No subagent isolation': 't3_6',
|
||||
'No dynamic skill context': 't3_7',
|
||||
'No autoMode classifier': 't3_8',
|
||||
'No project .mcp.json in git': 't4_1',
|
||||
'No custom plugin': 't4_2',
|
||||
'Agent teams not enabled': 't4_3',
|
||||
|
|
@ -404,4 +403,4 @@ export function generateHealthScorecard(areaScores, opportunityCount, options =
|
|||
return lines.join('\n');
|
||||
}
|
||||
|
||||
export { TITLE_TO_ID, TIER_WEIGHTS, TIER_COUNTS, MAX_WEIGHTED, MATURITY_LEVELS, SEGMENTS };
|
||||
export { TITLE_TO_ID, TIER_WEIGHTS, TIER_COUNTS, TOTAL_DIMENSIONS, MAX_WEIGHTED, MATURITY_LEVELS, SEGMENTS };
|
||||
|
|
|
|||
|
|
@ -36,6 +36,23 @@ export const DESCRIPTION_CAP = 1536;
|
|||
// The 200k/1M window constants live in context-window.mjs (single source of
|
||||
// truth, shared with the CML CLAUDE.md char-budget check); re-exported here so
|
||||
// existing importers of this module keep working.
|
||||
// D2 re-verification (2026-08-09, CC 2.1.226). CC 2.1.226's /doctor reports a
|
||||
// combined skill+command+agent listing and puts the budget near ~1% (~10,000
|
||||
// tok on a 1M window) — half of ours. Checked against the primary source
|
||||
// before touching the number: the CC changelog contains EXACTLY ONE
|
||||
// budget-fraction statement (L3786, under 2.1.32) and no later entry
|
||||
// supersedes it, so 2% stands and CA-SKL-002 is NOT a /doctor duplicate
|
||||
// carrying a stale figure. /doctor's arithmetic could not be reconciled from
|
||||
// the changelog, and /doctor discloses its own numbers as disk estimates
|
||||
// (chars÷4), so its ~1% is recorded, not adopted.
|
||||
//
|
||||
// NOT VERIFIED, deliberately left alone: L3786 says "skill CHARACTER budget
|
||||
// now scales with context window (2% of context)". We express the budget in
|
||||
// TOKENS (0.02 × 200k = 4000 tok). Whether CC's budget is 2% counted in
|
||||
// characters or in tokens is not resolvable from the changelog wording, and no
|
||||
// primary source settles it — a 4× difference rides on the answer. Changing
|
||||
// the constant on that ambiguity would be a guess; it stays until a primary
|
||||
// source decides it.
|
||||
export const BUDGET_FRACTION = 0.02;
|
||||
export const AGGREGATE_BUDGET_TOKENS = Math.round(BUDGET_FRACTION * CONTEXT_WINDOW_ANCHOR); // 4000
|
||||
export const LARGE_CONTEXT_BUDGET_TOKENS = Math.round(BUDGET_FRACTION * LARGE_CONTEXT_WINDOW); // 20000
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue