open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

feat(skl): flag oversized skill bodies on demand (v5.11 B7) [skip-docs]

Browse source 
New CA-SKL-003 (low): a SKILL.md body over ~5,000 tokens (~500 lines)
should split reference content into supporting files / use context: fork.

- measureActiveSkillListing() now returns body metrics (chars/lines/tokens);
  the body was already read in full, only the frontmatter was parsed before.
- Honest framing: BODY_CALIBRATION_NOTE marks this as ON-DEMAND cost (loads
  only when the skill is invoked, NOT every turn like the always-loaded
  listing) and an estimate — hence low severity.
- 5 new tests; full suite 1262 green; snapshots byte-stable (default branch
  untouched; new finding fires only on bodies >5k tok, none in fixtures).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-06-23 21:36:20 +02:00
commit 27988801be
3 changed files with 149 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

24
scanners/lib/skill-listing-budget.mjs
View file

@ -48,6 +48,19 @@ export const BUDGET_CALIBRATION_NOTE =
`window; at ${withCommas(LARGE_CONTEXT_WINDOW)} context the budget is ~${withCommas(LARGE_CONTEXT_BUDGET_TOKENS)} ` +
'tok and you are likely within it. this is an estimate, not measured telemetry';
// Skill-body size guidance (CA-SKL-003). A SKILL.md body over ~5,000 tokens
// (~500 lines / ~20k chars) should split reference content into supporting files
// (Claude Code skill-authoring guidance). Unlike the listing budget above, the
// body is an ON-DEMAND cost: it loads only when the skill is invoked, not every
// turn — so this is a LOW-severity efficiency signal, not an always-loaded bill.
export const BODY_TOKEN_THRESHOLD = 5000;
// Honest framing for the body-size finding: distinguishes on-demand from
// always-loaded cost and flags the figure as an estimate. Appended to evidence.
export const BODY_CALIBRATION_NOTE =
'this is the skill BODY (SKILL.md below the frontmatter), which loads ON DEMAND only when the ' +
'skill is invoked - NOT every turn like the always-loaded listing. estimate (chars/4), not measured telemetry';
/**
* @typedef {object} BudgetAssessment
* @property {number} scanned - number of descriptions assessed
@ -91,6 +104,9 @@ export function assessSkillListingBudget(descLengths) {
* @property {string|null} pluginName
* @property {string} path
* @property {number} descLength
* @property {number} bodyChars - SKILL.md body length below the frontmatter (on-demand cost)
* @property {number} bodyLines - body line count
* @property {number} bodyTokens - estimateTokens(bodyChars, 'markdown')
*/
/**
@ -110,14 +126,20 @@ export async function measureActiveSkillListing() {
if (!skill || typeof skill.path !== 'string') continue;
const content = await readTextFile(skill.path);
if (!content) continue;
const fm = parseFrontmatter(content)?.frontmatter || null;
const parsed = parseFrontmatter(content);
const fm = parsed?.frontmatter || null;
const desc = (fm && typeof fm.description === 'string') ? fm.description : '';
const body = (parsed && typeof parsed.body === 'string') ? parsed.body : '';
const bodyChars = body.length;
skills.push({
name: skill.name,
source: skill.source,
pluginName: skill.pluginName,
path: skill.path,
descLength: desc.length,
bodyChars,
bodyLines: bodyChars === 0 ? 0 : body.split('\n').length,
bodyTokens: estimateTokens(bodyChars, 'markdown'),
});
}