fix(acr): on-demand copy for oversized skill-body finding (M-BUG-16)
The skill-listing check emits a third finding for an oversized skill BODY
(v5.11 B7, RAW title "Skill body is large (loads on demand when the skill
runs)"). The body is an ON-DEMAND cost — it loads only when the skill is
invoked, not the always-loaded listing Claude reads every turn. The scanner is
careful to distinguish the two (RAW title + comment + evidence note).
But the humanizer-data SKL.static map had no entry for this title, so it fell
through to SKL._default ("A skill is using more of the listing budget than it
should"). The humanized title therefore claimed a listing-budget cost and
directly contradicted the finding's own humanized evidence ("loads ON DEMAND
only ... NOT every turn like the always-loaded listing") — the same internal
contradiction class as M-BUG-15/M-BUG-14, and the same "new finding type added
without a matching humanizer entry" gap the scanner checklist warns about.
Found by finding-granularity premise-verification of the linkedin-posts scan
before feeding it to the analyze pipeline (the prior session's pass focused on
the GAP findings and did not catch the SKL fall-through).
- humanizer-data.mjs: add SKL.static entry for "Skill body is large (loads on
demand when the skill runs)" with on-demand-correct title ("A skill's body is
large (it loads only when that skill runs)"), description, and recommendation.
No listing-budget language; tier1/tier3 forbidden-word checks pass.
- RED-first tests at both layers: humanizer.test.mjs (humanizeFinding path:
title is not the listing-budget _default, conveys on-demand body) and
humanizer-data.test.mjs (static entry exists, on-demand-correct).
RAW envelope unaffected (humanizer bypassed for --raw/--json), frozen v5.0.0
snapshots untouched, default-output fixtures contain no oversized-body skill so
no snapshot regen. Suite 1357->1359/0 (+2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01683eAqVecv9VZfQzL8CQ9h
This commit is contained in:
parent
2975b0563f
commit
239e88cecb
3 changed files with 42 additions and 0 deletions
|
|
@ -817,6 +817,11 @@ export const TRANSLATIONS = {
|
|||
description: 'Claude Code keeps every active skill\'s description in one shared listing it reads to choose which skill to use, and that listing has a limited size. Added up, your skills\' descriptions run past that size on a smaller setup, so Claude Code may drop some of them — and stop seeing those skills. This is an estimate; a larger setup has more room.',
|
||||
recommendation: 'Free up room: turn off bundled skills you do not use, collapse the heaviest ones so only their names show, or shorten the longest descriptions. The details show the measured total and the room available.',
|
||||
},
|
||||
'Skill body is large (loads on demand when the skill runs)': {
|
||||
title: 'A skill\'s body is large (it loads only when that skill runs)',
|
||||
description: 'This skill\'s instructions run longer than the rough guidance for a skill body. The body is not part of the always-loaded listing Claude reads every turn — it loads only when you invoke the skill, so it costs nothing until then. Once it loads, though, it stays in context for the rest of that session.',
|
||||
recommendation: 'Move reference material into supporting files the skill opens only when needed, so the body stays lean. For a heavy skill you can also run its body in a separate context with `context: fork` in the skill\'s settings.',
|
||||
},
|
||||
},
|
||||
patterns: [],
|
||||
_default: {
|
||||
|
|
|
|||
|
|
@ -203,6 +203,22 @@ test('GAP enhancement-gap titles do not presuppose the feature exists (M-BUG-15)
|
|||
`t3_6 title should still name subagent isolation: "${iso.title}"`);
|
||||
});
|
||||
|
||||
test('SKL oversized-body copy is on-demand, not listing-budget (M-BUG-16)', () => {
|
||||
// The skill-listing check emits a third finding for an oversized skill BODY,
|
||||
// which loads ON DEMAND only when the skill runs — not the always-loaded
|
||||
// listing. Before the fix this title had no static entry and fell through to
|
||||
// the SKL _default ("using more of the listing budget"), contradicting the
|
||||
// finding's own evidence ("NOT every turn like the always-loaded listing").
|
||||
const t = TRANSLATIONS.SKL.static['Skill body is large (loads on demand when the skill runs)'];
|
||||
assert.ok(t, 'SKL static map missing "Skill body is large (loads on demand when the skill runs)"');
|
||||
assert.ok(!/listing budget/i.test(t.title), `body title must not claim listing budget: "${t.title}"`);
|
||||
assert.ok(!/listing budget/i.test(t.description), `body description must not claim listing budget: "${t.description}"`);
|
||||
// Must convey the on-demand body cost.
|
||||
assert.ok(/body/i.test(t.title), `body title should name the body: "${t.title}"`);
|
||||
assert.ok(/loads only|on demand|when (it|that|the) skill runs|when you invoke/i.test(t.title + ' ' + t.description),
|
||||
`copy should convey the on-demand load: "${t.title}" / "${t.description}"`);
|
||||
});
|
||||
|
||||
test('CNF, COL, PLH have at least one pattern entry (template-literal titles)', () => {
|
||||
// These scanners use template-literal titles for some findings.
|
||||
for (const prefix of ['CNF', 'COL', 'PLH']) {
|
||||
|
|
|
|||
|
|
@ -116,6 +116,27 @@ test('humanizeFinding falls back to _default when title unknown', () => {
|
|||
assert.ok(/instructions file/i.test(out.title), `expected CML _default title, got: ${out.title}`);
|
||||
});
|
||||
|
||||
test('humanizeFinding maps SKL oversized-body finding to an on-demand title, not the listing-budget _default', () => {
|
||||
// RAW title emitted by skill-listing-scanner for body > threshold (v5.11 B7).
|
||||
// It is an ON-DEMAND body cost, NOT the always-loaded listing budget — the
|
||||
// scanner is careful to distinguish them, so the humanized title must not
|
||||
// regress into "listing budget" language via the SKL _default (M-BUG-16).
|
||||
const input = makeFinding({
|
||||
scanner: 'SKL',
|
||||
severity: 'low',
|
||||
title: 'Skill body is large (loads on demand when the skill runs)',
|
||||
description: 'Skill "repo-init" (user) has a body of about 6223 tokens (712 lines), over the ~5000-token guidance for a skill body.',
|
||||
evidence: 'body_tokens~6223; threshold=5000 tok; skill="repo-init"; source=user - this is the skill BODY which loads ON DEMAND only when the skill is invoked - NOT every turn like the always-loaded listing.',
|
||||
});
|
||||
const out = humanizeFinding(input);
|
||||
assert.doesNotMatch(out.title, /listing budget/i,
|
||||
`body finding must not be framed as a listing-budget cost, got: ${out.title}`);
|
||||
assert.notEqual(out.title, 'A skill is using more of the listing budget than it should',
|
||||
'body finding must have its own mapping, not the SKL _default');
|
||||
assert.match(out.title, /body|on demand|when (it|the skill) runs/i,
|
||||
`humanized title must convey the on-demand body cost, got: ${out.title}`);
|
||||
});
|
||||
|
||||
test('humanizeFinding passes through original strings when scanner prefix unknown', () => {
|
||||
const input = makeFinding({ scanner: 'XXX', title: 'whatever' });
|
||||
const out = humanizeFinding(input);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue