fix(acr): on-demand copy for oversized skill-body finding (M-BUG-16)
The skill-listing check emits a third finding for an oversized skill BODY
(v5.11 B7, RAW title "Skill body is large (loads on demand when the skill
runs)"). The body is an ON-DEMAND cost — it loads only when the skill is
invoked, not the always-loaded listing Claude reads every turn. The scanner is
careful to distinguish the two (RAW title + comment + evidence note).
But the humanizer-data SKL.static map had no entry for this title, so it fell
through to SKL._default ("A skill is using more of the listing budget than it
should"). The humanized title therefore claimed a listing-budget cost and
directly contradicted the finding's own humanized evidence ("loads ON DEMAND
only ... NOT every turn like the always-loaded listing") — the same internal
contradiction class as M-BUG-15/M-BUG-14, and the same "new finding type added
without a matching humanizer entry" gap the scanner checklist warns about.
Found by finding-granularity premise-verification of the linkedin-posts scan
before feeding it to the analyze pipeline (the prior session's pass focused on
the GAP findings and did not catch the SKL fall-through).
- humanizer-data.mjs: add SKL.static entry for "Skill body is large (loads on
demand when the skill runs)" with on-demand-correct title ("A skill's body is
large (it loads only when that skill runs)"), description, and recommendation.
No listing-budget language; tier1/tier3 forbidden-word checks pass.
- RED-first tests at both layers: humanizer.test.mjs (humanizeFinding path:
title is not the listing-budget _default, conveys on-demand body) and
humanizer-data.test.mjs (static entry exists, on-demand-correct).
RAW envelope unaffected (humanizer bypassed for --raw/--json), frozen v5.0.0
snapshots untouched, default-output fixtures contain no oversized-body skill so
no snapshot regen. Suite 1357->1359/0 (+2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01683eAqVecv9VZfQzL8CQ9h
This commit is contained in:
parent
2975b0563f
commit
239e88cecb
3 changed files with 42 additions and 0 deletions
|
|
@ -203,6 +203,22 @@ test('GAP enhancement-gap titles do not presuppose the feature exists (M-BUG-15)
|
|||
`t3_6 title should still name subagent isolation: "${iso.title}"`);
|
||||
});
|
||||
|
||||
test('SKL oversized-body copy is on-demand, not listing-budget (M-BUG-16)', () => {
|
||||
// The skill-listing check emits a third finding for an oversized skill BODY,
|
||||
// which loads ON DEMAND only when the skill runs — not the always-loaded
|
||||
// listing. Before the fix this title had no static entry and fell through to
|
||||
// the SKL _default ("using more of the listing budget"), contradicting the
|
||||
// finding's own evidence ("NOT every turn like the always-loaded listing").
|
||||
const t = TRANSLATIONS.SKL.static['Skill body is large (loads on demand when the skill runs)'];
|
||||
assert.ok(t, 'SKL static map missing "Skill body is large (loads on demand when the skill runs)"');
|
||||
assert.ok(!/listing budget/i.test(t.title), `body title must not claim listing budget: "${t.title}"`);
|
||||
assert.ok(!/listing budget/i.test(t.description), `body description must not claim listing budget: "${t.description}"`);
|
||||
// Must convey the on-demand body cost.
|
||||
assert.ok(/body/i.test(t.title), `body title should name the body: "${t.title}"`);
|
||||
assert.ok(/loads only|on demand|when (it|that|the) skill runs|when you invoke/i.test(t.title + ' ' + t.description),
|
||||
`copy should convey the on-demand load: "${t.title}" / "${t.description}"`);
|
||||
});
|
||||
|
||||
test('CNF, COL, PLH have at least one pattern entry (template-literal titles)', () => {
|
||||
// These scanners use template-literal titles for some findings.
|
||||
for (const prefix of ['CNF', 'COL', 'PLH']) {
|
||||
|
|
|
|||
|
|
@ -116,6 +116,27 @@ test('humanizeFinding falls back to _default when title unknown', () => {
|
|||
assert.ok(/instructions file/i.test(out.title), `expected CML _default title, got: ${out.title}`);
|
||||
});
|
||||
|
||||
test('humanizeFinding maps SKL oversized-body finding to an on-demand title, not the listing-budget _default', () => {
|
||||
// RAW title emitted by skill-listing-scanner for body > threshold (v5.11 B7).
|
||||
// It is an ON-DEMAND body cost, NOT the always-loaded listing budget — the
|
||||
// scanner is careful to distinguish them, so the humanized title must not
|
||||
// regress into "listing budget" language via the SKL _default (M-BUG-16).
|
||||
const input = makeFinding({
|
||||
scanner: 'SKL',
|
||||
severity: 'low',
|
||||
title: 'Skill body is large (loads on demand when the skill runs)',
|
||||
description: 'Skill "repo-init" (user) has a body of about 6223 tokens (712 lines), over the ~5000-token guidance for a skill body.',
|
||||
evidence: 'body_tokens~6223; threshold=5000 tok; skill="repo-init"; source=user - this is the skill BODY which loads ON DEMAND only when the skill is invoked - NOT every turn like the always-loaded listing.',
|
||||
});
|
||||
const out = humanizeFinding(input);
|
||||
assert.doesNotMatch(out.title, /listing budget/i,
|
||||
`body finding must not be framed as a listing-budget cost, got: ${out.title}`);
|
||||
assert.notEqual(out.title, 'A skill is using more of the listing budget than it should',
|
||||
'body finding must have its own mapping, not the SKL _default');
|
||||
assert.match(out.title, /body|on demand|when (it|the skill) runs/i,
|
||||
`humanized title must convey the on-demand body cost, got: ${out.title}`);
|
||||
});
|
||||
|
||||
test('humanizeFinding passes through original strings when scanner prefix unknown', () => {
|
||||
const input = makeFinding({ scanner: 'XXX', title: 'whatever' });
|
||||
const out = humanizeFinding(input);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue