feat(ost): v5.6 C — output-style scanner (CA-OST, count 13→14)

New orchestrated scanner output-style-scanner.mjs — first new family since
SKL. Three findings, each pinned to a CONFIRMED V-row of the steering-model
plan + re-verified against code.claude.com/docs/en/output-styles:

- CA-OST-001 (medium, V10): user/project custom style missing
  keep-coding-instructions:true (default false) → silently strips built-in
  software-engineering instructions when active. Scoped to user/project.
- CA-OST-002 (low, V11): plugin style with force-for-plugin:true overrides the
  user's selected outputStyle. Verifiseringsplikt correction — the plan bullet
  said "project/user style," but force-for-plugin is plugin-styles-only per the
  docs, so the check keys on source==='plugin'.
- CA-OST-003 (medium): settings outputStyle matching no built-in
  (Default/Explanatory/Learning/Proactive, case-insensitive) nor discovered
  custom style → dead config.

Byte-stability — a scanner addition, not a field addition. Growing the
scanners array + scanners_ok cannot be hidden by a field strip, but re-seeding
frozen v5.0.0 (the SKL precedent) would now bake in B2's hotspot triple +
claudeMd drift. So, per the B2 lesson, frozen v5.0.0 snapshots are PRESERVED
and the OST entry is stripped at compare time via new
tests/helpers/strip-added-scanner.mjs (wired into json/raw-backcompat + the
Step 5/6 humanizer tests); only SC-5 default-output is regenerated (additive
OST entry, diff reviewed). OST is fixture-gated (no output styles on
marketplace-medium / hermetic HOME → silent).

Wiring: orchestrator; humanizer (OST→Configuration mistake) + humanizer-data
OST family (title-coupled); scoring (OST→Settings, keeps 10 areas). Suite
1012→1023 (+11). Badges: scanners 14, tests 1023, TRANSLATIONS families 15.
Lore swept: README, CLAUDE.md, scanner-internals, humanizer.md. self-audit A/A.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-06-20 21:02:44 +02:00
commit e3b044a476
20 changed files with 557 additions and 47 deletions

View file

@ -820,4 +820,34 @@ export const TRANSLATIONS = {
recommendation: 'See the details for which skill to trim or turn off.',
},
},
// ─────────────────────────────────────────────────────────────
// OST — Output-Style Validation
// Category: Configuration mistake
// ─────────────────────────────────────────────────────────────
OST: {
static: {
'Custom output style removes built-in coding instructions': {
title: 'A custom output style turns off Claude\'s coding know-how',
description: 'This style replaces Claude Code\'s built-in coding guidance with only your own text, so while it\'s active Claude forgets how to scope changes, comment, and verify work. The setting that keeps that guidance is off by default.',
recommendation: 'Add `keep-coding-instructions: true` to the top of the style file to keep that guidance. If you meant to drop it for a non-coding style, leave it as is.',
},
'Plugin output style overrides your selected output style': {
title: 'A plugin is forcing its own output style on you',
description: 'This plugin applies its own output style automatically whenever it\'s on, replacing the one you picked. If two plugins both do this, the first one to load wins.',
recommendation: 'If you didn\'t want this, turn off the plugin or remove the force setting from its style. Otherwise there\'s nothing to do — the plugin works this way on purpose.',
},
'Configured output style does not exist': {
title: 'Your chosen output style can\'t be found',
description: 'Your settings point to an output style that doesn\'t exist by that name, so Claude Code quietly uses the default instead. The style you wanted never takes effect.',
recommendation: 'Check the spelling against your styles (built-ins are Default, Explanatory, Learning, Proactive), add the missing style file, or remove the setting.',
},
},
patterns: [],
_default: {
title: 'Something about your output styles needs a look',
description: 'A check on your output styles flagged something worth reviewing.',
recommendation: 'See the details for which output style to adjust.',
},
},
};

View file

@ -41,6 +41,7 @@ const SCANNER_TO_CATEGORY = {
DIS: 'Dead config',
GAP: 'Missed opportunity',
PLH: 'Configuration mistake',
OST: 'Configuration mistake',
};
/**

View file

@ -168,6 +168,7 @@ const SCANNER_AREA_MAP = {
SKL: 'Token Efficiency',
DIS: 'Settings',
COL: 'Plugin Hygiene',
OST: 'Settings',
};
/**

View file

@ -0,0 +1,179 @@
/**
* OST Scanner Output-style validation (v5.6 C)
*
* Output styles are live (the standalone `/output-style` command was removed in
* v2.1.91; styles are now managed via `/config`). They are the most surprising
* steering surface because they rewrite the system prompt:
*
* CA-OST-001 A custom (user/project) output style that does NOT set
* `keep-coding-instructions: true` when active, Claude Code
* REMOVES its built-in software-engineering instructions (how to
* scope changes, write comments, verify work) and keeps only the
* style's text. `keep-coding-instructions` defaults to false, so
* this is the headline footgun. Severity medium.
*
* CA-OST-002 A PLUGIN output style with `force-for-plugin: true` Claude
* Code auto-applies it whenever the plugin is enabled, OVERRIDING
* the user's selected `outputStyle`. If several enabled plugins set
* it, the first loaded wins. Severity low (awareness). Note:
* `force-for-plugin` is plugin-styles-only per the docs, so this
* keys on `source === 'plugin'` a user/project style cannot
* trigger the override (it would simply be ignored).
*
* CA-OST-003 A settings `outputStyle` value that matches no built-in and no
* discovered custom style dead config: Claude Code falls back to
* the default style, so the configured behavior is silently not
* applied. Severity medium.
*
* Every claim traces to a CONFIRMED row of docs/v5.5-steering-model-plan.md
* (V9/V10/V11/V12), verified against code.claude.com/docs/en/output-styles and
* .../plugins-reference. The scanner is fixture-gated: with no output styles and
* no `outputStyle` setting it emits nothing (keeps the SC-5 snapshot byte-stable).
*
* Zero external dependencies.
*/
import { readFile } from 'node:fs/promises';
import { finding, scannerResult } from './lib/output.mjs';
import { SEVERITY } from './lib/severity.mjs';
import { readActiveConfig } from './lib/active-config-reader.mjs';
import { parseFrontmatter, parseJson } from './lib/yaml-parser.mjs';
const SCANNER = 'OST';
// Built-in output styles, verified against code.claude.com/docs/en/output-styles.
// Compared case-insensitively so OST-003 never false-flags a valid built-in.
const BUILTIN_STYLES = ['default', 'explanatory', 'learning', 'proactive'];
/**
* Read + parse the frontmatter of each enumerated output style once.
* @param {Array<object>} styles - readActiveConfig().outputStyles entries
*/
async function withFrontmatter(styles) {
const out = [];
for (const s of styles) {
let frontmatter = null;
try {
frontmatter = parseFrontmatter(await readFile(s.path, 'utf-8')).frontmatter;
} catch { /* unreadable → treat as no frontmatter */ }
out.push({ ...s, frontmatter });
}
return out;
}
/**
* Resolve the effective `outputStyle` setting from the cascade (user project
* local; later scope wins). Returns null when unset everywhere.
* @param {object} activeConfig
* @returns {Promise<{value:string, scope:string, path:string} | null>}
*/
async function resolveOutputStyleSetting(activeConfig) {
const cascade = (activeConfig.settings && activeConfig.settings.cascade) || [];
let resolved = null;
for (const entry of cascade) {
if (!entry.exists || !entry.path) continue;
let json = null;
try { json = parseJson(await readFile(entry.path, 'utf-8')); } catch { continue; }
if (json && typeof json.outputStyle === 'string' && json.outputStyle.trim()) {
resolved = { value: json.outputStyle.trim(), scope: entry.scope, path: entry.path };
}
}
return resolved;
}
/**
* Main scanner entry point.
* @param {string} targetPath - repo root to scan
* @param {object} _discovery - unused (OST reads the active config cascade itself)
*/
export async function scan(targetPath, _discovery) {
const start = Date.now();
const findings = [];
const activeConfig = await readActiveConfig(targetPath);
const styles = await withFrontmatter(activeConfig.outputStyles || []);
// CA-OST-001 — user/project custom style missing keep-coding-instructions:true.
for (const s of styles) {
if (s.source !== 'project' && s.source !== 'user') continue;
const kci = s.frontmatter ? s.frontmatter.keep_coding_instructions : undefined;
if (kci === true) continue;
findings.push(finding({
scanner: SCANNER,
severity: SEVERITY.medium,
title: 'Custom output style removes built-in coding instructions',
description:
`The ${s.source} output style "${s.name}" does not set ` +
'`keep-coding-instructions: true`. While this style is active, Claude Code ' +
'drops its built-in software-engineering instructions — how to scope changes, ' +
'write comments, and verify work — and keeps only this style\'s text. The ' +
'frontmatter flag defaults to false, so the strip is easy to miss.',
file: s.path,
evidence:
`output_style="${s.name}"; source=${s.source}; ` +
`keep-coding-instructions=${kci === undefined ? 'unset (default false)' : String(kci)}`,
recommendation:
'To keep Claude Code\'s software-engineering behavior while applying this style, ' +
'add `keep-coding-instructions: true` to the frontmatter. If the strip is ' +
'intentional (a non-coding persona), no change is needed.',
category: 'output-styles',
}));
}
// CA-OST-002 — plugin output style with force-for-plugin:true (overrides user choice).
for (const s of styles) {
if (s.source !== 'plugin') continue;
const ffp = s.frontmatter ? s.frontmatter.force_for_plugin : undefined;
if (ffp !== true) continue;
findings.push(finding({
scanner: SCANNER,
severity: SEVERITY.low,
title: 'Plugin output style overrides your selected output style',
description:
`The plugin "${s.pluginName}" ships an output style "${s.name}" with ` +
'`force-for-plugin: true`, so Claude Code applies it automatically whenever the ' +
'plugin is enabled — overriding whatever `outputStyle` you selected. When more ' +
'than one enabled plugin does this, the first one loaded wins.',
file: s.path,
evidence: `output_style="${s.name}"; source=plugin:${s.pluginName}; force-for-plugin=true`,
recommendation:
'If you did not expect this style, disable the plugin or remove ' +
'`force-for-plugin: true` from its output style. This is awareness only — the ' +
'plugin is behaving as designed.',
category: 'output-styles',
}));
}
// CA-OST-003 — settings outputStyle resolving to a non-existent style (dead config).
const resolved = await resolveOutputStyleSetting(activeConfig);
if (resolved) {
const known = new Set([
...BUILTIN_STYLES,
...styles.map(s => String(s.name).toLowerCase()),
]);
if (!known.has(resolved.value.toLowerCase())) {
const customNames = styles.map(s => s.name);
findings.push(finding({
scanner: SCANNER,
severity: SEVERITY.medium,
title: 'Configured output style does not exist',
description:
`Your ${resolved.scope} settings set \`outputStyle: "${resolved.value}"\`, but no ` +
'built-in or discovered custom style has that name. Claude Code falls back to the ' +
'default style, so the output behavior you configured is silently never applied.',
file: resolved.path,
evidence:
`outputStyle="${resolved.value}"; scope=${resolved.scope}; ` +
`builtins=[${BUILTIN_STYLES.join(', ')}]; ` +
`known_custom=[${customNames.join(', ')}]`,
recommendation:
'Fix the value to match an existing style (built-ins: Default, Explanatory, ' +
'Learning, Proactive), create the missing style under `.claude/output-styles/`, ' +
'or remove the `outputStyle` setting.',
category: 'output-styles',
}));
}
}
return scannerResult(SCANNER, 'ok', findings, styles.length, Date.now() - start);
}

View file

@ -29,6 +29,7 @@ import { scan as scanCachePrefix } from './cache-prefix-scanner.mjs';
import { scan as scanDisabledInSchema } from './disabled-in-schema-scanner.mjs';
import { scan as scanCollision } from './collision-scanner.mjs';
import { scan as scanSkillListing } from './skill-listing-scanner.mjs';
import { scan as scanOutputStyle } from './output-style-scanner.mjs';
// Directory names that identify test fixture / example directories
const FIXTURE_DIR_NAMES = ['tests', 'examples', '__tests__', 'test-fixtures'];
@ -64,6 +65,7 @@ const SCANNERS = [
{ name: 'DIS', fn: scanDisabledInSchema, label: 'Disabled-In-Schema' },
{ name: 'COL', fn: scanCollision, label: 'Plugin Skill Collision' },
{ name: 'SKL', fn: scanSkillListing, label: 'Skill-Listing Budget' },
{ name: 'OST', fn: scanOutputStyle, label: 'Output-Style Validation' },
];
/**