feat(ost): v5.6 C — output-style scanner (CA-OST, count 13→14)
New orchestrated scanner output-style-scanner.mjs — first new family since SKL. Three findings, each pinned to a CONFIRMED V-row of the steering-model plan + re-verified against code.claude.com/docs/en/output-styles: - CA-OST-001 (medium, V10): user/project custom style missing keep-coding-instructions:true (default false) → silently strips built-in software-engineering instructions when active. Scoped to user/project. - CA-OST-002 (low, V11): plugin style with force-for-plugin:true overrides the user's selected outputStyle. Verifiseringsplikt correction — the plan bullet said "project/user style," but force-for-plugin is plugin-styles-only per the docs, so the check keys on source==='plugin'. - CA-OST-003 (medium): settings outputStyle matching no built-in (Default/Explanatory/Learning/Proactive, case-insensitive) nor discovered custom style → dead config. Byte-stability — a scanner addition, not a field addition. Growing the scanners array + scanners_ok cannot be hidden by a field strip, but re-seeding frozen v5.0.0 (the SKL precedent) would now bake in B2's hotspot triple + claudeMd drift. So, per the B2 lesson, frozen v5.0.0 snapshots are PRESERVED and the OST entry is stripped at compare time via new tests/helpers/strip-added-scanner.mjs (wired into json/raw-backcompat + the Step 5/6 humanizer tests); only SC-5 default-output is regenerated (additive OST entry, diff reviewed). OST is fixture-gated (no output styles on marketplace-medium / hermetic HOME → silent). Wiring: orchestrator; humanizer (OST→Configuration mistake) + humanizer-data OST family (title-coupled); scoring (OST→Settings, keeps 10 areas). Suite 1012→1023 (+11). Badges: scanners 14, tests 1023, TRANSLATIONS families 15. Lore swept: README, CLAUDE.md, scanner-internals, humanizer.md. self-audit A/A. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
43d8873339
commit
e3b044a476
20 changed files with 557 additions and 47 deletions
46
CLAUDE.md
46
CLAUDE.md
|
|
@ -101,7 +101,7 @@ Default: auto-detects scope from git context. Override with `/config-audit full|
|
|||
```
|
||||
|
||||
### Finding ID Format
|
||||
`CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-DIS-001`, `CA-COL-001`, `CA-SKL-001`
|
||||
`CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-DIS-001`, `CA-COL-001`, `CA-SKL-001`, `CA-OST-001`
|
||||
|
||||
## Testing
|
||||
|
||||
|
|
@ -109,7 +109,7 @@ Default: auto-detects scope from git context. Override with `/config-audit full|
|
|||
node --test 'tests/**/*.test.mjs'
|
||||
```
|
||||
|
||||
1012 tests across 56 test files (17 lib + 29 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`.
|
||||
1023 tests across 57 test files (17 lib + 30 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`.
|
||||
|
||||
### active-config-reader — load-pattern model + rule/agent/output-style enumeration (v5.6 Foundation)
|
||||
|
||||
|
|
@ -307,6 +307,48 @@ The two sub-checks are independent (a malformed autoMode in shared scope yields
|
|||
orchestrator, so SC-5 was re-checked after this change — byte-equal (the snapshot fixture has no
|
||||
`autoMode`, so the block never fires there).
|
||||
|
||||
### OST scanner — output-style validation (`CA-OST`, v5.6 C, count 13→14)
|
||||
|
||||
New orchestrated scanner `output-style-scanner.mjs` — the first new scanner family since SKL
|
||||
(v5.2.0). It reads the active config (`readActiveConfig`) and each output-style file's frontmatter
|
||||
(via `parseFrontmatter`, keys hyphen→underscore-normalized, so it reads `keep_coding_instructions` /
|
||||
`force_for_plugin`). Three findings, every claim pinned to a CONFIRMED row of
|
||||
`docs/v5.5-steering-model-plan.md` (V9/V10/V11/V12), re-verified against
|
||||
`code.claude.com/docs/en/output-styles` + `.../plugins-reference`:
|
||||
|
||||
- **`CA-OST-001`** (medium) — a **user/project** custom style not setting `keep-coding-instructions:
|
||||
true`. The flag defaults to **false**, so the style silently **removes** Claude Code's built-in
|
||||
software-engineering instructions when active (V10). Scoped to user/project (the styles the user
|
||||
authors); a plugin author's choice is out of scope.
|
||||
- **`CA-OST-002`** (low) — a **plugin** style with `force-for-plugin: true`, which auto-applies and
|
||||
**overrides** the user's selected `outputStyle` (V11). **Verifiseringsplikt correction:** the v5.5+
|
||||
plan's CA-OST-002 bullet said "in a project/user style," but `force-for-plugin` is
|
||||
**plugin-styles-only** per the docs (its own cited V11 + `output-styles.md`), so the check keys on
|
||||
`source === 'plugin'` — a user/project style with the flag is simply ignored, not an override.
|
||||
- **`CA-OST-003`** (medium) — a settings `outputStyle` value resolving to **no** built-in
|
||||
(`Default`/`Explanatory`/`Learning`/`Proactive`, matched case-insensitively) and **no** discovered
|
||||
custom style → dead config (CC falls back to default; the configured behavior never applies).
|
||||
|
||||
**Byte-stability — a scanner addition, NOT a field addition.** Adding the 14th scanner grows
|
||||
`envelope.scanners` by one entry and bumps `aggregate.scanners_ok` 12→13 on the deterministic
|
||||
fixture **regardless of findings** — a field-strip helper cannot paper this over. The SKL precedent
|
||||
(`7bb2547`) re-seeded the frozen v5.0.0 snapshots, but that predates B2's strip-preservation regime;
|
||||
re-seeding now would **bake in** B2's hotspot triple + `claudeMdEstimatedTokens` drift (verified by
|
||||
inspecting the seed diff). So, consistent with the B2 lesson ("preserve frozen via strip-helper;
|
||||
regen ONLY SC-5"), C **preserves** the frozen v5.0.0 snapshots and **strips the OST entry at compare
|
||||
time**: shared `tests/helpers/strip-added-scanner.mjs` (`stripAddedScanners` removes OST entries +
|
||||
decrements `scanners_ok`; `stripAddedScannerStderr` drops the `[OST]` progress line) is wired into
|
||||
json/raw-backcompat + the Step 5/6 humanizer wiring tests (cli-humanizer did **not** break — its
|
||||
v5.0.0 compares don't grow a scanners array). Only **SC-5 default-output** (scan-orchestrator +
|
||||
posture) is regenerated (additive OST entry only — diff reviewed). OST is fixture-gated: the
|
||||
`marketplace-medium` fixture and the hermetic HOME have no output styles, so it emits nothing there.
|
||||
|
||||
Wiring: orchestrator import + `SCANNERS` entry; `humanizer.mjs` `SCANNER_TO_CATEGORY`
|
||||
(`OST: 'Configuration mistake'`); `humanizer-data.mjs` OST family (title-coupled to the three exact
|
||||
finding titles); `scoring.mjs` `SCANNER_AREA_MAP` (`OST: 'Settings'` — keeps the 10 quality areas,
|
||||
byte-stable on zero-finding projects). Count badges: self-audit scanner count 13→14; humanizer-data
|
||||
TRANSLATIONS families 14→15 (PLH is a translation family but not orchestrated).
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Session directories accumulate — use `/config-audit cleanup` to manage
|
||||
|
|
|
|||
13
README.md
13
README.md
|
|
@ -8,14 +8,14 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 13 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, and cross-plugin collision detection. Zero external dependencies.
|
||||
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 14 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, cross-plugin collision, and output-style detection. Zero external dependencies.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -76,7 +76,7 @@ Claude Code reads instructions from at least 7 different file types across multi
|
|||
|
||||
This plugin provides three layers of configuration intelligence:
|
||||
|
||||
- **Health** — 13 deterministic scanners verify correctness across every configuration file, catching broken imports, deprecated settings, conflicting rules, format errors, permission contradictions, prompt-cache token waste, cache-prefix instability, dead tool grants, and cross-plugin skill collisions
|
||||
- **Health** — 14 deterministic scanners verify correctness across every configuration file, catching broken imports, deprecated settings, conflicting rules, format errors, permission contradictions, prompt-cache token waste, cache-prefix instability, dead tool grants, cross-plugin skill collisions, and output styles that silently strip Claude Code's coding instructions
|
||||
- **Opportunities** — context-aware recommendations for Claude Code features that could benefit your specific project, backed by Anthropic's official guidance
|
||||
- **Action** — auto-fix with mandatory backups, syntax validation, rollback support, and a human-in-the-loop workflow for anything non-trivial
|
||||
|
||||
|
|
@ -305,7 +305,7 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
|
||||
## Deterministic Scanners
|
||||
|
||||
13 Node.js scanners that perform structural analysis an LLM cannot reliably do: schema validation, circular reference detection, import resolution, conflict detection across scopes, prompt-cache-aware token-cost analysis, cache-prefix stability, dead-tool detection, and cross-plugin skill collisions. Plus a standalone plugin-health scanner. Zero external dependencies.
|
||||
14 Node.js scanners that perform structural analysis an LLM cannot reliably do: schema validation, circular reference detection, import resolution, conflict detection across scopes, prompt-cache-aware token-cost analysis, cache-prefix stability, dead-tool detection, cross-plugin skill collisions, and output-style validation. Plus a standalone plugin-health scanner. Zero external dependencies.
|
||||
|
||||
**Why deterministic?** LLMs are powerful at understanding intent and context. But they cannot reliably validate JSON schemas, detect circular `@import` chains, or catch that your global `settings.json` contradicts your project-level one. These scanners fill that gap — fast, repeatable, and zero false positives on structural issues.
|
||||
|
||||
|
|
@ -324,6 +324,7 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries: (1) tools in BOTH `permissions.deny` and `permissions.allow` — deny wins (incl. the `Tool(*)` deny-all glob, equivalent to a bare deny); (2) unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — valid only as `mcp__<server>__*`; (3) `Tool(param:value)` rules whose key is the tool's own canonicalizing field (`command`/`file_path`/`path`/`notebook_path`/`url`) — CC ignores these and emits a startup warning |
|
||||
| `collision-scanner.mjs` | COL | Cross-plugin skill name collisions; user-vs-plugin overlaps |
|
||||
| `skill-listing-scanner.mjs` | SKL | Skill-listing token budget: a single skill description over the ~1,536-char listing cap Claude Code truncates (`CA-SKL-001`), and the summed active-skill descriptions exceeding the ~2%-of-context listing budget (`CA-SKL-002`) |
|
||||
| `output-style-scanner.mjs` | OST | Output-style validation: a custom (user/project) style missing `keep-coding-instructions: true` that silently strips built-in software-engineering instructions (`CA-OST-001`), a plugin style with `force-for-plugin: true` overriding the user's selected `outputStyle` (`CA-OST-002`), and a settings `outputStyle` resolving to no built-in or custom style — dead config (`CA-OST-003`) |
|
||||
|
||||
> **Cross-scanner remediation — diagnosis meets the fix.** SKL diagnoses an over-budget
|
||||
> skill listing (`CA-SKL-002`); GAP prescribes the remedy. When the active skill listing
|
||||
|
|
@ -523,7 +524,7 @@ node scanners/posture.mjs examples/optimal-setup/
|
|||
|
||||
### Self-Audit: Scanning the Scanner
|
||||
|
||||
The plugin runs all 13 scanners + the standalone plugin-health scanner on itself via `self-audit.mjs`. Test fixtures and example files are automatically excluded from scoring — a configuration plugin that ships deliberately broken examples shouldn't fail its own audit. Use `--check-readme` to verify badge counts are in sync with the filesystem.
|
||||
The plugin runs all 14 scanners + the standalone plugin-health scanner on itself via `self-audit.mjs`. Test fixtures and example files are automatically excluded from scoring — a configuration plugin that ships deliberately broken examples shouldn't fail its own audit. Use `--check-readme` to verify badge counts are in sync with the filesystem.
|
||||
|
||||
```bash
|
||||
node scanners/self-audit.mjs
|
||||
|
|
|
|||
38
STATE.md
38
STATE.md
|
|
@ -2,34 +2,34 @@
|
|||
|
||||
_Current state-of-play for DENNE pluginen. Auto-injisert ved sesjonsstart. Overskrives (ikke append) ved sesjonsslutt. Historikk → git (STATE.md er tracked). Stier relative til plugin-mappa._
|
||||
|
||||
## Aktiv kontekst (v5.6 B KOMPLETT på main — C er GO-godkjent, START C NESTE SESJON)
|
||||
v5.5.0 released. v5.6 Foundation (`62d910e`) + **B1 manifest (`bb647ce`)** + **B2 token-hotspots (`778b517`)** landet på main (TDD). **Hele B (load-pattern accounting) er ferdig.** Suite **1012** (996→1008→1012), self-audit **A/A**, count **13** (B la ingen scanner — C gir 14). v5.6 IKKE release-cuttet (krever C + eget GO).
|
||||
## Aktiv kontekst (v5.6 C KOMPLETT på main — HELE v5.6 ferdig, IKKE release-cuttet)
|
||||
v5.5.0 released. v5.6 Foundation (`62d910e`) + B1 manifest (`bb647ce`) + B2 token-hotspots (`778b517`) + **C output-style-scanner (denne økten)** landet på main (TDD). **Hele v5.6 (Foundation + B + C) er ferdig.** Suite **1023** (1012→1023, +11 OST), self-audit **A/A** (health 100, quality 93), scanner-count **14** (C la `CA-OST`). v5.6 IKKE release-cuttet (krever eget GO).
|
||||
|
||||
**→ NESTE SESJON: C er allerede GO-godkjent av operatør (2026-06-20, utsatt pga. kontekst-budsjett). Start C direkte — ikke spør om GO på nytt.** Følg vanlig sesjonsstart (les denne, sjekk git), bekreft, og gå rett inn i C (se «Åpne tråder» #1 for full spec).
|
||||
**→ NESTE SESJON: v5.6 release-cut er neste, men KREVER eget GO (som v5.5.0).** Ikke start release uten klarsignal. Følg vanlig sesjonsstart (les denne, sjekk git), bekreft, spør om GO for release-cut.
|
||||
|
||||
## Gjort denne økten (2026-06-20, lør — pushet hvis i vindu)
|
||||
**B1 (manifest, committet `bb647ce`):** komponent-nivå, plugin roll-up fjernet (dobbelttalte), loadPattern-triple per kilde + `summary` (always/onDemand/external) + always-loaded subtotal-headline. manifest = env-aware → mode-equivalence → byte-stabil uten regen.
|
||||
**B2 (token-hotspots, denne committen):**
|
||||
- **`token-hotspots.mjs`:** hver ranked hotspot får loadPattern-triple via `hotspotLoadPattern(h, ruleMap)` — `HOTSPOT_TYPE_TO_KIND`-mapping; rules slår opp `activeConfig.rules` for presis `scoped`; claude-md mapper via scope. To nye `deriveLoadPattern`-kinds: **`command`** (on-demand) + **`harness-config`** (external — settings/keybindings/.mcp.json/hooks.json/plugin.json = harness-config, IKKE model-context). Ærlig split: `.mcp.json`-FILA = external, MCP-serverens tool-schemas = separat `always`-hotspot.
|
||||
- **Snapshot-blast (6 brudd / 5 filer):** TOK ER byte-equal SC-6/7 OG hotspots embedder i scan-orchestrator + posture. Løst ved å **bevare frozen v5.0.0**: delt `tests/helpers/strip-hotspot-load-pattern.mjs` stripper de additive feltene før byte-equal (json/raw-backcompat + de 3 Step 5/6/7 humanizer-testene). **SC-5 default-output regenerert** (`UPDATE_SNAPSHOT=1`, scan-orch + token-hotspots) — diff reviewet additiv-only.
|
||||
- **Tester:** +4 (deriveLoadPattern command+harness-config; TOK hotspots loadPattern-triple + CLAUDE.md-always/skill-on-demand). Docs: README badge/prosa 1012, CLAUDE.md (1012 + ny B2-seksjon), tokens.md Load-kolonne, scanner-internals.
|
||||
**C (output-style-scanner `CA-OST`, count 13→14):** ny orkestrert scanner. 3 findings, hver pinnet til CONFIRMED V-rad + re-verifisert mot `code.claude.com/docs/en/output-styles`:
|
||||
- **`CA-OST-001`** (medium, V10): user/project custom style uten `keep-coding-instructions:true` (default false) → stripper built-in SWE-instruksjoner når aktiv. Scoped til user/project.
|
||||
- **`CA-OST-002`** (low, V11): **plugin**-style med `force-for-plugin:true` → overstyrer brukerens `outputStyle`. **Verifiseringsplikt-korreksjon:** planens bullet sa «project/user style», men docs sier force-for-plugin er **plugin-only** → keyer på `source==='plugin'`.
|
||||
- **`CA-OST-003`** (medium): settings `outputStyle` som matcher ingen built-in (Default/Explanatory/Learning/Proactive, case-insensitiv) eller custom style → dead config.
|
||||
- **Scanner-tillegg ≠ felt-tillegg (byte-stabilitet):** 14. scanner vokser `envelope.scanners` + bumper `scanners_ok` UANSETT findings → strip-helper kan IKKE skjule det. SKL-presedens (`7bb2547`) re-seedet frozen, men det er FØR B2s strip-regime; re-seed nå ville bake inn B2s hotspot-triple + claudeMd-drift (verifisert i seed-diff). Derfor **bevart frozen v5.0.0** + **strippet OST ved compare**: ny `tests/helpers/strip-added-scanner.mjs` (`stripAddedScanners` fjerner OST-entry + dekrementerer `scanners_ok`; `stripAddedScannerStderr` dropper `[OST]`-linja) wiret i json/raw-backcompat + Step 5/6 humanizer-tester. cli-humanizer brøt IKKE (dens v5.0.0-compare vokser ingen scanners-array). KUN **SC-5 default-output** (scan-orch + posture) regenerert (additiv OST-entry — diff reviewet). OST fixture-gated: marketplace-medium + hermetisk HOME har ingen output styles → fyrer ikke der.
|
||||
- **Wiring:** orchestrator import+SCANNERS; `humanizer.mjs` (`OST:'Configuration mistake'`); `humanizer-data.mjs` OST-familie (tittelkoblet); `scoring.mjs` (`OST:'Settings'` — holder 10 areas). Tester +11. Badges: scanner 14, tests 1023, TRANSLATIONS-familier 14→15 (PLH er familie men ikke orkestrert). Lore: README (badge/prosa/scanner-rad), CLAUDE.md (Finding ID + testtelling + ny OST-seksjon), scanner-internals, humanizer.md.
|
||||
|
||||
## Åpne tråder / neste steg (prioritert)
|
||||
1. **v5.6 C (output-style scanner) — GO-GODKJENT, START HER:** ny `CA-OST` (count 13→**14**): `CA-OST-001` custom style mangler `keep-coding-instructions:true` (medium, V10), `CA-OST-002` `force-for-plugin` override (low, V11), `CA-OST-003` dead `outputStyle` (dead config). Foundation `enumerateOutputStyles` ligger klar. Sjekkliste (TDD, hver finding: positiv+negativ fixture): scanner-fil + orchestrator-wiring; badge 13→14 + lore-sweep (README/CLAUDE/scanner-internals + «count stays 13»-noter); humanizer-data (tittelkobling!); scoring; discovery-wiring. Avgjør: ny discovery-type for `.claude/output-styles/` vs direkte (Foundation gjorde det DIREKTE i `enumerateOutputStyles`, ikke via file-discovery). Byte-stabilitet: ny scanner er fixture-gated → SC-5 stabil HVIS den ikke fyrer på snapshot-prosjektet (re-verifiser); count-badge driver self-audit readmeCheck. **Resten KREVER egne GO:**
|
||||
2. **v5.6 release-cut:** etter C (eget GO, som v5.5.0): tag + catalog-bump (separat repo).
|
||||
3. **v5.7:** D (mechanism-fit, heuristisk, precision-gated).
|
||||
4. **llm-security KRITISK** (annet repo) + 3 aktive plugins + STATE-sveip — uendret.
|
||||
1. **v5.6 release-cut — KREVER EGET GO:** tag + catalog-bump (separat repo), oppdater README «What's New» (står på v5.4.0 — bevisst, oppdateres ved release), version-badge 5.5.0→5.6.0. Som v5.5.0-prosessen.
|
||||
2. **v5.7:** D (mechanism-fit, heuristisk, precision-gated — positiv+negativ korpus, behind flag hvis støy). Egen GO.
|
||||
3. **llm-security KRITISK** (annet repo) + 3 aktive plugins + STATE-sveip — uendret. Egne GO.
|
||||
|
||||
## Gotchas (UFRAVIKELIG)
|
||||
- **To motsatte byte-stabilitets-regimer:** manifest = env-aware → mode-equivalence (additive felt gratis). TOK = byte-equal v5.0.0 OG embedder i scan-orch+posture → **5 frozen-v5.0.0-komparatorer** (json/raw-backcompat + cli-humanizer + posture-humanizer + scan-orchestrator-humanizer). **Grep ALLE 5 før du antar blast-radius** ved nytt scanner-output-felt. Bevar frozen via strip-helper; regen KUN SC-5 default-output.
|
||||
- **manifest har INGEN findings** → ingen humanizer/fix-engine-tittelkobling (kun finding-scannere).
|
||||
- **Skill on-demand** (body), ikke always. Always-listing (navn+desc) liten, spores av skill-listing-budget/posture.
|
||||
- **harness-config = external** (settings/manifests koster ingen per-tur context-tokens). `.mcp.json`-fil ≠ MCP-tool-schemas (sistnevnte always).
|
||||
- tests-badge = EKSAKT `ℹ tests N` (self-audit `countTestCases`); verifiser `readmeCheck.passed:true`/`mismatches:[]`. 1012 nå. Versjon KUN i plugin.json + README version-badge.
|
||||
- **docs-gate (non-marketplace) krever ≥3 ikke-ws-linjer i README OG CLAUDE.md for `feat:`.** Path-guard blokkerer Write på `.claude-plugin/`+settings/hooks → hermetisk `mkdtemp`. Push-vindu: hverdag 20–23, helg fritt.
|
||||
- **Scanner-tillegg = strip, IKKE re-seed (POST-B2).** Ny scanner → bevar frozen v5.0.0 via `strip-added-scanner.mjs`; re-seed KUN SC-5 default-output. Re-seed av frozen baker inn B2-hotspot-triple + claudeMd-drift. SKL-presedensen (re-seed) er utdatert.
|
||||
- **To motsatte byte-stabilitets-regimer:** manifest = env-aware → mode-equivalence. TOK + scanner-array embedder i scan-orch+posture → frozen-v5.0.0-komparatorer i **json/raw-backcompat + cli/posture/scan-orch-humanizer**. Grep ALLE før blast-radius-antakelse. To strip-helpere nå: `strip-hotspot-load-pattern` (B2-felt) + `strip-added-scanner` (C-scanner).
|
||||
- **OST keyer:** 001=user/project (ikke plugin), 002=plugin-only (`force-for-plugin` er plugin-only per docs). 003 case-insensitiv built-in-match.
|
||||
- **manifest har INGEN findings** → ingen humanizer/fix-tittelkobling (kun finding-scannere). Skill on-demand (body); harness-config = external.
|
||||
- tests-badge = EKSAKT `ℹ tests N` (self-audit `countTestCases`); `readmeCheck.passed:true`/`mismatches:[]`. **1023** nå, scanner **14**. Versjon KUN i plugin.json + README version-badge.
|
||||
- **docs-gate (non-marketplace) krever ≥3 ikke-ws-linjer i README OG CLAUDE.md for `feat:`.** Path-guard blokkerer Write på `.claude-plugin/`+settings/hooks. Push-vindu: hverdag 20–23, helg fritt.
|
||||
|
||||
## Scope-gjerde
|
||||
v5.6 B (B1+B2) godkjent (GO) og landet. **C er GO-godkjent (2026-06-20) — start direkte neste sesjon, ikke spør om GO på nytt.** Release-cut (tag + catalog-bump) + v5.7 + andre repos = egne GO.
|
||||
Hele v5.6 (Foundation+B+C) godkjent (GO) og landet på main. **Release-cut (tag + catalog-bump) + v5.7 D + andre repos = egne GO.** Ikke start release uten klarsignal.
|
||||
|
||||
## Kontinuitet
|
||||
Tre lag: STATE.md (tracked) + auto-memory + CLAUDE.md. Oppdater DENNE som siste handling.
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ User-impact category (added to each finding as `userImpactCategory`, derived fro
|
|||
|
||||
| Label | Scanners |
|
||||
|-------|----------|
|
||||
| Configuration mistake | CML, SET, HKV, RUL, MCP, IMP, PLH |
|
||||
| Configuration mistake | CML, SET, HKV, RUL, MCP, IMP, PLH, OST |
|
||||
| Conflict | CNF, COL |
|
||||
| Wasted tokens | TOK, CPS |
|
||||
| Dead config | DIS |
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
|
|||
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries (low). (1) Tools in BOTH `permissions.deny` AND `permissions.allow` — deny wins; dominance is param-aware and treats the `Tool(*)` deny-all glob as equivalent to a bare deny (covers a bare allow). (2) Unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — CC accepts allow globs only after a literal glob-free `mcp__<server>__` prefix. Predicates shared with CNF live in `lib/permission-rules.mjs` |
|
||||
| `collision-scanner.mjs` | COL | Cross-plugin skill name collisions (low); user-vs-plugin overlaps (medium); `details.namespaces` payload |
|
||||
| `skill-listing-scanner.mjs` | SKL | (1) `CA-SKL-001` (medium): active skill descriptions over the verified 1,536-char listing cap (CC 2.1.105) → silently truncated in the model's skill listing. (2) `CA-SKL-002` (low): sum of active descriptions (each counted up to the cap) over the listing budget (~2% of context, CC 2.1.32), anchored on a conservative 200k window with a calibration note that the budget scales 5× on 1M-context models — leads with the measured sum, an estimate not telemetry. HOME-scoped (all user + plugin skills). Remediation surfaces `disableBundledSkills` / `skillOverrides` / trim. Distinct lens from TOK pattern F (project-local 500-char bloat heuristic) |
|
||||
| `output-style-scanner.mjs` | OST | (1) `CA-OST-001` (medium): a user/project custom output style not setting `keep-coding-instructions: true` (defaults false) → silently strips Claude Code's built-in software-engineering instructions when active (V10). (2) `CA-OST-002` (low): a **plugin** style with `force-for-plugin: true` auto-applies and overrides the user's `outputStyle` (V11; plugin-styles-only per docs). (3) `CA-OST-003` (medium): a settings `outputStyle` matching no built-in (Default/Explanatory/Learning/Proactive, case-insensitive) nor discovered custom style → dead config (CC falls back to default). Reads each style's frontmatter via `parseFrontmatter`; fixture-gated (silent with no output styles). New scanner family in v5.6 C (count 13→14) |
|
||||
|
||||
## Scanner Lib (`scanners/lib/`)
|
||||
|
||||
|
|
@ -42,7 +43,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
|
|||
| `active-config-reader.mjs` | Read-only inventory: readActiveConfig(), detectGitRoot(), walkClaudeMdCascade(), readClaudeJsonProjectSlice() (longest-prefix match), enumeratePlugins(), enumerateSkills(), readActiveHooks(), readActiveMcpServers() (with cache → package.json tool-count fallback), estimateTokens() (v5: `'mcp'` kind = 500 + toolCount × 200) |
|
||||
| `tokenizer-api.mjs` | Anthropic `count_tokens` wrapper for `--accurate-tokens` (v5 N5); 5s AbortController timeout, exponential 429 backoff, key masking |
|
||||
| `humanizer.mjs` | Plain-language output translator (v5.1.0): `humanizeFinding`, `humanizeFindings`, `humanizeEnvelope`, `computeRelevanceContext`. Pure functions; never mutate inputs. Adds `userImpactCategory`, `userActionLanguage`, `relevanceContext` fields and replaces title/description/recommendation when a translation exists. Bypassed by `--raw` and `--json` paths. |
|
||||
| `humanizer-data.mjs` | TRANSLATIONS table for 14 scanner prefixes (CML/SET/HKV/RUL/MCP/IMP/CNF/COL/TOK/CPS/DIS/GAP/PLH/SKL). Three-step lookup: exact title → regex pattern → `_default` → fall through to original |
|
||||
| `humanizer-data.mjs` | TRANSLATIONS table for 15 scanner prefixes (CML/SET/HKV/RUL/MCP/IMP/CNF/COL/TOK/CPS/DIS/GAP/PLH/SKL/OST). Three-step lookup: exact title → regex pattern → `_default` → fall through to original |
|
||||
|
||||
## Action Engines (`scanners/`)
|
||||
|
||||
|
|
|
|||
|
|
@ -820,4 +820,34 @@ export const TRANSLATIONS = {
|
|||
recommendation: 'See the details for which skill to trim or turn off.',
|
||||
},
|
||||
},
|
||||
|
||||
// ─────────────────────────────────────────────────────────────
|
||||
// OST — Output-Style Validation
|
||||
// Category: Configuration mistake
|
||||
// ─────────────────────────────────────────────────────────────
|
||||
OST: {
|
||||
static: {
|
||||
'Custom output style removes built-in coding instructions': {
|
||||
title: 'A custom output style turns off Claude\'s coding know-how',
|
||||
description: 'This style replaces Claude Code\'s built-in coding guidance with only your own text, so while it\'s active Claude forgets how to scope changes, comment, and verify work. The setting that keeps that guidance is off by default.',
|
||||
recommendation: 'Add `keep-coding-instructions: true` to the top of the style file to keep that guidance. If you meant to drop it for a non-coding style, leave it as is.',
|
||||
},
|
||||
'Plugin output style overrides your selected output style': {
|
||||
title: 'A plugin is forcing its own output style on you',
|
||||
description: 'This plugin applies its own output style automatically whenever it\'s on, replacing the one you picked. If two plugins both do this, the first one to load wins.',
|
||||
recommendation: 'If you didn\'t want this, turn off the plugin or remove the force setting from its style. Otherwise there\'s nothing to do — the plugin works this way on purpose.',
|
||||
},
|
||||
'Configured output style does not exist': {
|
||||
title: 'Your chosen output style can\'t be found',
|
||||
description: 'Your settings point to an output style that doesn\'t exist by that name, so Claude Code quietly uses the default instead. The style you wanted never takes effect.',
|
||||
recommendation: 'Check the spelling against your styles (built-ins are Default, Explanatory, Learning, Proactive), add the missing style file, or remove the setting.',
|
||||
},
|
||||
},
|
||||
patterns: [],
|
||||
_default: {
|
||||
title: 'Something about your output styles needs a look',
|
||||
description: 'A check on your output styles flagged something worth reviewing.',
|
||||
recommendation: 'See the details for which output style to adjust.',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@ const SCANNER_TO_CATEGORY = {
|
|||
DIS: 'Dead config',
|
||||
GAP: 'Missed opportunity',
|
||||
PLH: 'Configuration mistake',
|
||||
OST: 'Configuration mistake',
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -168,6 +168,7 @@ const SCANNER_AREA_MAP = {
|
|||
SKL: 'Token Efficiency',
|
||||
DIS: 'Settings',
|
||||
COL: 'Plugin Hygiene',
|
||||
OST: 'Settings',
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
|
|||
179
scanners/output-style-scanner.mjs
Normal file
179
scanners/output-style-scanner.mjs
Normal file
|
|
@ -0,0 +1,179 @@
|
|||
/**
|
||||
* OST Scanner — Output-style validation (v5.6 C)
|
||||
*
|
||||
* Output styles are live (the standalone `/output-style` command was removed in
|
||||
* v2.1.91; styles are now managed via `/config`). They are the most surprising
|
||||
* steering surface because they rewrite the system prompt:
|
||||
*
|
||||
* CA-OST-001 A custom (user/project) output style that does NOT set
|
||||
* `keep-coding-instructions: true` → when active, Claude Code
|
||||
* REMOVES its built-in software-engineering instructions (how to
|
||||
* scope changes, write comments, verify work) and keeps only the
|
||||
* style's text. `keep-coding-instructions` defaults to false, so
|
||||
* this is the headline footgun. Severity medium.
|
||||
*
|
||||
* CA-OST-002 A PLUGIN output style with `force-for-plugin: true` → Claude
|
||||
* Code auto-applies it whenever the plugin is enabled, OVERRIDING
|
||||
* the user's selected `outputStyle`. If several enabled plugins set
|
||||
* it, the first loaded wins. Severity low (awareness). Note:
|
||||
* `force-for-plugin` is plugin-styles-only per the docs, so this
|
||||
* keys on `source === 'plugin'` — a user/project style cannot
|
||||
* trigger the override (it would simply be ignored).
|
||||
*
|
||||
* CA-OST-003 A settings `outputStyle` value that matches no built-in and no
|
||||
* discovered custom style → dead config: Claude Code falls back to
|
||||
* the default style, so the configured behavior is silently not
|
||||
* applied. Severity medium.
|
||||
*
|
||||
* Every claim traces to a CONFIRMED row of docs/v5.5-steering-model-plan.md
|
||||
* (V9/V10/V11/V12), verified against code.claude.com/docs/en/output-styles and
|
||||
* .../plugins-reference. The scanner is fixture-gated: with no output styles and
|
||||
* no `outputStyle` setting it emits nothing (keeps the SC-5 snapshot byte-stable).
|
||||
*
|
||||
* Zero external dependencies.
|
||||
*/
|
||||
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { readActiveConfig } from './lib/active-config-reader.mjs';
|
||||
import { parseFrontmatter, parseJson } from './lib/yaml-parser.mjs';
|
||||
|
||||
const SCANNER = 'OST';
|
||||
|
||||
// Built-in output styles, verified against code.claude.com/docs/en/output-styles.
|
||||
// Compared case-insensitively so OST-003 never false-flags a valid built-in.
|
||||
const BUILTIN_STYLES = ['default', 'explanatory', 'learning', 'proactive'];
|
||||
|
||||
/**
|
||||
* Read + parse the frontmatter of each enumerated output style once.
|
||||
* @param {Array<object>} styles - readActiveConfig().outputStyles entries
|
||||
*/
|
||||
async function withFrontmatter(styles) {
|
||||
const out = [];
|
||||
for (const s of styles) {
|
||||
let frontmatter = null;
|
||||
try {
|
||||
frontmatter = parseFrontmatter(await readFile(s.path, 'utf-8')).frontmatter;
|
||||
} catch { /* unreadable → treat as no frontmatter */ }
|
||||
out.push({ ...s, frontmatter });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the effective `outputStyle` setting from the cascade (user → project →
|
||||
* local; later scope wins). Returns null when unset everywhere.
|
||||
* @param {object} activeConfig
|
||||
* @returns {Promise<{value:string, scope:string, path:string} | null>}
|
||||
*/
|
||||
async function resolveOutputStyleSetting(activeConfig) {
|
||||
const cascade = (activeConfig.settings && activeConfig.settings.cascade) || [];
|
||||
let resolved = null;
|
||||
for (const entry of cascade) {
|
||||
if (!entry.exists || !entry.path) continue;
|
||||
let json = null;
|
||||
try { json = parseJson(await readFile(entry.path, 'utf-8')); } catch { continue; }
|
||||
if (json && typeof json.outputStyle === 'string' && json.outputStyle.trim()) {
|
||||
resolved = { value: json.outputStyle.trim(), scope: entry.scope, path: entry.path };
|
||||
}
|
||||
}
|
||||
return resolved;
|
||||
}
|
||||
|
||||
/**
|
||||
* Main scanner entry point.
|
||||
* @param {string} targetPath - repo root to scan
|
||||
* @param {object} _discovery - unused (OST reads the active config cascade itself)
|
||||
*/
|
||||
export async function scan(targetPath, _discovery) {
|
||||
const start = Date.now();
|
||||
const findings = [];
|
||||
|
||||
const activeConfig = await readActiveConfig(targetPath);
|
||||
const styles = await withFrontmatter(activeConfig.outputStyles || []);
|
||||
|
||||
// CA-OST-001 — user/project custom style missing keep-coding-instructions:true.
|
||||
for (const s of styles) {
|
||||
if (s.source !== 'project' && s.source !== 'user') continue;
|
||||
const kci = s.frontmatter ? s.frontmatter.keep_coding_instructions : undefined;
|
||||
if (kci === true) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Custom output style removes built-in coding instructions',
|
||||
description:
|
||||
`The ${s.source} output style "${s.name}" does not set ` +
|
||||
'`keep-coding-instructions: true`. While this style is active, Claude Code ' +
|
||||
'drops its built-in software-engineering instructions — how to scope changes, ' +
|
||||
'write comments, and verify work — and keeps only this style\'s text. The ' +
|
||||
'frontmatter flag defaults to false, so the strip is easy to miss.',
|
||||
file: s.path,
|
||||
evidence:
|
||||
`output_style="${s.name}"; source=${s.source}; ` +
|
||||
`keep-coding-instructions=${kci === undefined ? 'unset (default false)' : String(kci)}`,
|
||||
recommendation:
|
||||
'To keep Claude Code\'s software-engineering behavior while applying this style, ' +
|
||||
'add `keep-coding-instructions: true` to the frontmatter. If the strip is ' +
|
||||
'intentional (a non-coding persona), no change is needed.',
|
||||
category: 'output-styles',
|
||||
}));
|
||||
}
|
||||
|
||||
// CA-OST-002 — plugin output style with force-for-plugin:true (overrides user choice).
|
||||
for (const s of styles) {
|
||||
if (s.source !== 'plugin') continue;
|
||||
const ffp = s.frontmatter ? s.frontmatter.force_for_plugin : undefined;
|
||||
if (ffp !== true) continue;
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
title: 'Plugin output style overrides your selected output style',
|
||||
description:
|
||||
`The plugin "${s.pluginName}" ships an output style "${s.name}" with ` +
|
||||
'`force-for-plugin: true`, so Claude Code applies it automatically whenever the ' +
|
||||
'plugin is enabled — overriding whatever `outputStyle` you selected. When more ' +
|
||||
'than one enabled plugin does this, the first one loaded wins.',
|
||||
file: s.path,
|
||||
evidence: `output_style="${s.name}"; source=plugin:${s.pluginName}; force-for-plugin=true`,
|
||||
recommendation:
|
||||
'If you did not expect this style, disable the plugin or remove ' +
|
||||
'`force-for-plugin: true` from its output style. This is awareness only — the ' +
|
||||
'plugin is behaving as designed.',
|
||||
category: 'output-styles',
|
||||
}));
|
||||
}
|
||||
|
||||
// CA-OST-003 — settings outputStyle resolving to a non-existent style (dead config).
|
||||
const resolved = await resolveOutputStyleSetting(activeConfig);
|
||||
if (resolved) {
|
||||
const known = new Set([
|
||||
...BUILTIN_STYLES,
|
||||
...styles.map(s => String(s.name).toLowerCase()),
|
||||
]);
|
||||
if (!known.has(resolved.value.toLowerCase())) {
|
||||
const customNames = styles.map(s => s.name);
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.medium,
|
||||
title: 'Configured output style does not exist',
|
||||
description:
|
||||
`Your ${resolved.scope} settings set \`outputStyle: "${resolved.value}"\`, but no ` +
|
||||
'built-in or discovered custom style has that name. Claude Code falls back to the ' +
|
||||
'default style, so the output behavior you configured is silently never applied.',
|
||||
file: resolved.path,
|
||||
evidence:
|
||||
`outputStyle="${resolved.value}"; scope=${resolved.scope}; ` +
|
||||
`builtins=[${BUILTIN_STYLES.join(', ')}]; ` +
|
||||
`known_custom=[${customNames.join(', ')}]`,
|
||||
recommendation:
|
||||
'Fix the value to match an existing style (built-ins: Default, Explanatory, ' +
|
||||
'Learning, Proactive), create the missing style under `.claude/output-styles/`, ' +
|
||||
'or remove the `outputStyle` setting.',
|
||||
category: 'output-styles',
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
return scannerResult(SCANNER, 'ok', findings, styles.length, Date.now() - start);
|
||||
}
|
||||
|
|
@ -29,6 +29,7 @@ import { scan as scanCachePrefix } from './cache-prefix-scanner.mjs';
|
|||
import { scan as scanDisabledInSchema } from './disabled-in-schema-scanner.mjs';
|
||||
import { scan as scanCollision } from './collision-scanner.mjs';
|
||||
import { scan as scanSkillListing } from './skill-listing-scanner.mjs';
|
||||
import { scan as scanOutputStyle } from './output-style-scanner.mjs';
|
||||
|
||||
// Directory names that identify test fixture / example directories
|
||||
const FIXTURE_DIR_NAMES = ['tests', 'examples', '__tests__', 'test-fixtures'];
|
||||
|
|
@ -64,6 +65,7 @@ const SCANNERS = [
|
|||
{ name: 'DIS', fn: scanDisabledInSchema, label: 'Disabled-In-Schema' },
|
||||
{ name: 'COL', fn: scanCollision, label: 'Plugin Skill Collision' },
|
||||
{ name: 'SKL', fn: scanSkillListing, label: 'Skill-Listing Budget' },
|
||||
{ name: 'OST', fn: scanOutputStyle, label: 'Output-Style Validation' },
|
||||
];
|
||||
|
||||
/**
|
||||
|
|
|
|||
62
tests/helpers/strip-added-scanner.mjs
Normal file
62
tests/helpers/strip-added-scanner.mjs
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
/**
|
||||
* v5.6 C added the OST (Output-Style Validation) scanner — a 14th orchestrated
|
||||
* scanner. The frozen v5.0.0 byte-equal baselines predate it (13 scanners), so
|
||||
* this strips the additive OST scanner entry before comparison, the same way
|
||||
* strip-hotspot-load-pattern.mjs handles B2's additive hotspot fields. Stripping
|
||||
* (rather than re-seeding the frozen snapshots) keeps the strongest invariant:
|
||||
* the ORIGINAL 13 scanners still emit byte-identical output, and the frozen
|
||||
* snapshots stay free of post-v5.0.0 drift (the hotspot triple, ancestor token
|
||||
* counts) that re-seeding would bake in.
|
||||
*
|
||||
* Removing a zero-finding scanner entry only moves `aggregate.scanners_ok`
|
||||
* (OST is always status 'ok' and emits nothing on the deterministic fixture, so
|
||||
* total_findings / counts / risk_score / verdict are unchanged). The strip
|
||||
* decrements scanners_ok to match.
|
||||
*
|
||||
* Mutates in place and returns the payload for chaining inside a normalizer:
|
||||
* return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
*/
|
||||
|
||||
const ADDED_SCANNERS = new Set(['OST']);
|
||||
|
||||
function stripFromEnvelope(env) {
|
||||
if (!env || typeof env !== 'object' || !Array.isArray(env.scanners)) return;
|
||||
let removedOk = 0;
|
||||
const kept = [];
|
||||
for (const s of env.scanners) {
|
||||
if (ADDED_SCANNERS.has(s.scanner)) {
|
||||
if (s.status === 'ok') removedOk++;
|
||||
continue;
|
||||
}
|
||||
kept.push(s);
|
||||
}
|
||||
env.scanners = kept;
|
||||
if (env.aggregate && typeof env.aggregate.scanners_ok === 'number') {
|
||||
env.aggregate.scanners_ok -= removedOk;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip added (post-v5.0.0) scanner entries from a CLI payload / envelope —
|
||||
* both top-level (`scanners`) and nested (`scannerEnvelope.scanners`).
|
||||
* @template T
|
||||
* @param {T} payload
|
||||
* @returns {T}
|
||||
*/
|
||||
export function stripAddedScanners(payload) {
|
||||
if (!payload || typeof payload !== 'object') return payload;
|
||||
stripFromEnvelope(payload);
|
||||
if (payload.scannerEnvelope) stripFromEnvelope(payload.scannerEnvelope);
|
||||
return payload;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove `[OST] …` per-scanner progress lines from a captured stderr scorecard,
|
||||
* so a 14-scanner live run matches the frozen 13-scanner stderr snapshot.
|
||||
* @param {string} text
|
||||
* @returns {string}
|
||||
*/
|
||||
export function stripAddedScannerStderr(text) {
|
||||
if (typeof text !== 'string') return text;
|
||||
return text.replace(/^[ \t]*\[OST\][^\n]*\n/gm, '');
|
||||
}
|
||||
|
|
@ -30,6 +30,7 @@ import { readFile, writeFile, access, mkdir } from 'node:fs/promises';
|
|||
import { homedir } from 'node:os';
|
||||
import { hermeticEnv, HERMETIC_HOME } from './helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from './helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from './helpers/strip-added-scanner.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -103,7 +104,7 @@ function normalizeScanOrchestrator(env) {
|
|||
}
|
||||
}
|
||||
stripAncestorDerived(out);
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizePosture(p) {
|
||||
|
|
@ -120,7 +121,7 @@ function normalizePosture(p) {
|
|||
}
|
||||
stripAncestorDerived(out.scannerEnvelope);
|
||||
}
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizeTokenHotspots(p) {
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import { TRANSLATIONS } from '../../scanners/lib/humanizer-data.mjs';
|
|||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const FORBIDDEN_PATH = resolve(__dirname, '..', 'lint-forbidden-words.json');
|
||||
|
||||
const EXPECTED_SCANNERS = ['CML', 'SET', 'HKV', 'RUL', 'MCP', 'IMP', 'CNF', 'GAP', 'TOK', 'CPS', 'DIS', 'COL', 'PLH', 'SKL'];
|
||||
const EXPECTED_SCANNERS = ['CML', 'SET', 'HKV', 'RUL', 'MCP', 'IMP', 'CNF', 'GAP', 'TOK', 'CPS', 'DIS', 'COL', 'PLH', 'SKL', 'OST'];
|
||||
|
||||
function stripBacktickSpans(s) {
|
||||
return s.replace(/`[^`]*`/g, '');
|
||||
|
|
@ -24,7 +24,7 @@ test('TRANSLATIONS exports an object', () => {
|
|||
assert.ok(TRANSLATIONS !== null);
|
||||
});
|
||||
|
||||
test('TRANSLATIONS covers all 14 expected scanner prefixes', () => {
|
||||
test('TRANSLATIONS covers all 15 expected scanner prefixes', () => {
|
||||
for (const prefix of EXPECTED_SCANNERS) {
|
||||
assert.ok(TRANSLATIONS[prefix], `missing scanner prefix: ${prefix}`);
|
||||
}
|
||||
|
|
@ -156,7 +156,7 @@ test('no translated string contains tier3 jargon (outside backtick spans)', asyn
|
|||
|
||||
test('CML, SET, HKV, RUL, MCP, IMP, GAP, TOK, PLH have non-empty static maps', () => {
|
||||
// These scanners produce findings with titles we documented. Empty static map suggests missed coverage.
|
||||
for (const prefix of ['CML', 'SET', 'HKV', 'RUL', 'MCP', 'IMP', 'GAP', 'TOK', 'PLH']) {
|
||||
for (const prefix of ['CML', 'SET', 'HKV', 'RUL', 'MCP', 'IMP', 'GAP', 'TOK', 'PLH', 'OST']) {
|
||||
const count = Object.keys(TRANSLATIONS[prefix].static).length;
|
||||
assert.ok(count > 0, `${prefix}.static is empty — expected at least 1 translated title`);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -157,8 +157,8 @@ test('humanizeFinding falls back to "FYI" for unknown severity', () => {
|
|||
|
||||
// ─── userImpactCategory ────────────────────────────────────────────────
|
||||
|
||||
test('humanizeFinding sets category Configuration mistake for CML/SET/HKV/RUL/MCP/IMP/PLH', () => {
|
||||
for (const s of ['CML', 'SET', 'HKV', 'RUL', 'MCP', 'IMP', 'PLH']) {
|
||||
test('humanizeFinding sets category Configuration mistake for CML/SET/HKV/RUL/MCP/IMP/PLH/OST', () => {
|
||||
for (const s of ['CML', 'SET', 'HKV', 'RUL', 'MCP', 'IMP', 'PLH', 'OST']) {
|
||||
const out = humanizeFinding(makeFinding({ scanner: s }));
|
||||
assert.equal(out.userImpactCategory, 'Configuration mistake', `${s} should map to Configuration mistake`);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ import { readFile, writeFile, access, mkdir } from 'node:fs/promises';
|
|||
import { homedir } from 'node:os';
|
||||
import { hermeticEnv, HERMETIC_HOME } from './helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from './helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from './helpers/strip-added-scanner.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -98,7 +99,7 @@ function normalizeScanOrchestrator(env) {
|
|||
}
|
||||
}
|
||||
stripAncestorDerived(out);
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizePosture(p) {
|
||||
|
|
@ -115,7 +116,7 @@ function normalizePosture(p) {
|
|||
}
|
||||
stripAncestorDerived(out.scannerEnvelope);
|
||||
}
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
function normalizeTokenHotspots(p) {
|
||||
|
|
@ -275,8 +276,8 @@ describe('SC-7 --raw posture stderr scorecard verbatim', () => {
|
|||
}
|
||||
const expected = await readFile(stderrSnapshotPath, 'utf8');
|
||||
assert.equal(
|
||||
normalizeStderrDurations(stderr.trim()),
|
||||
normalizeStderrDurations(expected.trim()),
|
||||
normalizeStderrDurations(stripAddedScannerStderr(stderr.trim())),
|
||||
normalizeStderrDurations(stripAddedScannerStderr(expected.trim())),
|
||||
'posture --raw stderr must reproduce the v5.0.0 scorecard verbatim (apart from durations)',
|
||||
);
|
||||
});
|
||||
|
|
|
|||
169
tests/scanners/output-style-scanner.test.mjs
Normal file
169
tests/scanners/output-style-scanner.test.mjs
Normal file
|
|
@ -0,0 +1,169 @@
|
|||
/**
|
||||
* OST scanner tests — output-style validation (v5.6 C).
|
||||
*
|
||||
* Each finding has a positive fixture (fires) and a negative fixture (silent),
|
||||
* per the v5.5+ plan acceptance criteria. The scanner reads the active config
|
||||
* (project + user + plugin output styles, settings cascade) via HOME, so every
|
||||
* test builds a hermetic temp HOME + repo, overrides process.env.HOME, runs,
|
||||
* then restores and cleans up — never touching real user state.
|
||||
*
|
||||
* CA-OST-001 user/project custom style missing keep-coding-instructions:true (medium)
|
||||
* CA-OST-002 plugin output style with force-for-plugin:true (low)
|
||||
* CA-OST-003 settings outputStyle resolving to a non-existent style (medium)
|
||||
*/
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { join } from 'node:path';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan } from '../../scanners/output-style-scanner.mjs';
|
||||
|
||||
function tmp(prefix) {
|
||||
return mkdtemp(join(tmpdir(), `ca-ost-${prefix}-`));
|
||||
}
|
||||
|
||||
async function writeStyle(dir, name, frontmatter) {
|
||||
await mkdir(dir, { recursive: true });
|
||||
const fm = frontmatter ?? `name: ${name}\ndescription: test style`;
|
||||
await writeFile(join(dir, `${name}.md`), `---\n${fm}\n---\n\nStyle body for ${name}.\n`, 'utf-8');
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a hermetic HOME + repo from a spec, run the OST scanner, restore + clean.
|
||||
* @param {object} opts
|
||||
* @param {Array<{name:string, frontmatter?:string}>} [opts.projectStyles]
|
||||
* @param {Array<{name:string, frontmatter?:string}>} [opts.userStyles]
|
||||
* @param {Array<{plugin:string, name:string, frontmatter?:string}>} [opts.pluginStyles]
|
||||
* @param {object} [opts.projectSettings]
|
||||
*/
|
||||
async function runOst(opts = {}) {
|
||||
const home = await tmp('home');
|
||||
const repo = await tmp('repo');
|
||||
|
||||
for (const s of opts.projectStyles || []) {
|
||||
await writeStyle(join(repo, '.claude', 'output-styles'), s.name, s.frontmatter);
|
||||
}
|
||||
for (const s of opts.userStyles || []) {
|
||||
await writeStyle(join(home, '.claude', 'output-styles'), s.name, s.frontmatter);
|
||||
}
|
||||
for (const s of opts.pluginStyles || []) {
|
||||
const root = join(home, '.claude', 'plugins', 'marketplaces', 'mp', 'plugins', s.plugin);
|
||||
await mkdir(join(root, '.claude-plugin'), { recursive: true });
|
||||
await writeFile(
|
||||
join(root, '.claude-plugin', 'plugin.json'),
|
||||
JSON.stringify({ name: s.plugin, version: '1.0.0' }),
|
||||
'utf-8',
|
||||
);
|
||||
await writeStyle(join(root, 'output-styles'), s.name, s.frontmatter);
|
||||
}
|
||||
if (opts.projectSettings) {
|
||||
await mkdir(join(repo, '.claude'), { recursive: true });
|
||||
await writeFile(join(repo, '.claude', 'settings.json'), JSON.stringify(opts.projectSettings), 'utf-8');
|
||||
}
|
||||
|
||||
const originalHome = process.env.HOME;
|
||||
const originalProfile = process.env.USERPROFILE;
|
||||
process.env.HOME = home;
|
||||
process.env.USERPROFILE = home;
|
||||
resetCounter();
|
||||
try {
|
||||
return await scan(repo, { files: [] });
|
||||
} finally {
|
||||
process.env.HOME = originalHome;
|
||||
process.env.USERPROFILE = originalProfile;
|
||||
await rm(home, { recursive: true, force: true });
|
||||
await rm(repo, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
const has = (result, re) => result.findings.find(f => re.test(String(f.title || '')));
|
||||
|
||||
describe('OST scanner — basic structure', () => {
|
||||
it('reports scanner prefix OST', async () => {
|
||||
const result = await runOst({ projectStyles: [{ name: 'plain' }] });
|
||||
assert.equal(result.scanner, 'OST');
|
||||
assert.equal(result.status, 'ok');
|
||||
});
|
||||
|
||||
it('finding IDs match CA-OST-NNN', async () => {
|
||||
const result = await runOst({ projectStyles: [{ name: 'plain' }] });
|
||||
for (const f of result.findings) {
|
||||
assert.match(f.id, /^CA-OST-\d{3}$/);
|
||||
}
|
||||
});
|
||||
|
||||
it('is fixture-gated — no output styles, no settings → zero findings', async () => {
|
||||
const result = await runOst({});
|
||||
assert.equal(result.findings.length, 0, `expected silent; got: ${result.findings.map(f => f.title).join(' | ')}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('OST scanner — CA-OST-001 keep-coding-instructions', () => {
|
||||
it('flags a project custom style missing keep-coding-instructions:true (medium)', async () => {
|
||||
const result = await runOst({ projectStyles: [{ name: 'myco' }] });
|
||||
const f = has(result, /coding instructions/i);
|
||||
assert.ok(f, `expected OST-001; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
assert.equal(f.severity, 'medium');
|
||||
assert.match(String(f.file), /myco\.md$/);
|
||||
});
|
||||
|
||||
it('flags a user custom style missing the flag (source=user in evidence)', async () => {
|
||||
const result = await runOst({ userStyles: [{ name: 'mine' }] });
|
||||
const f = has(result, /coding instructions/i);
|
||||
assert.ok(f);
|
||||
assert.match(String(f.evidence), /source=user/);
|
||||
});
|
||||
|
||||
it('does NOT flag a style that sets keep-coding-instructions:true', async () => {
|
||||
const result = await runOst({
|
||||
projectStyles: [{ name: 'keeper', frontmatter: 'name: keeper\ndescription: test\nkeep-coding-instructions: true' }],
|
||||
});
|
||||
assert.equal(has(result, /coding instructions/i), undefined,
|
||||
`expected silent; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('OST scanner — CA-OST-002 force-for-plugin', () => {
|
||||
it('flags a plugin style with force-for-plugin:true (low, names the plugin)', async () => {
|
||||
const result = await runOst({
|
||||
pluginStyles: [{ plugin: 'styler', name: 'forced', frontmatter: 'name: forced\ndescription: test\nforce-for-plugin: true' }],
|
||||
});
|
||||
const f = has(result, /override/i);
|
||||
assert.ok(f, `expected OST-002; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
assert.equal(f.severity, 'low');
|
||||
assert.match(String(f.evidence), /styler/);
|
||||
});
|
||||
|
||||
it('does NOT flag a plugin style without force-for-plugin (and OST-001 stays project/user-only)', async () => {
|
||||
const result = await runOst({
|
||||
pluginStyles: [{ plugin: 'styler', name: 'plain', frontmatter: 'name: plain\ndescription: test' }],
|
||||
});
|
||||
assert.equal(result.findings.length, 0,
|
||||
`expected silent for a plain plugin style; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('OST scanner — CA-OST-003 dead outputStyle', () => {
|
||||
it('flags settings.outputStyle that resolves to no known style (medium)', async () => {
|
||||
const result = await runOst({ projectSettings: { outputStyle: 'Ghostwriter' } });
|
||||
const f = has(result, /does not exist/i);
|
||||
assert.ok(f, `expected OST-003; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
assert.equal(f.severity, 'medium');
|
||||
assert.match(String(f.evidence), /Ghostwriter/);
|
||||
});
|
||||
|
||||
it('does NOT flag a built-in style name (Explanatory)', async () => {
|
||||
const result = await runOst({ projectSettings: { outputStyle: 'Explanatory' } });
|
||||
assert.equal(has(result, /does not exist/i), undefined);
|
||||
});
|
||||
|
||||
it('does NOT flag an outputStyle matching an existing custom style', async () => {
|
||||
const result = await runOst({
|
||||
projectStyles: [{ name: 'myco', frontmatter: 'name: myco\ndescription: test\nkeep-coding-instructions: true' }],
|
||||
projectSettings: { outputStyle: 'myco' },
|
||||
});
|
||||
assert.equal(has(result, /does not exist/i), undefined,
|
||||
`expected no dead-config finding; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
});
|
||||
});
|
||||
|
|
@ -7,6 +7,7 @@ import { promisify } from 'node:util';
|
|||
import { readFile, unlink } from 'node:fs/promises';
|
||||
import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners, stripAddedScannerStderr } from '../helpers/strip-added-scanner.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -39,12 +40,16 @@ function normalizePosture(p) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
/** Strip time-varying durations (Xms) so progress lines compare verbatim across runs. */
|
||||
/**
|
||||
* Strip time-varying durations (Xms) so progress lines compare verbatim across
|
||||
* runs, and drop the additive [OST] progress line (v5.6 C) so a 14-scanner live
|
||||
* run matches the frozen 13-scanner v5.0.0 stderr scorecard.
|
||||
*/
|
||||
function normalizeStderr(s) {
|
||||
return s.replace(/\(\d+ms\)/g, '(0ms)');
|
||||
return stripAddedScannerStderr(s).replace(/\(\d+ms\)/g, '(0ms)');
|
||||
}
|
||||
|
||||
async function runPosture(flags) {
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ import { promisify } from 'node:util';
|
|||
import { readFile, unlink } from 'node:fs/promises';
|
||||
import { hermeticEnv } from '../helpers/hermetic-home.mjs';
|
||||
import { stripHotspotLoadPattern } from '../helpers/strip-hotspot-load-pattern.mjs';
|
||||
import { stripAddedScanners } from '../helpers/strip-added-scanner.mjs';
|
||||
|
||||
const exec = promisify(execFile);
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
|
|
@ -37,7 +38,7 @@ function normalizeEnvelope(env) {
|
|||
}
|
||||
}
|
||||
}
|
||||
return stripHotspotLoadPattern(out);
|
||||
return stripAddedScanners(stripHotspotLoadPattern(out));
|
||||
}
|
||||
|
||||
async function runOrchestrator(flags) {
|
||||
|
|
@ -102,7 +103,7 @@ describe('scan-orchestrator humanizer wiring (Step 5)', () => {
|
|||
assert.ok(actual.meta, 'meta present');
|
||||
assert.ok(Array.isArray(actual.scanners), 'scanners array present');
|
||||
assert.ok(actual.aggregate, 'aggregate present');
|
||||
assert.equal(actual.scanners.length, 13, 'all 13 scanners present');
|
||||
assert.equal(actual.scanners.length, 14, 'all 14 scanners present');
|
||||
});
|
||||
|
||||
it('preserves scanner shape (scanner/status/findings/counts)', async () => {
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
{
|
||||
"kind": "text",
|
||||
"payload": "`[CML] CLAUDE.md Linter`: 1 finding(s) (0ms)\n `[SET] Settings Validator`: 0 finding(s) (0ms)\n `[HKV] Hook Validator`: 0 finding(s) (0ms)\n `[RUL] Rules Validator`: 0 finding(s) (0ms)\n `[MCP] MCP Config Validator`: 0 finding(s) (0ms)\n `[IMP] Import Resolver`: 0 finding(s) (0ms)\n `[CNF] Conflict Detector`: 0 finding(s) (0ms)\n `[GAP] Feature Gap Scanner`: 17 finding(s) (0ms)\n `[TOK] Token Hotspots`: 1 finding(s) (0ms)\n `[CPS] Cache-Prefix Stability`: 0 finding(s) (0ms)\n `[DIS] Disabled-In-Schema`: 0 finding(s) (0ms)\n `[COL] Plugin Skill Collision`: 0 finding(s) (0ms)\n `[SKL] Skill-Listing Budget`: 0 finding(s) (0ms)\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Configuration health\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n Health: A (98/100) — Healthy setup, only minor polish needed\n 9 areas reviewed\n\n Area scores\n ───────────\n `CLAUDE.md` ........... A (90) `Settings` ............ A (100)\n `Hooks` ............... A (100) `Rules` ............... A (100)\n `MCP` ................. A (100) `Imports` ............. A (100)\n `Conflicts` ........... A (100) `Token Efficiency` .... A (90)\n `Plugin Hygiene` ...... A (100)\n\n 17 ways you could get more out of Claude Code — see /config-audit feature-gap\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
"payload": "`[CML] CLAUDE.md Linter`: 1 finding(s) (0ms)\n `[SET] Settings Validator`: 0 finding(s) (0ms)\n `[HKV] Hook Validator`: 0 finding(s) (0ms)\n `[RUL] Rules Validator`: 0 finding(s) (0ms)\n `[MCP] MCP Config Validator`: 0 finding(s) (0ms)\n `[IMP] Import Resolver`: 0 finding(s) (0ms)\n `[CNF] Conflict Detector`: 0 finding(s) (0ms)\n `[GAP] Feature Gap Scanner`: 17 finding(s) (0ms)\n `[TOK] Token Hotspots`: 1 finding(s) (0ms)\n `[CPS] Cache-Prefix Stability`: 0 finding(s) (0ms)\n `[DIS] Disabled-In-Schema`: 0 finding(s) (0ms)\n `[COL] Plugin Skill Collision`: 0 finding(s) (0ms)\n `[SKL] Skill-Listing Budget`: 0 finding(s) (0ms)\n `[OST] Output-Style Validation`: 0 finding(s) (0ms)\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n Configuration health\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n Health: A (98/100) — Healthy setup, only minor polish needed\n 9 areas reviewed\n\n Area scores\n ───────────\n `CLAUDE.md` ........... A (90) `Settings` ............ A (100)\n `Hooks` ............... A (100) `Rules` ............... A (100)\n `MCP` ................. A (100) `Imports` ............. A (100)\n `Conflicts` ........... A (100) `Token Efficiency` .... A (90)\n `Plugin Hygiene` ...... A (100)\n\n 17 ways you could get more out of Claude Code — see /config-audit feature-gap\n\n━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -568,6 +568,20 @@
|
|||
"low": 0,
|
||||
"info": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"scanner": "OST",
|
||||
"status": "ok",
|
||||
"files_scanned": 0,
|
||||
"duration_ms": 0,
|
||||
"findings": [],
|
||||
"counts": {
|
||||
"critical": 0,
|
||||
"high": 0,
|
||||
"medium": 0,
|
||||
"low": 0,
|
||||
"info": 0
|
||||
}
|
||||
}
|
||||
],
|
||||
"aggregate": {
|
||||
|
|
@ -582,7 +596,7 @@
|
|||
"risk_score": 11,
|
||||
"risk_band": "Medium",
|
||||
"verdict": "PASS",
|
||||
"scanners_ok": 12,
|
||||
"scanners_ok": 13,
|
||||
"scanners_error": 0,
|
||||
"scanners_skipped": 1
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue