Compare commits
No commits in common. "main" and "v5.12.5" have entirely different histories.
23 changed files with 90 additions and 1078 deletions
92
CLAUDE.md
92
CLAUDE.md
|
|
@ -1,8 +1,13 @@
|
||||||
# Config-Audit Plugin
|
# Config-Audit Plugin
|
||||||
|
|
||||||
Claude Code Configuration Intelligence — know if your config is correct, find what could improve it, fix it automatically. Three pillars: **Health** (deterministic scanners), **Opportunities** (context-aware recommendations), **Action** (auto-fix with backup/rollback).
|
Claude Code Configuration Intelligence — know if your configuration is correct, find what could improve it, fix it automatically.
|
||||||
|
|
||||||
Per-command flags, patterns, and feature lists live in `README.md` and `/config-audit help`. This file carries what's invariant for working on the plugin.
|
## What this plugin does
|
||||||
|
|
||||||
|
Analyzes and optimizes Claude Code configuration across three pillars:
|
||||||
|
- **Health** — Deterministic scanners verify correctness, consistency, and completeness
|
||||||
|
- **Opportunities** — Context-aware recommendations for features that could benefit your project
|
||||||
|
- **Action** — Auto-fix with backup/rollback
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
|
|
@ -10,15 +15,15 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
||||||
|
|
||||||
| Command | Description |
|
| Command | Description |
|
||||||
|---------|-------------|
|
|---------|-------------|
|
||||||
| `/config-audit` | Full audit with auto-scope detection |
|
| `/config-audit` | Full audit with auto-scope detection (no setup needed) |
|
||||||
| `/config-audit posture` | A-F health scorecard (10 quality areas) |
|
| `/config-audit posture` | Quick health scorecard (A-F grades, 10 quality areas incl. Token Efficiency, Plugin Hygiene) |
|
||||||
| `/config-audit tokens` | Prompt-cache-aware token hotspots, each tagged with its load pattern; cache-aware |
|
| `/config-audit tokens` | prompt-cache-aware token hotspots (8 patterns: cache-breaking, redundant perms, deep imports, oversized cascade, bloated SKILL.md desc, MCP tool-schema budget, MCP tool-schema deferral, stale plugin-cache disk-cleanup), each ranked hotspot tagged with its load pattern (always / on-demand / external) — **cache-aware** (stale `~/.claude/plugins/cache` versions excluded by default; only each plugin's active version counts; `--no-exclude-cache` for the full walk), optional `--accurate-tokens` API calibration, `--with-telemetry-recipe` cache-hit recipe pointer |
|
||||||
| `/config-audit manifest` | Ranked table of every token source + always-loaded subtotal |
|
| `/config-audit manifest` | Ranked table of every token source (CLAUDE.md, rules, agents, skills, output styles, MCP, hooks) sorted by estimated tokens, each tagged with its load pattern (always-loaded / on-demand / external) + an always-loaded subtotal ("tokens that enter context every turn") |
|
||||||
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact |
|
| `/config-audit feature-gap` | Context-aware feature recommendations grouped by impact (incl. a conditional `disableBundledSkills` lever when the active skill listing is over budget — remediation companion to SKL `CA-SKL-002`) |
|
||||||
| `/config-audit optimize` | Mechanism-fit lens (procedure→skill, lifecycle→hook, path→rule, never→permission). Agent-driven, **not byte-stable** |
|
| `/config-audit optimize` | Optimization lens (mechanism-fit) — config that works but fits a better mechanism: procedure→skill (CA-OPT-001, deterministic), lifecycle→hook / unscoped path→rule / "never"→permission (prose-judgment via opus `optimization-lens-agent`). Hybrid motor; every finding cites a best-practices-register rule. Agent-driven, **not byte-stable** |
|
||||||
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
| `/config-audit fix` | Auto-fix deterministic issues with backup + verification |
|
||||||
| `/config-audit rollback` | Restore configuration from backup |
|
| `/config-audit rollback` | Restore configuration from backup |
|
||||||
| `/config-audit plan` | Create action plan from findings |
|
| `/config-audit plan` | Create action plan from audit findings |
|
||||||
| `/config-audit implement` | Execute plan with backups + auto-verify |
|
| `/config-audit implement` | Execute plan with backups + auto-verify |
|
||||||
| `/config-audit help` | Show all commands |
|
| `/config-audit help` | Show all commands |
|
||||||
|
|
||||||
|
|
@ -28,9 +33,9 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
||||||
|---------|-------------|
|
|---------|-------------|
|
||||||
| `/config-audit drift` | Compare current config against saved baseline |
|
| `/config-audit drift` | Compare current config against saved baseline |
|
||||||
| `/config-audit plugin-health` | Audit plugin structure, frontmatter, cross-plugin coherence |
|
| `/config-audit plugin-health` | Audit plugin structure, frontmatter, cross-plugin coherence |
|
||||||
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/MCP/hooks/CLAUDE.md (with token estimates) |
|
| `/config-audit whats-active` | Read-only inventory of plugins, skills, MCP, hooks, CLAUDE.md active for a repo (with token estimates) |
|
||||||
| `/config-audit knowledge-refresh` | Refresh the best-practices register (stale check + web poll). Human-approved writes; **not byte-stable** |
|
| `/config-audit knowledge-refresh` | Keep the best-practices register fresh — deterministic stale check (sources older than ~90d) + web candidate poll (CC changelog + Anthropic blog); **human-approved writes only** (Verifiseringsplikt). The "living" half of the knowledge base. Web/judgment-driven, **not byte-stable** |
|
||||||
| `/config-audit campaign` | Machine-wide audit ledger + token bill across repos. Human-approved writes; **not byte-stable** |
|
| `/config-audit campaign` | Machine-wide audit campaign — durable ledger ABOVE sessions: per-repo lifecycle (pending→audited→planned→implemented) + machine-wide roll-up by severity + **machine-wide always-loaded token bill** (`refresh-tokens` live cross-repo sweep — shared global layer counted once + per-repo deltas) + cross-repo prioritized backlog + plan **export** (drop a planned repo's plan into its own `docs/`), resumable across sessions. Read-only report (campaign-cli) + **human-approved** writes via deterministic write/export CLIs. THIN: tracks+routes state, reuses existing implement/rollback for execution. Judgment-driven, **not byte-stable** |
|
||||||
| `/config-audit discover` | Run discovery phase only |
|
| `/config-audit discover` | Run discovery phase only |
|
||||||
| `/config-audit analyze` | Run analysis phase only |
|
| `/config-audit analyze` | Run analysis phase only |
|
||||||
| `/config-audit interview` | Gather user preferences (opt-in) |
|
| `/config-audit interview` | Gather user preferences (opt-in) |
|
||||||
|
|
@ -46,43 +51,68 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
||||||
| planner-agent | Create action plan | opus | yellow | Read, Glob, Write |
|
| planner-agent | Create action plan | opus | yellow | Read, Glob, Write |
|
||||||
| implementer-agent | Execute changes | sonnet | magenta | Read, Write, Edit, Bash, Glob |
|
| implementer-agent | Execute changes | sonnet | magenta | Read, Write, Edit, Bash, Glob |
|
||||||
| verifier-agent | Verify results | sonnet | purple | Read, Glob, Grep |
|
| verifier-agent | Verify results | sonnet | purple | Read, Glob, Grep |
|
||||||
| feature-gap-agent | Feature recommendations | opus | green | Read, Glob, Grep, Write |
|
| feature-gap-agent | Context-aware feature recommendations | opus | green | Read, Glob, Grep, Write |
|
||||||
| optimization-lens-agent | Mechanism-fit precision gate | opus | orange | Read, Glob, Grep, Write |
|
| optimization-lens-agent | Mechanism-fit precision gate (prose-judgment lens cases) | opus | orange | Read, Glob, Grep, Write |
|
||||||
|
|
||||||
## Hooks
|
## Hooks
|
||||||
|
|
||||||
| Event | Script | Purpose |
|
| Event | Script | Purpose |
|
||||||
|-------|--------|---------|
|
|-------|--------|---------|
|
||||||
| PreToolUse | `auto-backup-config.mjs` | Backup config files before Edit/Write |
|
| PreToolUse | `auto-backup-config.mjs` | Auto-backup config files before Edit/Write |
|
||||||
| PostToolUse | `post-edit-verify.mjs` | Verify after Edit/Write, block on new critical/high |
|
| PostToolUse | `post-edit-verify.mjs` | Verify config files after Edit/Write, block on new critical/high |
|
||||||
| SessionStart | `session-start.mjs` | Check for active (unfinished) sessions |
|
| SessionStart | `session-start.mjs` | Checks for active (unfinished) sessions |
|
||||||
| Stop | `stop-session-reminder.mjs` | Remind about current session phase |
|
| Stop | `stop-session-reminder.mjs` | Reminds about current session phase |
|
||||||
|
|
||||||
## Reference docs (read on demand)
|
## Reference docs (read on demand)
|
||||||
|
|
||||||
- `docs/scanner-internals.md` — scanner inventory, lib modules, action engines, knowledge base, per-scanner/per-block implementation notes (design rationale, primary-source verification, byte-stability lessons)
|
- **Scanner inventory, lib modules, action engines, knowledge base, per-scanner/per-block implementation notes:** `docs/scanner-internals.md`
|
||||||
- `docs/humanizer.md` — plain-language output (v5.1.0), humanizer vocabularies, output modes
|
- **Plain-language output (v5.1.0), humanizer vocabularies, output modes:** `docs/humanizer.md`
|
||||||
|
|
||||||
## Plain-Language Output (v5.1.0)
|
## Plain-Language Output (v5.1.0) — summary
|
||||||
|
|
||||||
Default output of all commands routes through `humanizeEnvelope` (`lib/humanizer.mjs`), decorating each finding with `userImpactCategory`, `userActionLanguage`, and `relevanceContext`. `--raw` and `--json` bypass the humanizer for byte-stable v5.0.0 output. Full detail: `docs/humanizer.md`.
|
Default output of all 18 commands routes through `humanizeEnvelope` from `lib/humanizer.mjs`. Findings get three decorated fields:
|
||||||
|
|
||||||
|
- `userImpactCategory` — Configuration mistake / Conflict / Wasted tokens / Dead config / Missed opportunity
|
||||||
|
- `userActionLanguage` — Fix this now / Fix soon / Fix when convenient / Optional cleanup / FYI (derived from severity)
|
||||||
|
- `relevanceContext` — `affects-everyone` (default) / `affects-this-machine-only` (`*.local.*` files) / `test-fixture-no-impact`
|
||||||
|
|
||||||
|
`--raw` bypasses the humanizer for byte-stable v5.0.0 output. `--json` is also byte-stable. Full detail and Wave 5 lessons: `docs/humanizer.md`.
|
||||||
|
|
||||||
## Suppressions
|
## Suppressions
|
||||||
|
|
||||||
Create `.config-audit-ignore` at project root — one exact ID or glob per line (`CA-SET-003`, `CA-GAP-*`). Suppressed findings are tracked in the envelope's `suppressed_findings` for audit trail. Disable with `--no-suppress`.
|
Create `.config-audit-ignore` at project root to suppress known findings:
|
||||||
|
```
|
||||||
|
CA-SET-003 # Exact ID
|
||||||
|
CA-GAP-* # Glob pattern (all GAP findings)
|
||||||
|
```
|
||||||
|
Suppressed findings tracked in envelope's `suppressed_findings` for audit trail. Disable with `--no-suppress`.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
Workflow: `/config-audit → discover + analyze (auto) → plan → implement → verify`. Auto-detects scope from git context; override with `full|repo|home|current`; `--delta` for incremental. Session state lives under `~/.claude/config-audit/sessions/{id}/` (scope.yaml, discovery.json, state.yaml, findings/, analysis-report.md, action-plan.md, backups/, implementation-log.md).
|
### Workflow
|
||||||
|
```
|
||||||
|
/config-audit → discover + analyze (auto) → plan → implement → verify
|
||||||
|
```
|
||||||
|
Default: auto-detects scope from git context. Override with `/config-audit full|repo|home|current`. Delta mode: `--delta` (incremental).
|
||||||
|
|
||||||
Finding ID format: `CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`.
|
### Session Directory
|
||||||
|
```
|
||||||
|
~/.claude/config-audit/sessions/{session-id}/
|
||||||
|
├── scope.yaml, discovery.json, state.yaml
|
||||||
|
├── findings/, analysis-report.md, action-plan.md
|
||||||
|
├── backups/, implementation-log.md
|
||||||
|
└── interview.md (if interview run)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Finding ID Format
|
||||||
|
`CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-DIS-001`, `CA-COL-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
Enforced conventions live in `.claude/rules/` (auto-loaded as project instructions):
|
Enforced project conventions live in `.claude/rules/` (auto-loaded as project instructions):
|
||||||
- `ux-rules.md` — output/narration/formatting for all commands (never dump raw JSON, narrate before each step, space-separated command suggestions)
|
|
||||||
|
- `ux-rules.md` — output/narration/formatting rules for all commands (never dump raw JSON, narrate before each step, space-separated command suggestions)
|
||||||
- `command-development.md` — required command frontmatter + `plugin:action` naming
|
- `command-development.md` — required command frontmatter + `plugin:action` naming
|
||||||
- `agent-development.md` — agent frontmatter + "when to use" conventions
|
|
||||||
- `state-management.md` — update `state.yaml` after every workflow phase
|
- `state-management.md` — update `state.yaml` after every workflow phase
|
||||||
|
|
||||||
Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{SCANNER}-{NNN}` ID format; byte-stable CLIs are verified against frozen `tests/snapshots/v5.0.0/` baselines.
|
Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{SCANNER}-{NNN}` ID format; byte-stable CLIs are verified against frozen `tests/snapshots/v5.0.0/` baselines.
|
||||||
|
|
@ -93,10 +123,12 @@ Coding style: scanners are zero-dependency Node ESM; new findings use the `CA-{S
|
||||||
node --test 'tests/**/*.test.mjs'
|
node --test 'tests/**/*.test.mjs'
|
||||||
```
|
```
|
||||||
|
|
||||||
Test fixtures in `tests/fixtures/`. Per-scanner and per-build-block implementation notes (design rationale, primary-source verification, byte-stability lessons) live in `docs/scanner-internals.md` → **Implementation notes**.
|
1279 tests across 72 test files (23 lib + 39 scanner + 1 hook + 1 agent + 3 commands + 1 knowledge + 4 top-level). Test fixtures in `tests/fixtures/`. Top-level humanizer tests: `json-backcompat.test.mjs`, `raw-backcompat.test.mjs`, `scenario-read-test.test.mjs`, `snapshot-default-output.test.mjs`.
|
||||||
|
|
||||||
|
Per-scanner and per-build-block implementation notes (design rationale, primary-source verification, byte-stability lessons) live in `docs/scanner-internals.md` → **Implementation notes**.
|
||||||
|
|
||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
- Session directories accumulate — use `/config-audit cleanup` to manage
|
- Session directories accumulate — use `/config-audit cleanup` to manage
|
||||||
- Scanners run on Node.js ≥ 18 (uses node:test, node:fs/promises)
|
- Scanners run on Node.js >= 18 (uses node:test, node:fs/promises)
|
||||||
- Plugin CLAUDE.md files in node_modules should be excluded via scope
|
- Plugin CLAUDE.md files in node_modules should be excluded via scope
|
||||||
|
|
|
||||||
|
|
@ -51,16 +51,11 @@ In `--raw` mode, fall back to v5.0.0 severity prefiks and verbatim scanner title
|
||||||
5. **Identify optimizations**: Rules to globalize, missing configs, orphaned files
|
5. **Identify optimizations**: Rules to globalize, missing configs, orphaned files
|
||||||
6. **Security scan**: Aggregate secret warnings, check for insecure patterns
|
6. **Security scan**: Aggregate secret warnings, check for insecure patterns
|
||||||
7. **CLAUDE.md quality assessment**: Score each file against rubric, assign letter grades
|
7. **CLAUDE.md quality assessment**: Score each file against rubric, assign letter grades
|
||||||
8. **Generate report**: Compose the comprehensive markdown report — group findings by `userImpactCategory`, lead with `userActionLanguage`
|
8. **Generate report**: Write comprehensive markdown report — group findings by `userImpactCategory`, lead with `userActionLanguage`
|
||||||
|
|
||||||
## Output
|
## Output
|
||||||
|
|
||||||
Return the complete report as your final message — do not write it to a file
|
Write to: `~/.claude/config-audit/sessions/{session-id}/analysis-report.md`
|
||||||
yourself. The Claude Code subagent harness instructs agents not to write
|
|
||||||
report/analysis files; your text output IS the deliverable. The orchestrating
|
|
||||||
command saves your returned report verbatim to
|
|
||||||
`~/.claude/config-audit/sessions/{session-id}/analysis-report.md` for the
|
|
||||||
downstream plan/interview/status phases.
|
|
||||||
|
|
||||||
**Output MUST NOT exceed 300 lines.** Prioritize findings by severity. Use tables, not prose.
|
**Output MUST NOT exceed 300 lines.** Prioritize findings by severity. Use tables, not prose.
|
||||||
|
|
||||||
|
|
@ -188,4 +183,4 @@ Verify report: all findings referenced, recommendations actionable, severity lev
|
||||||
|
|
||||||
- Process findings in memory (typically < 1MB total)
|
- Process findings in memory (typically < 1MB total)
|
||||||
- Generate report in single pass
|
- Generate report in single pass
|
||||||
- No file modifications (read-only; the report is returned as your final message)
|
- No file modifications (read-only except report output)
|
||||||
|
|
|
||||||
|
|
@ -146,11 +146,6 @@ Move content from one file to another.
|
||||||
|
|
||||||
Append to: `~/.claude/config-audit/sessions/{session-id}/implementation-log.md`
|
Append to: `~/.claude/config-audit/sessions/{session-id}/implementation-log.md`
|
||||||
|
|
||||||
**Append discipline (shared log):** other implementer agents may be writing this
|
|
||||||
log concurrently. ALWAYS append your entry with a Bash `>>` heredoc;
|
|
||||||
NEVER use the Write or Edit tool on the log file — a full-file Write silently
|
|
||||||
clobbers entries other agents appended after you read the file.
|
|
||||||
|
|
||||||
### Success
|
### Success
|
||||||
|
|
||||||
```markdown
|
```markdown
|
||||||
|
|
|
||||||
|
|
@ -60,21 +60,12 @@ Agent(subagent_type: "config-audit:analyzer-agent")
|
||||||
raw severity. The humanizer already replaced jargon-heavy
|
raw severity. The humanizer already replaced jargon-heavy
|
||||||
title/description/recommendation strings with plain-language
|
title/description/recommendation strings with plain-language
|
||||||
equivalents — render them verbatim, do not paraphrase.
|
equivalents — render them verbatim, do not paraphrase.
|
||||||
Return the complete report as your final message. Do not write it
|
Output to: ~/.claude/config-audit/sessions/{session-id}/analysis-report.md
|
||||||
to a file — the orchestrating command saves it to the session directory.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Step 4: Save the report
|
### Step 4: Present summary
|
||||||
|
|
||||||
The agent returns the complete report as its final message — the Claude Code
|
After the agent completes, read the generated report and show a brief summary:
|
||||||
subagent harness instructs agents not to write report/analysis files themselves,
|
|
||||||
so the command must persist it. Write the returned report verbatim (no edits,
|
|
||||||
no truncation) to `~/.claude/config-audit/sessions/{session-id}/analysis-report.md`
|
|
||||||
using the Write tool. Downstream phases (`plan`, `interview`, `status`) read this file.
|
|
||||||
|
|
||||||
### Step 5: Present summary
|
|
||||||
|
|
||||||
After saving the report, show a brief summary:
|
|
||||||
|
|
||||||
```markdown
|
```markdown
|
||||||
### Analysis Complete
|
### Analysis Complete
|
||||||
|
|
@ -93,6 +84,6 @@ Full report: `~/.claude/config-audit/sessions/{session-id}/analysis-report.md`
|
||||||
- **`/config-audit fix`** — Auto-fix deterministic issues right away
|
- **`/config-audit fix`** — Auto-fix deterministic issues right away
|
||||||
```
|
```
|
||||||
|
|
||||||
### Step 6: Update state
|
### Step 5: Update state
|
||||||
|
|
||||||
Update `state.yaml` with `current_phase: "analyze"`, `next_phase: "plan"`.
|
Update `state.yaml` with `current_phase: "analyze"`, `next_phase: "plan"`.
|
||||||
|
|
|
||||||
|
|
@ -78,8 +78,6 @@ Agent(subagent_type: "config-audit:implementer-agent")
|
||||||
fields from the action plan (the planner already rendered them) —
|
fields from the action plan (the planner already rendered them) —
|
||||||
do not re-derive severity prose. Append result to:
|
do not re-derive severity prose. Append result to:
|
||||||
~/.claude/config-audit/sessions/{session-id}/implementation-log.md
|
~/.claude/config-audit/sessions/{session-id}/implementation-log.md
|
||||||
Append with Bash `>>` (heredoc) — NEVER the Write tool on this log;
|
|
||||||
parallel agents share it and a full-file Write clobbers their entries.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Show progress between groups using the humanized titles already present in the action plan:
|
Show progress between groups using the humanized titles already present in the action plan:
|
||||||
|
|
|
||||||
|
|
@ -1,242 +0,0 @@
|
||||||
# Brief — Delete-and-Rebuild (config subtraction)
|
|
||||||
|
|
||||||
**Status:** BRIEF, not a plan. No code, no chunk breakdown, no version number committed.
|
|
||||||
Written 2026-07-29 so a session starting Thursday evening has a durable starting point.
|
|
||||||
STATE.md is gitignored in this repo, so this file — not STATE — is the record.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Trigger and its provenance
|
|
||||||
|
|
||||||
The operator relayed a third-party YouTube summary (Hyper Automation Labs) of a talk
|
|
||||||
Boris Cherny reportedly gave at Y Combinator Startup School, one day after Opus 5
|
|
||||||
shipped. Claims attributed to him in that summary:
|
|
||||||
|
|
||||||
- Anthropic deleted ~80 % of Claude Code's own system prompt when Opus 5 landed.
|
|
||||||
- Advice to users: every six months, delete your CLAUDE.md, your skills, your hooks —
|
|
||||||
see what the model does.
|
|
||||||
- Rebuild method: delete everything, use it, add one line back only when the model
|
|
||||||
stumbles on the same thing repeatedly.
|
|
||||||
- The model measured *slightly more intelligent* with the built-in prompts stripped.
|
|
||||||
|
|
||||||
**Two levels of confidence here, and they must not be collapsed** (updated 2026-07-29
|
|
||||||
after the operator corrected the first draft):
|
|
||||||
|
|
||||||
- **Attribution — confirmed.** The operator watched the recording and identifies Boris
|
|
||||||
Cherny on stage. This is not a channel's claim about who spoke; it is direct
|
|
||||||
observation by the operator. The talk happened and it is him.
|
|
||||||
- **The verbatim figures — still summary-level.** "80 % of the system prompt", "the
|
|
||||||
model measured slightly more intelligent without the prompts", the Bun numbers: these
|
|
||||||
reach us through the channel's editing, not through a primary transcript. They are
|
|
||||||
plausible and consistent with §2, and they are not quoted as fact anywhere below.
|
|
||||||
|
|
||||||
So if this becomes a `BP-*` entry in the best-practices register, the source field reads
|
|
||||||
*"Boris Cherny, YC Startup School talk (attribution confirmed by operator); figures via
|
|
||||||
third-party summary, not primary-verified"* — not "unverified", and not a bare citation
|
|
||||||
either. Getting a primary transcript for the figures is a nice-to-have, never a
|
|
||||||
prerequisite: the feature is argued from this repo's own logic (§3), and this repo has
|
|
||||||
one scar from treating a plausible quote as load-bearing fact («Fable low ≈ Opus high»,
|
|
||||||
fabricated, rejected 2026-07-14).
|
|
||||||
|
|
||||||
**What the operator has affirmed as in scope:** delete CLAUDE.md, then rebuild by adding
|
|
||||||
back what the model needs help with — *starting with what it must have*. That last clause
|
|
||||||
is not a detail; it is the design constraint in §6.0.
|
|
||||||
|
|
||||||
## 2. Verified ground truth (checked against the local CLI, 2026-07-29)
|
|
||||||
|
|
||||||
These *are* facts, and they are what make an empirical variant buildable:
|
|
||||||
|
|
||||||
| Fact | How verified |
|
|
||||||
|---|---|
|
|
||||||
| `claude --bare` exists — "Minimal mode: skip hooks, LSP, plugin sync, attribution, auto-memory, background prefetches, keychain reads, and CLAUDE.md auto-discovery. Sets `CLAUDE_CODE_SIMPLE=1`." | `claude --help` |
|
|
||||||
| `claude --system-prompt <prompt>` — replaces the session system prompt | `claude --help` |
|
|
||||||
| `--append-system-prompt`, `--add-dir`, `--setting-sources <user,project,local>`, `--plugin-dir`, `--settings`, `--agents` — all present and all usable to compose an isolated config for a controlled run | `claude --help` |
|
|
||||||
|
|
||||||
So the `CLAUDE_CODE_SIMPLE=1` env var the video calls "undocumented" is, in this CLI
|
|
||||||
version, a documented flag (`--bare`). That matters: an A/B ablation harness would not
|
|
||||||
need an undocumented hook.
|
|
||||||
|
|
||||||
## 3. Why this fits this repo (the argument that does *not* depend on §1)
|
|
||||||
|
|
||||||
The plugin has three pillars — Health, Opportunities, Action. Every existing command
|
|
||||||
answers a question on the **addition** axis:
|
|
||||||
|
|
||||||
- `feature-gap` — what could you add?
|
|
||||||
- `optimize` — what would fit a better mechanism?
|
|
||||||
- `posture` / `tokens` / `manifest` — how good/expensive is what you have?
|
|
||||||
- `fix` / `implement` — apply changes.
|
|
||||||
|
|
||||||
**Nothing answers the subtraction question: what is no longer earning its rent?**
|
|
||||||
`feature-gap` has no inverse. That is a real hole, independent of who said what on
|
|
||||||
a stage.
|
|
||||||
|
|
||||||
Two further reasons this belongs *here* specifically:
|
|
||||||
|
|
||||||
1. **Nothing in the repo measures instruction age.** Verified by grep over `scanners/`:
|
|
||||||
`stale` appears only for knowledge-register entry age (`lib/knowledge-refresh.mjs`)
|
|
||||||
and for stale plugin-cache versions (`scan-orchestrator.mjs`, M-BUG-11). No scanner
|
|
||||||
touches `git blame`, `mtime`, or the vintage of a CLAUDE.md block. An instruction
|
|
||||||
written for Sonnet 3.5 and an instruction written last week are indistinguishable to
|
|
||||||
every current scanner.
|
|
||||||
2. **Deleting bravely is only sane if you can undo it.** That is already pillar three:
|
|
||||||
`lib/backup.mjs`, `rollback-engine.mjs`, `auto-backup-config.mjs` (PreToolUse), and
|
|
||||||
`drift`'s `saveBaseline` / `loadBaseline` / `diffEnvelopes`. The safety net exists;
|
|
||||||
the feature that would use it does not. This is arguably the strongest framing:
|
|
||||||
*config-audit is already the infrastructure that makes "delete it and see" a
|
|
||||||
measurement rather than a gamble.*
|
|
||||||
|
|
||||||
## 4. Reusable machinery (do not rebuild these)
|
|
||||||
|
|
||||||
| Need | Already exists |
|
|
||||||
|---|---|
|
|
||||||
| Snapshot config before deleting | `scanners/lib/baseline.mjs` (`saveBaseline`), used by `drift` |
|
|
||||||
| Diff before/after | `diffEnvelopes` in the same module |
|
|
||||||
| Backup + restore individual files w/ sha256 manifest | `scanners/lib/backup.mjs`, `scanners/rollback-engine.mjs` |
|
|
||||||
| Cost of each source, always-loaded subtotal | `scanners/manifest.mjs`, `token-hotspots.mjs` |
|
|
||||||
| Human-approved-writes command pattern | `knowledge-refresh`, `campaign` (both non-byte-stable by design) |
|
|
||||||
|
|
||||||
## 5. Candidate shapes (sketches — pick on Thursday, do not pre-commit)
|
|
||||||
|
|
||||||
All three inherit the floor constraint in §6.0: whatever the shape, load-bearing local
|
|
||||||
facts are never deletion candidates, and a rebuild restores them first. A shape that
|
|
||||||
cannot express that distinction is disqualified regardless of how cheap it is.
|
|
||||||
|
|
||||||
**A. Deterministic vintage scanner (`CA-VIN-*`).** Per instruction block in CLAUDE.md /
|
|
||||||
rules / skills / hooks: age from git history, plus a compensatory-phrasing signal
|
|
||||||
(blocks that exist to correct model behaviour — "ALWAYS", "never forget", "read the
|
|
||||||
whole file first", "don't guess"). Output: ranked deletion candidates with age + token
|
|
||||||
cost + why it looks compensatory. Cheapest, most testable, fits the existing scanner
|
|
||||||
architecture, and composes with `tokens`/`manifest` for the payoff figure.
|
|
||||||
*Caveat:* `~/.claude` is not git-tracked, so age there falls back to `mtime`, which is
|
|
||||||
weaker evidence. Project-level CLAUDE.md/rules get real git ages.
|
|
||||||
|
|
||||||
**B. Protocol command (`/config-audit rebuild`).** The delete → live with it → earned
|
|
||||||
re-add ledger loop, spanning sessions: archive current config, record what was archived,
|
|
||||||
and maintain a ledger where a line only returns when the operator records that the model
|
|
||||||
actually stumbled on it. Highest fidelity to the source idea; needs cross-session state
|
|
||||||
(the `sessions/` machinery already exists) and is inherently not byte-stable.
|
|
||||||
|
|
||||||
**C. Measured ablation harness.** Use `--bare` + `--system-prompt`/`--add-dir` to run the
|
|
||||||
same prompt with and without a block and compare. Closest to a real verifier, most
|
|
||||||
expensive in tokens, weakest determinism. Probably a later `--experimental` sub-mode of
|
|
||||||
A or B rather than its own command.
|
|
||||||
|
|
||||||
Likely landing: **A first** (deterministic, testable, immediately useful), with B as the
|
|
||||||
workflow that consumes A's output. C stays a documented idea until A+B exist.
|
|
||||||
|
|
||||||
## 6. Hard constraints (carry these into any implementation)
|
|
||||||
|
|
||||||
### 6.0 The floor: compensatory vs load-bearing instructions
|
|
||||||
|
|
||||||
**This is the invariant all three shapes in §5 must respect, and it is what makes the
|
|
||||||
feature safe to ship at all.** Not every line in a CLAUDE.md is the same kind of thing:
|
|
||||||
|
|
||||||
| Class | What it is | Test | Disposition |
|
|
||||||
|---|---|---|---|
|
|
||||||
| **Compensatory** | An instruction correcting model *behaviour* — "read the whole file first", "don't guess", "ALWAYS verify", "think before you code" | A smarter model would do this unprompted | **Deletion candidate.** Returns only when earned. |
|
|
||||||
| **Load-bearing** | A *local fact* the model cannot derive at any capability level — "never GitHub, only Forgejo at git.fromaitochitta.com", "system bash is 3.2, no `declare -A`", "test with `node --test 'tests/**/*.test.mjs'`", "`~/.claude` is not git-tracked" | No amount of intelligence produces this from the codebase alone | **Floor. Never a deletion candidate.** |
|
|
||||||
|
|
||||||
Model capability erodes the first class and does nothing to the second. That is the whole
|
|
||||||
mechanism behind the source idea — and it means "delete your CLAUDE.md" is only correct
|
|
||||||
for one of the two classes. A tool that treats them alike would delete the operator's
|
|
||||||
Forgejo constraint because Opus 5 "is smart enough now", which is a category error: the
|
|
||||||
model isn't failing at intelligence there, it simply cannot know.
|
|
||||||
|
|
||||||
**Consequence for the rebuild ordering.** A rebuild is not one undifferentiated
|
|
||||||
add-back-on-stumble loop. It is three tiers:
|
|
||||||
|
|
||||||
1. **Floor — goes back immediately, no trial period.** Load-bearing facts. The config is
|
|
||||||
never in a state where these are absent; a "delete everything" that drops them is a
|
|
||||||
broken experiment, not a brave one.
|
|
||||||
2. **Earned — out, returns only on repeated observed stumbling.** Compensatory
|
|
||||||
instructions that turn out to still be needed by *this* model.
|
|
||||||
3. **Dead — out and never missed.** The payoff, measurable in tokens via `manifest`.
|
|
||||||
|
|
||||||
A third class sits deliberately outside the axis: **policy prohibitions** ("never commit
|
|
||||||
secrets"). These may well be things the model would honour unprompted, but their cost of
|
|
||||||
being wrong is asymmetric and they are cheap. They stay in the floor by decision, not by
|
|
||||||
classification. Do not let a "the model knows this now" argument reach them.
|
|
||||||
|
|
||||||
The hard part is tier 1 vs tier 2, and that classification — not the deletion mechanics —
|
|
||||||
is the real engineering problem in this brief. Precision is asymmetric: a missed dead line
|
|
||||||
costs a few tokens per turn; a deleted load-bearing line costs a wrong remote, a broken
|
|
||||||
bash script, or a lost afternoon.
|
|
||||||
|
|
||||||
### 6.1 Existing repo constraints
|
|
||||||
|
|
||||||
- **`~/.claude` is NOT git-tracked.** A literal `rm` there is unrecoverable. Archive by
|
|
||||||
`mv` into `_archive/`, never delete. This is already an inviolable rule in STATE.md.
|
|
||||||
- **`settings.json` is pathguard-protected** — no Edit/Write. Use `jq` + temp file +
|
|
||||||
atomic `mv`, with operator OK ([[settings-json-pathguard-write]]).
|
|
||||||
- **New scanner ⇒ the 7-step byte-stability checklist** ([[adding-scanner-byte-stability]]):
|
|
||||||
scanner + orchestrator + scoring area-map + strip-added-scanner + humanizer count +
|
|
||||||
SC-5 + humanizer wiring (`SCANNER_TO_CATEGORY` entry and `TRANSLATIONS.static` per RAW
|
|
||||||
title — M-16/M-17). Frozen `tests/snapshots/v5.0.0/` must stay untouched.
|
|
||||||
- **TDD is inviolable** — red tests before implementation, including for .md contracts.
|
|
||||||
- **`feat:` commits require non-trivial diffs in both README.md and CLAUDE.md**
|
|
||||||
([[docs-gate-feat-requires-readme-claudemd]]).
|
|
||||||
|
|
||||||
## 7. Open questions for Thursday
|
|
||||||
|
|
||||||
1. Scope of the deletion candidate set: user-level `~/.claude` only, project only, or both?
|
|
||||||
(Age evidence is much stronger for project-level, per §5A.)
|
|
||||||
2. **The core question, given §6.0:** can compensatory-vs-load-bearing be classified
|
|
||||||
deterministically with acceptable precision, or does it need the agent layer (like
|
|
||||||
`optimize`'s precision-gated `optimization-lens-agent`, which stays silent when
|
|
||||||
unsure)? Note the two signals are independent — a load-bearing fact can be old, and a
|
|
||||||
compensatory instruction can be new — so age alone can never carry this call. Prior:
|
|
||||||
a deterministic pre-filter feeding a precision-gated judge, which is exactly the
|
|
||||||
`optimize` architecture already in the repo.
|
|
||||||
**Design the §8 fasit around the ambiguous middle, not the poles.** The four named
|
|
||||||
must-survive items are clear-cut and any mechanism will get them right; the gate is
|
|
||||||
really decided by blocks like "Conventional Commits: `type(scope): beskrivelse`"
|
|
||||||
(local convention, or a nag the model would follow anyway?), "commit ofte med
|
|
||||||
beskrivende meldinger" (pure behaviour correction?), or the model-routing rubric
|
|
||||||
(a table of local policy that reads like advice). Include 3–5 such blocks
|
|
||||||
deliberately. A fasit built only from obvious cases will pass a tool that fails on
|
|
||||||
real config.
|
|
||||||
3. Does this ship as its own command, or as a `--subtract` mode of `optimize`?
|
|
||||||
Both are defensible; command count is already 21.
|
|
||||||
4. ~~Ordering against the existing queue.~~ **Decided 2026-07-29:** the operator
|
|
||||||
prioritized this work ahead of pipeline step 4 (`rollback`), to start Thursday
|
|
||||||
2026-07-30. Do not re-litigate. The open part is only what follows it — the
|
|
||||||
prior order stands underneath: step 4 `rollback`, then the M-11→M-20 batch
|
|
||||||
release, then the v5.13 plan.
|
|
||||||
|
|
||||||
## 8. Verifisering (testable criteria)
|
|
||||||
|
|
||||||
**Before building anything:**
|
|
||||||
|
|
||||||
- [ ] `grep -rniE 'blame|mtime|birthtime' scanners/` returns no instruction-age handling
|
|
||||||
→ confirms §3.1 still true at build time.
|
|
||||||
- [ ] `node scanners/drift-cli.mjs <path> --save --name pre-subtraction` succeeds and
|
|
||||||
`lib/baseline.mjs` round-trips → confirms §4 reuse is real, not assumed.
|
|
||||||
- [ ] If any claim from §1 is to be written into the register: a primary source exists
|
|
||||||
(URL to the talk or an Anthropic doc). Otherwise the entry is not written.
|
|
||||||
- [ ] **The age signal actually exists.** §5A's fallback claim — "project-level
|
|
||||||
CLAUDE.md/rules get real git ages" — is untested. Run
|
|
||||||
`git log --format=%ad --follow -- CLAUDE.md .claude/rules/` across two or three
|
|
||||||
real repos. If most instruction blocks trace back to a single bulk commit, per-block
|
|
||||||
age carries no information, shape A's primary signal collapses to the
|
|
||||||
compensatory-phrasing heuristic alone, and §7.2 (agent layer) is answered for us.
|
|
||||||
Check this *before* writing any scanner code.
|
|
||||||
|
|
||||||
**If shape A is built:**
|
|
||||||
|
|
||||||
- [ ] Red tests exist and fail before implementation.
|
|
||||||
- [ ] `node --test 'tests/**/*.test.mjs'` green, count strictly greater than 1365.
|
|
||||||
- [ ] `git diff --stat tests/snapshots/v5.0.0/` is empty (frozen baselines untouched).
|
|
||||||
- [ ] `node scanners/self-audit.mjs --check-readme` passes (badge counts updated:
|
|
||||||
scanners 16 → 17).
|
|
||||||
- [ ] Every new finding renders with a non-`Other` `userImpactCategory` and a
|
|
||||||
non-`_default` action language → humanizer wiring correct (M-16/M-17).
|
|
||||||
- [ ] **Floor test (§6.0), the blocking one.** Build a fasit by hand first — per
|
|
||||||
[[judge-the-judge-build-fasit-first]] — over the operator's own global CLAUDE.md,
|
|
||||||
labelling each block compensatory or load-bearing. Then run the tool. Required
|
|
||||||
result: **zero load-bearing blocks proposed for deletion.** Non-negotiable; the
|
|
||||||
known-correct floor items to check for explicitly are "never GitHub, only Forgejo",
|
|
||||||
"system bash is 3.2", the test command, and "`~/.claude` is not git-tracked". One
|
|
||||||
of those in the delete list fails the gate outright, whatever the recall.
|
|
||||||
- [ ] Recall, secondary: at least one candidate the operator agrees is genuinely dead
|
|
||||||
weight. Precision over recall throughout — a missed dead line costs a few tokens
|
|
||||||
per turn, a deleted load-bearing line costs a wrong remote or a broken script.
|
|
||||||
|
|
@ -254,39 +254,6 @@ model-switch is a *runtime* behaviour, not static config a scanner can reliably
|
||||||
automation. "No overstated behavioral finding ships" — so even the permitted opusplan *info*-advisory was
|
automation. "No overstated behavioral finding ships" — so even the permitted opusplan *info*-advisory was
|
||||||
left out; the verified @import extension is the whole of B6.
|
left out; the verified @import extension is the whole of B6.
|
||||||
|
|
||||||
### GAP scanner — authored-config scoping + direct cascade read (M-BUG-13)
|
|
||||||
|
|
||||||
The 25 presence checks ask "does the user's effective config have feature X?" and GAP **always**
|
|
||||||
runs `includeGlobal: true`. Two failure modes made the answer wrong on a real machine, both surfaced
|
|
||||||
by dogfooding `feature-gap`/`posture --global`:
|
|
||||||
|
|
||||||
1. **Demo/vendored config masks real gaps.** This plugin's own `examples/optimal-setup/` is a complete
|
|
||||||
config (sets `outputStyle`/`statusLine`/`worktree`/`model`/`keybindings.json`/`.lsp.json`), and its
|
|
||||||
copies vendored under `~/.claude/plugins/cache/.../config-audit/<ver>/examples/` are pulled into the
|
|
||||||
includeGlobal discovery. Because `anySettingsHas`/`files.some(...)` accept ANY discovered file, that
|
|
||||||
one demo file drove every tier-3 check to "present" → **GAP=0 on any target** (false negative).
|
|
||||||
Fix: `isAuthoredConfig` filters `ctx.files`/`parsedSettings` to the user's authored cascade —
|
|
||||||
excludes `~/.claude/plugins/` (absPath marker, mirrors CNF's M-BUG-2 exclusion) and any file whose
|
|
||||||
path **relative to the scan target** sits under `examples/` or `tests/fixtures/`. relPath (not
|
|
||||||
absPath) is deliberate: a fixture scanned AS the target keeps its own files, so the frozen v5.0.0
|
|
||||||
snapshots (scanned from `tests/fixtures/marketplace-medium`, which has no such nested trees) are
|
|
||||||
byte-stable.
|
|
||||||
|
|
||||||
2. **The real `~/.claude/settings.json` is invisible to the settings-key checks.** Discovery misses it
|
|
||||||
(its relPath carries no `.claude` segment when the walk root IS `~/.claude` — the gotcha) AND, when
|
|
||||||
vendored plugins flood the walk, the `maxFiles=2000` cap drops it. After (1) removed the demo
|
|
||||||
maskers, `statusLine`/`autoMode`/`permissions` (which the user HAS) would flip to false **positives**.
|
|
||||||
Fix: `readSettingsCascade` reads the four canonical cascade paths (user `settings.json`/`.local`,
|
|
||||||
project `settings.json`/`.local`) directly and merges them INTO `parsedSettings` — immune to the
|
|
||||||
cap and the gotcha. Merge (not replace) keeps non-canonical project settings and leaves the snapshot
|
|
||||||
(hermetic empty HOME → cascade adds nothing new) byte-stable.
|
|
||||||
|
|
||||||
Net: an empty target now surfaces ~18 humanized opportunities (was masked to ~0); config-audit's own
|
|
||||||
repo still shows 0 in output via its intentional `.config-audit-ignore` `CA-GAP-*` self-suppression
|
|
||||||
(a plugin repo legitimately lacks user-project features) — suppression is an envelope-layer concern,
|
|
||||||
orthogonal to this scanner fix. Scoped GAP-local; the includeGlobal discovery gotcha itself is left
|
|
||||||
to other consumers (see auto-memory `discovery-includeglobal-user-settings-gotcha`).
|
|
||||||
|
|
||||||
### CML scanner — context-window-scaled char budget
|
### CML scanner — context-window-scaled char budget
|
||||||
|
|
||||||
Beyond the line-count checks (200/500 lines, both MEDIUM), the CML scanner mirrors
|
Beyond the line-count checks (200/500 lines, both MEDIUM), the CML scanner mirrors
|
||||||
|
|
|
||||||
|
|
@ -1,157 +0,0 @@
|
||||||
# v5.13 Plan — Model Routing, Effort Awareness, Dead References
|
|
||||||
|
|
||||||
Derived from an external video analysis ("The Model Isn't the Moat", 2026-07) cross-checked
|
|
||||||
against primary sources and against what config-audit already encodes. Every claim acted on
|
|
||||||
here was verified against Anthropic's own docs; video-only claims are explicitly rejected below.
|
|
||||||
|
|
||||||
## Source verification (done 2026-07-14)
|
|
||||||
|
|
||||||
| Claim from video | Verdict | Source |
|
|
||||||
|---|---|---|
|
|
||||||
| Orchestrator + cheaper worker models is a supported, recommended pattern | VERIFIED | code.claude.com/docs/en/sub-agents ("Control costs by routing tasks to faster, cheaper models like Haiku"), code.claude.com/docs/en/workflows |
|
|
||||||
| Reasoning effort is tunable per settings / session / launch / **per-agent frontmatter** / SDK; levels `low, medium, high, xhigh, max` | VERIFIED | code.claude.com/docs/en/model-config#adjust-effort-level, sub-agents doc |
|
|
||||||
| Leaked Fable 5 system prompt principles ("partial recognition ≠ current knowledge"; "a prompt implying a file is present doesn't mean one is"; answer-first-then-one-question; tool-call scaling 1 / 3–5 / 5–10) | VERIFIED near-verbatim, **provenance unconfirmed** (third-party leak repo, not Anthropic-confirmed) | github.com/asgeirtj/system_prompts_leaks `Anthropic/claude-fable-5.md` |
|
|
||||||
| "Fable 5 on low ≈ Opus 4.8 on high, slightly higher cost/quality" score-vs-cost chart | **CONTRADICTED** — no such chart/statement on Anthropic's pages; GPT-5.5 appears only in a testimonial | anthropic.com/news/claude-fable-5-mythos-5 |
|
|
||||||
|
|
||||||
## Already covered — no action
|
|
||||||
|
|
||||||
| Video idea | Existing coverage |
|
|
||||||
|---|---|
|
|
||||||
| "Process is the moat" (config/harness > raw model) | The plugin's entire thesis |
|
|
||||||
| Extract repeated procedure into a skill | BP-MECH-003 + CA-OPT-001 (`optimization-lens-scanner.mjs:121`) |
|
|
||||||
| CLAUDE.md size/ownership discipline | BP-SIZE-001 + CA-CML line/size checks (`claude-md-linter.mjs:109/:120/:140`) |
|
|
||||||
| Check that referenced imports exist | CA-IMP broken `@import` (`lib/import-resolver.mjs:88`) — but **only** `@import`, see Chunk 3 |
|
|
||||||
| Plugin's own agents are model-routed | Agents table already pins sonnet for mechanical, opus for judgment |
|
|
||||||
|
|
||||||
## Gaps → chunks
|
|
||||||
|
|
||||||
Verified gap summary (register-mapper sweep, 2026-07-14): no scanner audits per-agent
|
|
||||||
`model:`/`effort:` frontmatter; effort has validity-check only (`settings-validator.mjs:195`),
|
|
||||||
no recommendation; no dead-reference check for prose file mentions in CLAUDE.md; no
|
|
||||||
adversarial/failure-mode requirement in planner-agent; `fix-engine.mjs:26` effort list
|
|
||||||
omits `xhigh` (settings-validator has all five).
|
|
||||||
|
|
||||||
### Chunk 1 — Register entries: model routing + effort (dogfoods `knowledge-refresh`)
|
|
||||||
|
|
||||||
Add to `knowledge/best-practices.json` via the knowledge-refresh flow (human-approved write):
|
|
||||||
|
|
||||||
- **BP-MODEL-001** (`category: model-fit`): subagents doing mechanical/read-only work can pin a
|
|
||||||
cheaper model via `model:` frontmatter; orchestrator keeps the strong model. Source:
|
|
||||||
code.claude.com/docs/en/sub-agents → `confidence: confirmed`.
|
|
||||||
- **BP-MODEL-002** (`category: model-fit`): reasoning effort is tunable at five levels in five
|
|
||||||
places (settings `effortLevel`, `/effort`, `--effort`, per-agent `effort` frontmatter, SDK);
|
|
||||||
default `high`; higher effort is not universally better for simple tasks. Source:
|
|
||||||
code.claude.com/docs/en/model-config → `confidence: confirmed`.
|
|
||||||
|
|
||||||
Schema per `scanners/lib/best-practices-register.mjs:42-102` (id/claim/confidence/source.url/
|
|
||||||
source.verified required). This chunk doubles as the DEL B dogfood of `/config-audit
|
|
||||||
knowledge-refresh` (each chunk is also a plugin test).
|
|
||||||
|
|
||||||
### Chunk 2 — fix-engine effort hygiene (tiny, TDD)
|
|
||||||
|
|
||||||
`fix-engine.mjs:26` `VALID_EFFORT_LEVELS = ['low','medium','high','max']` — missing `xhigh`.
|
|
||||||
Consequence: nearest-match "fix" for a typo like `xhig` corrects to `high`, not `xhigh`.
|
|
||||||
Red test first: `findNearestEffortLevel('xhig') === 'xhigh'`. Align list with
|
|
||||||
`settings-validator.mjs:75`.
|
|
||||||
|
|
||||||
### Chunk 3 — CA-CML dead prose references (new deterministic check)
|
|
||||||
|
|
||||||
The strongest video-derived principle ("a prompt implying a file is present doesn't mean one
|
|
||||||
is") applied to CLAUDE.md quality: flag file paths mentioned in CLAUDE.md **prose** that do not
|
|
||||||
exist on disk. Today only `@import` targets are existence-checked; stale pointers like
|
|
||||||
`docs/foo.md` or `scripts/bar.sh` rot silently and burn always-loaded tokens on misdirection.
|
|
||||||
|
|
||||||
Conservative v1 to control false positives:
|
|
||||||
- Only backtick-quoted tokens that look like relative file paths (contain `/` or a known
|
|
||||||
extension), resolved against the CLAUDE.md's own directory.
|
|
||||||
- Skip URLs, globs (`*`), placeholders (`{...}`, `<...>`, `$VAR`, `${...}`), absolute and
|
|
||||||
`~/` paths (machine-specific), and paths under `.gitignore`d dirs if cheap to determine.
|
|
||||||
- Severity: low. New CA-CML-NNN (verify next free NNN at implementation — IDs are dynamic).
|
|
||||||
|
|
||||||
Byte-stability: follow [[adding-scanner-byte-stability]] steps for a new finding type in an
|
|
||||||
EXISTING scanner — frozen `tests/snapshots/v5.0.0/` must stay untouched; default-output
|
|
||||||
snapshots regenerate (`UPDATE_SNAPSHOT=1`) only if a fixture actually carries the new type;
|
|
||||||
humanizer step 7 (M-16/M-17 lessons): `TRANSLATIONS`-static entry for the new RAW title
|
|
||||||
(CML category mapping already exists).
|
|
||||||
|
|
||||||
### Chunk 4 — feature-gap + inventory: model/effort awareness
|
|
||||||
|
|
||||||
- New T3 opportunity check in `feature-gap-scanner.mjs`: authored agents
|
|
||||||
(`isAuthoredConfig`, M-BUG-13 lesson) where **no** agent sets `model:` or `effort:` →
|
|
||||||
"all agents inherit the session model/effort — mechanical agents can be routed cheaper /
|
|
||||||
effort-calibrated" citing BP-MODEL-001/002. Fires only when authored agents exist
|
|
||||||
(M-BUG-15 lesson: no enhancement-check on empty collections). Opportunity framing, never
|
|
||||||
failure — deliberate max-model setups are a valid choice; finding is suppressable
|
|
||||||
(`.config-audit-ignore`).
|
|
||||||
- `whats-active` / `manifest`: surface `model`/`effort` per agent in the inventory tables.
|
|
||||||
- Humanizer wiring step 7 for the new GAP finding; verify via direct `scan()` output, not the
|
|
||||||
self-suppressed default output ([[agent-commands-need-scanner-scoping]]).
|
|
||||||
- feat commit → docs-gate: README + CLAUDE.md diffs required.
|
|
||||||
|
|
||||||
### Chunk 5 — planner-agent adversarial gate (do AFTER DEL B pipeline dogfood)
|
|
||||||
|
|
||||||
Add a required "Failure modes" section to `agents/planner-agent.md`'s action-plan contract:
|
|
||||||
before an action plan is emitted, list what could go wrong per change + rollback trigger.
|
|
||||||
Mirrors the video's scoping-vs-devil's-advocate distinction; currently absent (zero
|
|
||||||
adversarial requirements in agents/). **Sequencing constraint:** DEL B step 3.2 judges
|
|
||||||
planner-agent against a fasit — change the agent only after that dogfood pass, or the
|
|
||||||
fasit target moves mid-evaluation.
|
|
||||||
|
|
||||||
## Explicitly rejected (do not revisit without new evidence)
|
|
||||||
|
|
||||||
1. **"Fable low ≈ Opus high" cost/score framing** — contradicted by Anthropic's own pages.
|
|
||||||
Never encode in register, copy, or recommendations.
|
|
||||||
2. **Tool-call-count effort scaling (1 / 3–5 / 5–10) as a register entry** — source is an
|
|
||||||
unconfirmed third-party leak → would be `confidence: inferred`, never surfaced. Not worth
|
|
||||||
carrying.
|
|
||||||
3. **Cost/intelligence/"taste" model-routing table generator** — subjective scores don't fit
|
|
||||||
the deterministic, provenance-gated design. BP-MODEL-001 covers the actionable core.
|
|
||||||
4. **"Fable mode" skill** — a user-level skill, not configuration auditing. Out of plugin scope.
|
|
||||||
5. **CLAUDE.md prose contradiction detection** — real gap (CA-CNF only covers
|
|
||||||
settings/permissions/hooks) but not video-driven; keep this plan surgical.
|
|
||||||
|
|
||||||
## Known tension (named, not resolved here)
|
|
||||||
|
|
||||||
The operator's own global policy is Opus/max-effort for ALL subagents, never Haiku — the
|
|
||||||
opposite of Chunk 4's recommendation. Both are legitimate: the docs-backed routing advice
|
|
||||||
optimizes cost at equal quality for the general user; the operator deliberately buys maximum
|
|
||||||
quality. Chunk 4's copy must respect that (opportunity framing + suppressability), and on this
|
|
||||||
machine the finding will simply be suppressed or ignored. The plugin serves general users;
|
|
||||||
the operator's setup is not the target of the check.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
Global, after every chunk:
|
|
||||||
- `node --test 'tests/**/*.test.mjs'` → green (baseline 1359/0; count grows with new tests)
|
|
||||||
- `git status --porcelain tests/snapshots/v5.0.0/` → empty (frozen untouched)
|
|
||||||
- TDD: red test exists and fails BEFORE each production change
|
|
||||||
|
|
||||||
Per chunk:
|
|
||||||
- **C1:** `node --test tests/lib/best-practices-register.test.mjs` green;
|
|
||||||
`node scanners/knowledge-refresh-cli.mjs` classifies BP-MODEL-001/002 as fresh
|
|
||||||
- **C2:** `findNearestEffortLevel('xhig')` → `xhigh` (red first); `grep xhigh scanners/fix-engine.mjs` non-empty
|
|
||||||
- **C3:** fixture CLAUDE.md referencing `docs/missing.md` → finding fires; existing file /
|
|
||||||
URL / glob / placeholder / `~/` path → silent; humanized output has non-contradictory copy
|
|
||||||
- **C4:** authored-agent fixture without model/effort → opportunity fires; with either set →
|
|
||||||
silent; zero authored agents → silent; `userImpactCategory` ≠ `Other` end-to-end via direct scan()
|
|
||||||
- **C5:** dogfood plan run produces a Failure-modes section; DEL B 3.2 fasit judged BEFORE the change
|
|
||||||
|
|
||||||
## Key assumptions (test before/at implementation)
|
|
||||||
|
|
||||||
1. **Per-agent `effort` frontmatter is official** — verified 2026-07-14 against
|
|
||||||
code.claude.com/docs/en/sub-agents + /model-config; re-fetch both pages at implementation
|
|
||||||
(docs move).
|
|
||||||
2. **New finding type in existing scanner leaves frozen snapshots untouched** — M-17 precedent
|
|
||||||
says yes when no v5.0.0 fixture carries the type; verify by running the suite and inspecting
|
|
||||||
which snapshots differ before committing.
|
|
||||||
3. **Next free CA-CML/CA-GAP finding numbers** — IDs are built dynamically; grep tests +
|
|
||||||
snapshots for the highest used NNN before assigning.
|
|
||||||
|
|
||||||
## Sequencing vs DEL B (one plan, no relitigation)
|
|
||||||
|
|
||||||
This plan does NOT preempt the active DEL B sequence. Recommended order:
|
|
||||||
1. DEL B step 3 pipeline dogfood (`analyze → plan → implement → rollback`) — unchanged, next.
|
|
||||||
2. Batch patch release M-11→M-17 — unchanged.
|
|
||||||
3. v5.13 chunks 1→5 (chunk 1 doubles as the `knowledge-refresh` dogfood already queued in
|
|
||||||
DEL B "Resten"; chunk 5 explicitly waits for step 3.2). Release as minor v5.13.0 via
|
|
||||||
`release-plugin.mjs` when all chunks land.
|
|
||||||
|
|
@ -7,7 +7,7 @@
|
||||||
* Finding IDs: CA-GAP-NNN
|
* Finding IDs: CA-GAP-NNN
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { resolve, join, sep } from 'node:path';
|
import { resolve } from 'node:path';
|
||||||
import { readTextFile, discoverConfigFiles } from './lib/file-discovery.mjs';
|
import { readTextFile, discoverConfigFiles } from './lib/file-discovery.mjs';
|
||||||
import { finding, scannerResult } from './lib/output.mjs';
|
import { finding, scannerResult } from './lib/output.mjs';
|
||||||
import { SEVERITY } from './lib/severity.mjs';
|
import { SEVERITY } from './lib/severity.mjs';
|
||||||
|
|
@ -46,68 +46,6 @@ function isTargetLocal(ctx, f) {
|
||||||
return f.absPath.startsWith(ctx.targetPath);
|
return f.absPath.startsWith(ctx.targetPath);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Files that are test/demo/vendored config — NOT part of the user's authored
|
|
||||||
// cascade — must not satisfy "is feature X present?" checks, or they mask real
|
|
||||||
// gaps. The canonical case: this plugin's own examples/optimal-setup sets
|
|
||||||
// outputStyle/statusLine/worktree/model/keybindings/.lsp.json, and (because GAP
|
|
||||||
// always runs includeGlobal) its copies vendored under ~/.claude/plugins/cache
|
|
||||||
// drive every tier-3 presence check to "present" — hiding the user's real gaps
|
|
||||||
// on ANY target. Two classes to exclude:
|
|
||||||
// - plugin-bundled: anything under ~/.claude/plugins/ (absPath marker, mirrors
|
|
||||||
// the CNF conflict-detector exclusion from M-BUG-2).
|
|
||||||
// - nested demo/test data: a file whose path RELATIVE TO THE SCAN TARGET sits
|
|
||||||
// under an examples/ or tests/fixtures/ subtree. relPath (not absPath) is
|
|
||||||
// deliberate: a fixture scanned AS the target keeps its own files, so the
|
|
||||||
// frozen v5.0.0 byte-snapshots (scanned from tests/fixtures/marketplace-medium)
|
|
||||||
// are untouched. (M-BUG-13)
|
|
||||||
const PLUGIN_TREE_MARKER = `.claude${sep}plugins${sep}`;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @param {import('./lib/file-discovery.mjs').ConfigFile} file
|
|
||||||
* @returns {boolean} true if the file is part of the user's authored config
|
|
||||||
*/
|
|
||||||
function isAuthoredConfig(file) {
|
|
||||||
if (file.absPath.includes(PLUGIN_TREE_MARKER)) return false;
|
|
||||||
const segs = (file.relPath || '').split(sep);
|
|
||||||
if (segs.includes('examples')) return false;
|
|
||||||
const ti = segs.indexOf('tests');
|
|
||||||
if (ti !== -1 && segs[ti + 1] === 'fixtures') return false;
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Read the user→project→local settings cascade directly from the filesystem.
|
|
||||||
* The settings-key gap checks ask "does the USER's resolved config set X?" — a
|
|
||||||
* question the includeGlobal discovery answers unreliably on a real machine: the
|
|
||||||
* top-level ~/.claude/settings.json is missed (its relPath carries no `.claude`
|
|
||||||
* segment when the walk root IS ~/.claude) and, when many vendored plugins flood
|
|
||||||
* the walk, dropped by the discovery file cap. Reading the canonical cascade
|
|
||||||
* paths directly is immune to both. Merged INTO (not replacing) the discovery
|
|
||||||
* settings so any non-canonical project settings still count and the frozen
|
|
||||||
* snapshots stay byte-stable. (M-BUG-13)
|
|
||||||
* @param {string} targetPath
|
|
||||||
* @returns {Promise<Array<{ key: string, parsed: object }>>}
|
|
||||||
*/
|
|
||||||
async function readSettingsCascade(targetPath) {
|
|
||||||
const home = process.env.HOME || process.env.USERPROFILE || '';
|
|
||||||
const paths = [];
|
|
||||||
if (home) {
|
|
||||||
paths.push(['user', join(home, '.claude', 'settings.json')]);
|
|
||||||
paths.push(['user-local', join(home, '.claude', 'settings.local.json')]);
|
|
||||||
}
|
|
||||||
paths.push(['project', join(targetPath, '.claude', 'settings.json')]);
|
|
||||||
paths.push(['local', join(targetPath, '.claude', 'settings.local.json')]);
|
|
||||||
|
|
||||||
const out = [];
|
|
||||||
for (const [scope, p] of paths) {
|
|
||||||
const content = await readTextFile(p);
|
|
||||||
if (!content) continue;
|
|
||||||
const parsed = parseJson(content);
|
|
||||||
if (parsed && typeof parsed === 'object') out.push({ key: `cascade:${scope}:${p}`, parsed });
|
|
||||||
}
|
|
||||||
return out;
|
|
||||||
}
|
|
||||||
|
|
||||||
const TIER_SEVERITY = {
|
const TIER_SEVERITY = {
|
||||||
t1: SEVERITY.medium,
|
t1: SEVERITY.medium,
|
||||||
t2: SEVERITY.low,
|
t2: SEVERITY.low,
|
||||||
|
|
@ -541,30 +479,18 @@ export async function scan(targetPath, sharedDiscovery) {
|
||||||
? sharedDiscovery
|
? sharedDiscovery
|
||||||
: await discoverConfigFiles(resolve(targetPath), { includeGlobal: true });
|
: await discoverConfigFiles(resolve(targetPath), { includeGlobal: true });
|
||||||
|
|
||||||
// Presence checks ("does the user have feature X?") must see only the user's
|
// Parse all settings files upfront
|
||||||
// authored cascade — not bundled/vendored/demo config, which masks real gaps
|
|
||||||
// (M-BUG-13, see isAuthoredConfig).
|
|
||||||
const authoredFiles = discovery.files.filter(isAuthoredConfig);
|
|
||||||
|
|
||||||
// Parse all settings files upfront (authored discovery files) ...
|
|
||||||
const parsedSettings = new Map();
|
const parsedSettings = new Map();
|
||||||
for (const file of authoredFiles.filter(f => f.type === 'settings-json')) {
|
for (const file of discovery.files.filter(f => f.type === 'settings-json')) {
|
||||||
const content = await readTextFile(file.absPath);
|
const content = await readTextFile(file.absPath);
|
||||||
if (content) {
|
if (content) {
|
||||||
const parsed = parseJson(content);
|
const parsed = parseJson(content);
|
||||||
parsedSettings.set(`${file.scope}:${file.relPath}`, parsed);
|
parsedSettings.set(`${file.scope}:${file.relPath}`, parsed);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
// ... plus the real user→project→local cascade read directly, so settings-key
|
|
||||||
// checks see the true resolved config regardless of the discovery cap/gotcha
|
|
||||||
// (M-BUG-13). Merged, not replacing — keeps non-canonical project settings and
|
|
||||||
// the frozen byte-snapshots unchanged.
|
|
||||||
for (const { key, parsed } of await readSettingsCascade(resolve(targetPath))) {
|
|
||||||
parsedSettings.set(key, parsed);
|
|
||||||
}
|
|
||||||
|
|
||||||
const ctx = {
|
const ctx = {
|
||||||
files: authoredFiles,
|
files: discovery.files,
|
||||||
targetPath: resolve(targetPath),
|
targetPath: resolve(targetPath),
|
||||||
parsedSettings,
|
parsedSettings,
|
||||||
fileContents: new Map(),
|
fileContents: new Map(),
|
||||||
|
|
|
||||||
|
|
@ -435,12 +435,12 @@ export const TRANSLATIONS = {
|
||||||
recommendation: 'Consider moving team-wide settings to project scope and keeping personal ones at user or local scope.',
|
recommendation: 'Consider moving team-wide settings to project scope and keeping personal ones at user or local scope.',
|
||||||
},
|
},
|
||||||
'CLAUDE.md not modular': {
|
'CLAUDE.md not modular': {
|
||||||
title: 'Your instructions all live in one file',
|
title: 'Your instructions file is one big block',
|
||||||
description: 'Splitting your instructions into smaller linked files with `@import` or `.claude/rules/` keeps each part focused and easier to maintain.',
|
description: 'Splitting long instructions into smaller linked files makes them easier to maintain and easier on the loading time.',
|
||||||
recommendation: 'Break out long sections into separate files and link them with `@import`.',
|
recommendation: 'Break out long sections into separate files and link them with `@import`.',
|
||||||
},
|
},
|
||||||
'No path-scoped rules': {
|
'No path-scoped rules': {
|
||||||
title: 'You haven\'t set up path-scoped rules yet',
|
title: 'Your rules all load on every conversation',
|
||||||
description: 'Path-scoped rules only load when you\'re working with files that match — keeps each conversation focused.',
|
description: 'Path-scoped rules only load when you\'re working with files that match — keeps each conversation focused.',
|
||||||
recommendation: 'Add scoping to your rules so they only load for the files they apply to.',
|
recommendation: 'Add scoping to your rules so they only load for the files they apply to.',
|
||||||
},
|
},
|
||||||
|
|
@ -490,7 +490,7 @@ export const TRANSLATIONS = {
|
||||||
recommendation: 'Add fields like `model`, `tools`, or `description` to your skill files where useful.',
|
recommendation: 'Add fields like `model`, `tools`, or `description` to your skill files where useful.',
|
||||||
},
|
},
|
||||||
'No subagent isolation': {
|
'No subagent isolation': {
|
||||||
title: 'You haven\'t set up subagent isolation yet',
|
title: 'Your subagents share Claude\'s main work folder',
|
||||||
description: 'Isolated subagents run in their own copy of the repo so they can\'t accidentally disturb your main work.',
|
description: 'Isolated subagents run in their own copy of the repo so they can\'t accidentally disturb your main work.',
|
||||||
recommendation: 'Add `isolation: worktree` to subagents that do destructive or experimental work.',
|
recommendation: 'Add `isolation: worktree` to subagents that do destructive or experimental work.',
|
||||||
},
|
},
|
||||||
|
|
@ -817,11 +817,6 @@ export const TRANSLATIONS = {
|
||||||
description: 'Claude Code keeps every active skill\'s description in one shared listing it reads to choose which skill to use, and that listing has a limited size. Added up, your skills\' descriptions run past that size on a smaller setup, so Claude Code may drop some of them — and stop seeing those skills. This is an estimate; a larger setup has more room.',
|
description: 'Claude Code keeps every active skill\'s description in one shared listing it reads to choose which skill to use, and that listing has a limited size. Added up, your skills\' descriptions run past that size on a smaller setup, so Claude Code may drop some of them — and stop seeing those skills. This is an estimate; a larger setup has more room.',
|
||||||
recommendation: 'Free up room: turn off bundled skills you do not use, collapse the heaviest ones so only their names show, or shorten the longest descriptions. The details show the measured total and the room available.',
|
recommendation: 'Free up room: turn off bundled skills you do not use, collapse the heaviest ones so only their names show, or shorten the longest descriptions. The details show the measured total and the room available.',
|
||||||
},
|
},
|
||||||
'Skill body is large (loads on demand when the skill runs)': {
|
|
||||||
title: 'A skill\'s body is large (it loads only when that skill runs)',
|
|
||||||
description: 'This skill\'s instructions run longer than the rough guidance for a skill body. The body is not part of the always-loaded listing Claude reads every turn — it loads only when you invoke the skill, so it costs nothing until then. Once it loads, though, it stays in context for the rest of that session.',
|
|
||||||
recommendation: 'Move reference material into supporting files the skill opens only when needed, so the body stays lean. For a heavy skill you can also run its body in a separate context with `context: fork` in the skill\'s settings.',
|
|
||||||
},
|
|
||||||
},
|
},
|
||||||
patterns: [],
|
patterns: [],
|
||||||
_default: {
|
_default: {
|
||||||
|
|
|
||||||
|
|
@ -38,7 +38,6 @@ const SCANNER_TO_CATEGORY = {
|
||||||
TOK: 'Wasted tokens',
|
TOK: 'Wasted tokens',
|
||||||
CPS: 'Wasted tokens',
|
CPS: 'Wasted tokens',
|
||||||
SKL: 'Wasted tokens',
|
SKL: 'Wasted tokens',
|
||||||
AGT: 'Wasted tokens',
|
|
||||||
DIS: 'Dead config',
|
DIS: 'Dead config',
|
||||||
GAP: 'Missed opportunity',
|
GAP: 'Missed opportunity',
|
||||||
PLH: 'Configuration mistake',
|
PLH: 'Configuration mistake',
|
||||||
|
|
|
||||||
|
|
@ -24,7 +24,7 @@
|
||||||
* Exit codes: 0=ok, 3=unrecoverable error. Zero external dependencies.
|
* Exit codes: 0=ok, 3=unrecoverable error. Zero external dependencies.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { resolve, sep } from 'node:path';
|
import { resolve } from 'node:path';
|
||||||
import { writeFile, readFile, stat } from 'node:fs/promises';
|
import { writeFile, readFile, stat } from 'node:fs/promises';
|
||||||
import { discoverConfigFiles } from './lib/file-discovery.mjs';
|
import { discoverConfigFiles } from './lib/file-discovery.mjs';
|
||||||
import { resetCounter } from './lib/output.mjs';
|
import { resetCounter } from './lib/output.mjs';
|
||||||
|
|
@ -33,15 +33,6 @@ import { loadRegister, getEntry } from './lib/best-practices-register.mjs';
|
||||||
import { prefilterClaudeMd, LENS_DETECTORS } from './lib/lens-prefilter.mjs';
|
import { prefilterClaudeMd, LENS_DETECTORS } from './lib/lens-prefilter.mjs';
|
||||||
import { scan as optScan } from './optimization-lens-scanner.mjs';
|
import { scan as optScan } from './optimization-lens-scanner.mjs';
|
||||||
|
|
||||||
// Files under `.claude/plugins/` are shipped by an installed plugin — vendored
|
|
||||||
// CLAUDE.md plus its bundled tests/fixtures and examples. They are not the user's
|
|
||||||
// authored config, so a mechanism-fit suggestion against them is not actionable
|
|
||||||
// (the user can't edit a file the plugin overwrites on update). Excluded from the
|
|
||||||
// lens regardless of active/stale version. (M-BUG-11; mirrors the M-BUG-2 rule
|
|
||||||
// that keeps plugin-bundled config out of the conflict detector.)
|
|
||||||
const PLUGIN_TREE_MARKER = `.claude${sep}plugins${sep}`;
|
|
||||||
const isPluginBundled = (file) => (file.absPath || '').includes(PLUGIN_TREE_MARKER);
|
|
||||||
|
|
||||||
/** Confirmed register entry for `id`, or null. */
|
/** Confirmed register entry for `id`, or null. */
|
||||||
function confirmedEntry(register, id) {
|
function confirmedEntry(register, id) {
|
||||||
const e = getEntry(register, id);
|
const e = getEntry(register, id);
|
||||||
|
|
@ -81,13 +72,7 @@ async function main() {
|
||||||
}
|
}
|
||||||
|
|
||||||
resetCounter();
|
resetCounter();
|
||||||
const rawDiscovery = await discoverConfigFiles(absPath, { includeGlobal });
|
const discovery = await discoverConfigFiles(absPath, { includeGlobal });
|
||||||
// Scope the lens to the user's authored config: drop plugin-bundled files for
|
|
||||||
// BOTH halves of the motor (the OPT scanner reads discovery.files directly).
|
|
||||||
const discovery = {
|
|
||||||
...rawDiscovery,
|
|
||||||
files: (rawDiscovery.files || []).filter((f) => !isPluginBundled(f)),
|
|
||||||
};
|
|
||||||
|
|
||||||
// ── Deterministic half: the OPT scanner (CA-OPT-001) ──
|
// ── Deterministic half: the OPT scanner (CA-OPT-001) ──
|
||||||
const opt = await optScan(absPath, discovery);
|
const opt = await optScan(absPath, discovery);
|
||||||
|
|
@ -109,11 +94,7 @@ async function main() {
|
||||||
const entry = register ? confirmedEntry(register, cand.registerId) : null;
|
const entry = register ? confirmedEntry(register, cand.registerId) : null;
|
||||||
if (!entry) continue; // never surface an unverifiable recommendation
|
if (!entry) continue; // never surface an unverifiable recommendation
|
||||||
candidates.push({
|
candidates.push({
|
||||||
// Absolute path: unique + readable. relPath collides across scopes
|
file: file.relPath || file.absPath,
|
||||||
// (a repo-root `CLAUDE.md` and the user-global `~/.claude/CLAUDE.md`
|
|
||||||
// both relPath to `CLAUDE.md`), which would send the agent's Read() to
|
|
||||||
// the wrong file. (M-BUG-11)
|
|
||||||
file: file.absPath,
|
|
||||||
line: bodyStartLine - 1 + cand.line,
|
line: bodyStartLine - 1 + cand.line,
|
||||||
lensCheck: cand.lensCheck,
|
lensCheck: cand.lensCheck,
|
||||||
mechanism: cand.mechanism,
|
mechanism: cand.mechanism,
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,6 @@
|
||||||
import { resolve } from 'node:path';
|
import { resolve } from 'node:path';
|
||||||
import { writeFile } from 'node:fs/promises';
|
import { writeFile } from 'node:fs/promises';
|
||||||
import { runAllScanners } from './scan-orchestrator.mjs';
|
import { runAllScanners } from './scan-orchestrator.mjs';
|
||||||
import { humanizeEnvelope } from './lib/humanizer.mjs';
|
|
||||||
import {
|
import {
|
||||||
calculateUtilization,
|
calculateUtilization,
|
||||||
determineMaturityLevel,
|
determineMaturityLevel,
|
||||||
|
|
@ -115,14 +114,7 @@ async function main() {
|
||||||
}
|
}
|
||||||
|
|
||||||
if (outputFile) {
|
if (outputFile) {
|
||||||
// Consumers (feature-gap.md, posture.md) read scannerEnvelope.scanners[].findings
|
const json = JSON.stringify(result, null, 2);
|
||||||
// and group on humanizer fields. posture's result nests the envelope under
|
|
||||||
// `scannerEnvelope`, so humanize THAT (not `result`, which has no top-level
|
|
||||||
// `scanners` array — humanizeEnvelope would no-op). --json/--raw stay raw.
|
|
||||||
const fileEnv = (jsonMode || rawMode)
|
|
||||||
? result
|
|
||||||
: { ...result, scannerEnvelope: humanizeEnvelope(result.scannerEnvelope) };
|
|
||||||
const json = JSON.stringify(fileEnv, null, 2);
|
|
||||||
await writeFile(outputFile, json, 'utf-8');
|
await writeFile(outputFile, json, 'utf-8');
|
||||||
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
process.stderr.write(`\nResults written to ${outputFile}\n`);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -251,9 +251,9 @@ function globToRegex(pattern) {
|
||||||
.replace(/\/\*\*\//g, '{{GLOBSTAR_SLASH}}')
|
.replace(/\/\*\*\//g, '{{GLOBSTAR_SLASH}}')
|
||||||
.replace(/\*\*/g, '{{GLOBSTAR}}')
|
.replace(/\*\*/g, '{{GLOBSTAR}}')
|
||||||
.replace(/\*/g, '[^/]*')
|
.replace(/\*/g, '[^/]*')
|
||||||
.replace(/\?/g, '[^/]') // must run BEFORE placeholder restore — '(?:' would corrupt
|
|
||||||
.replace(/\{\{GLOBSTAR_SLASH\}\}/g, '(?:/.+/|/)') // **/ matches 0+ intermediate dirs
|
.replace(/\{\{GLOBSTAR_SLASH\}\}/g, '(?:/.+/|/)') // **/ matches 0+ intermediate dirs
|
||||||
.replace(/\{\{GLOBSTAR\}\}/g, '.*');
|
.replace(/\{\{GLOBSTAR\}\}/g, '.*')
|
||||||
|
.replace(/\?/g, '[^/]');
|
||||||
|
|
||||||
// Handle leading patterns
|
// Handle leading patterns
|
||||||
if (!regex.startsWith('.*') && !regex.startsWith('/')) {
|
if (!regex.startsWith('.*') && !regex.startsWith('/')) {
|
||||||
|
|
|
||||||
|
|
@ -1,62 +0,0 @@
|
||||||
/**
|
|
||||||
* M-BUG-18 — analysis-report.md persistence contract.
|
|
||||||
*
|
|
||||||
* The Claude Code subagent harness instructs spawned agents NOT to write
|
|
||||||
* report/summary/findings/analysis .md files — the parent reads the agent's
|
|
||||||
* final text message, not files it creates. The analyzer-agent therefore
|
|
||||||
* cannot be the one that persists analysis-report.md (verified live: the
|
|
||||||
* agent skipped Write and returned the report inline).
|
|
||||||
*
|
|
||||||
* New contract (orchestrator-writes pattern):
|
|
||||||
* - analyzer-agent returns the complete report as its final message
|
|
||||||
* - the analyze command saves that returned report verbatim to
|
|
||||||
* ~/.claude/config-audit/sessions/{session-id}/analysis-report.md,
|
|
||||||
* which downstream phases (plan, interview, status) read.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { test } from 'node:test';
|
|
||||||
import { strict as assert } from 'node:assert';
|
|
||||||
import { readFile } from 'node:fs/promises';
|
|
||||||
import { resolve, dirname } from 'node:path';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
|
||||||
|
|
||||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
||||||
const COMMANDS_DIR = resolve(__dirname, '..', '..', 'commands');
|
|
||||||
const AGENTS_DIR = resolve(__dirname, '..', '..', 'agents');
|
|
||||||
|
|
||||||
test('analyze.md: agent prompt does not tell the agent to write the report file', async () => {
|
|
||||||
const content = await readFile(resolve(COMMANDS_DIR, 'analyze.md'), 'utf-8');
|
|
||||||
assert.doesNotMatch(
|
|
||||||
content,
|
|
||||||
/Output to:.*analysis-report\.md/,
|
|
||||||
'the spawn prompt must not instruct the subagent to write analysis-report.md — the harness blocks agent-written report files'
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('analyze.md: command saves the returned report to analysis-report.md', async () => {
|
|
||||||
const content = await readFile(resolve(COMMANDS_DIR, 'analyze.md'), 'utf-8');
|
|
||||||
assert.match(
|
|
||||||
content,
|
|
||||||
/return[s]? the complete report as (its|your) final message/i,
|
|
||||||
'analyze.md must state that the agent returns the report inline'
|
|
||||||
);
|
|
||||||
assert.match(
|
|
||||||
content,
|
|
||||||
/Write tool[\s\S]{0,200}analysis-report\.md|analysis-report\.md[\s\S]{0,200}Write tool/,
|
|
||||||
'analyze.md must instruct the command to persist the returned report to analysis-report.md with the Write tool'
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('analyzer-agent.md: output contract is return-inline, not self-write', async () => {
|
|
||||||
const content = await readFile(resolve(AGENTS_DIR, 'analyzer-agent.md'), 'utf-8');
|
|
||||||
assert.match(
|
|
||||||
content,
|
|
||||||
/return the complete report as your final message/i,
|
|
||||||
'analyzer-agent must be told its final message IS the report'
|
|
||||||
);
|
|
||||||
assert.doesNotMatch(
|
|
||||||
content,
|
|
||||||
/^Write to: .*analysis-report\.md/m,
|
|
||||||
'analyzer-agent must not carry the old self-write output contract'
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
/**
|
|
||||||
* M-BUG-20 — shared implementation-log clobbering under parallel agents.
|
|
||||||
*
|
|
||||||
* implement.md step 4 spawns implementer agents in parallel batches, and every
|
|
||||||
* agent appends its result to the SAME implementation-log.md. Dogfooding
|
|
||||||
* (2026-07-17, throwaway linkedin-posts copy) showed agents satisfying
|
|
||||||
* "Append result to:" with a full-file Write: each agent read the log, added
|
|
||||||
* its entry, and wrote the whole file back — the last writer silently
|
|
||||||
* clobbered 4 of 6 entries.
|
|
||||||
*
|
|
||||||
* Contract: both the command template and the agent prompt must pin the append
|
|
||||||
* mechanism — Bash `>>`, never the Write/Edit tool — on the shared log.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import { test } from 'node:test';
|
|
||||||
import { strict as assert } from 'node:assert';
|
|
||||||
import { readFile } from 'node:fs/promises';
|
|
||||||
import { resolve, dirname } from 'node:path';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
|
||||||
|
|
||||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
||||||
const ROOT = resolve(__dirname, '..', '..');
|
|
||||||
|
|
||||||
const APPEND_MECHANISM_REGEX = />>/;
|
|
||||||
const FORBID_WRITE_TOOL_REGEX = /never[^.\n]*\bwrite\b[^.\n]*tool|\bwrite\b[^.\n]*tool[^.\n]*never/i;
|
|
||||||
|
|
||||||
test('implement.md: agent-spawn template pins Bash >> append on the shared log', async () => {
|
|
||||||
const content = await readFile(resolve(ROOT, 'commands', 'implement.md'), 'utf-8');
|
|
||||||
assert.match(content, APPEND_MECHANISM_REGEX,
|
|
||||||
'implement.md must instruct appending to implementation-log.md with Bash >>');
|
|
||||||
assert.match(content, FORBID_WRITE_TOOL_REGEX,
|
|
||||||
'implement.md must forbid the Write tool on the shared implementation log');
|
|
||||||
});
|
|
||||||
|
|
||||||
test('implementer-agent.md: output section pins Bash >> append and forbids Write tool on the log', async () => {
|
|
||||||
const content = await readFile(resolve(ROOT, 'agents', 'implementer-agent.md'), 'utf-8');
|
|
||||||
assert.match(content, APPEND_MECHANISM_REGEX,
|
|
||||||
'implementer-agent.md must instruct appending to the log with Bash >>');
|
|
||||||
assert.match(content, FORBID_WRITE_TOOL_REGEX,
|
|
||||||
'implementer-agent.md must forbid the Write tool on the shared implementation log');
|
|
||||||
});
|
|
||||||
|
|
@ -162,63 +162,6 @@ test('CML, SET, HKV, RUL, MCP, IMP, GAP, TOK, PLH have non-empty static maps', (
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
test('GAP "CLAUDE.md not modular" copy is size-neutral (M-BUG-14)', () => {
|
|
||||||
// The t2_2 check is a pure presence check (no length gate), so a small
|
|
||||||
// non-modular CLAUDE.md must not be told it is "big"/"long" or that it
|
|
||||||
// costs "loading time". Title + description must stay size/cost-neutral
|
|
||||||
// and keep only the honest structural "split into linked files" framing.
|
|
||||||
const t = TRANSLATIONS.GAP.static['CLAUDE.md not modular'];
|
|
||||||
assert.ok(t, 'GAP static map missing "CLAUDE.md not modular"');
|
|
||||||
const title = t.title.toLowerCase();
|
|
||||||
const desc = t.description.toLowerCase();
|
|
||||||
assert.ok(!/\bbig\b/.test(title), `title overclaims size: "${t.title}"`);
|
|
||||||
assert.ok(!/\blong\b/.test(title), `title overclaims size: "${t.title}"`);
|
|
||||||
assert.ok(!/\bbig\b/.test(desc), `description overclaims size: "${t.description}"`);
|
|
||||||
assert.ok(!/\blong\b/.test(desc), `description overclaims size: "${t.description}"`);
|
|
||||||
assert.ok(!/loading time/.test(desc), `description overclaims load cost: "${t.description}"`);
|
|
||||||
assert.ok(/split/.test(desc), `description should keep structural split framing: "${t.description}"`);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('GAP enhancement-gap titles do not presuppose the feature exists (M-BUG-15)', () => {
|
|
||||||
// t2_3 ("No path-scoped rules") and t3_6 ("No subagent isolation") iterate a
|
|
||||||
// collection and return false for an EMPTY one — so they fire even when the
|
|
||||||
// user has zero rules / zero subagents. The humanized title must therefore
|
|
||||||
// not assert the feature exists, or it contradicts the sibling presence gap
|
|
||||||
// ("No custom subagents" → "You haven't set up any specialized helper agents
|
|
||||||
// yet"). Mirror the house "You haven't set up X yet" framing — honest for
|
|
||||||
// both the zero-state and the has-but-unconfigured state.
|
|
||||||
const rules = TRANSLATIONS.GAP.static['No path-scoped rules'];
|
|
||||||
const iso = TRANSLATIONS.GAP.static['No subagent isolation'];
|
|
||||||
assert.ok(rules, 'GAP static map missing "No path-scoped rules"');
|
|
||||||
assert.ok(iso, 'GAP static map missing "No subagent isolation"');
|
|
||||||
// Title must not presuppose existing rules / subagents.
|
|
||||||
assert.ok(!/your rules all load/i.test(rules.title),
|
|
||||||
`t2_3 title presupposes existing rules: "${rules.title}"`);
|
|
||||||
assert.ok(!/your subagents/i.test(iso.title),
|
|
||||||
`t3_6 title presupposes existing subagents: "${iso.title}"`);
|
|
||||||
// Title must still name the feature it points at.
|
|
||||||
assert.ok(/path-scoped rules/i.test(rules.title),
|
|
||||||
`t2_3 title should still name path-scoped rules: "${rules.title}"`);
|
|
||||||
assert.ok(/isolation/i.test(iso.title),
|
|
||||||
`t3_6 title should still name subagent isolation: "${iso.title}"`);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('SKL oversized-body copy is on-demand, not listing-budget (M-BUG-16)', () => {
|
|
||||||
// The skill-listing check emits a third finding for an oversized skill BODY,
|
|
||||||
// which loads ON DEMAND only when the skill runs — not the always-loaded
|
|
||||||
// listing. Before the fix this title had no static entry and fell through to
|
|
||||||
// the SKL _default ("using more of the listing budget"), contradicting the
|
|
||||||
// finding's own evidence ("NOT every turn like the always-loaded listing").
|
|
||||||
const t = TRANSLATIONS.SKL.static['Skill body is large (loads on demand when the skill runs)'];
|
|
||||||
assert.ok(t, 'SKL static map missing "Skill body is large (loads on demand when the skill runs)"');
|
|
||||||
assert.ok(!/listing budget/i.test(t.title), `body title must not claim listing budget: "${t.title}"`);
|
|
||||||
assert.ok(!/listing budget/i.test(t.description), `body description must not claim listing budget: "${t.description}"`);
|
|
||||||
// Must convey the on-demand body cost.
|
|
||||||
assert.ok(/body/i.test(t.title), `body title should name the body: "${t.title}"`);
|
|
||||||
assert.ok(/loads only|on demand|when (it|that|the) skill runs|when you invoke/i.test(t.title + ' ' + t.description),
|
|
||||||
`copy should convey the on-demand load: "${t.title}" / "${t.description}"`);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('CNF, COL, PLH have at least one pattern entry (template-literal titles)', () => {
|
test('CNF, COL, PLH have at least one pattern entry (template-literal titles)', () => {
|
||||||
// These scanners use template-literal titles for some findings.
|
// These scanners use template-literal titles for some findings.
|
||||||
for (const prefix of ['CNF', 'COL', 'PLH']) {
|
for (const prefix of ['CNF', 'COL', 'PLH']) {
|
||||||
|
|
|
||||||
|
|
@ -116,27 +116,6 @@ test('humanizeFinding falls back to _default when title unknown', () => {
|
||||||
assert.ok(/instructions file/i.test(out.title), `expected CML _default title, got: ${out.title}`);
|
assert.ok(/instructions file/i.test(out.title), `expected CML _default title, got: ${out.title}`);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('humanizeFinding maps SKL oversized-body finding to an on-demand title, not the listing-budget _default', () => {
|
|
||||||
// RAW title emitted by skill-listing-scanner for body > threshold (v5.11 B7).
|
|
||||||
// It is an ON-DEMAND body cost, NOT the always-loaded listing budget — the
|
|
||||||
// scanner is careful to distinguish them, so the humanized title must not
|
|
||||||
// regress into "listing budget" language via the SKL _default (M-BUG-16).
|
|
||||||
const input = makeFinding({
|
|
||||||
scanner: 'SKL',
|
|
||||||
severity: 'low',
|
|
||||||
title: 'Skill body is large (loads on demand when the skill runs)',
|
|
||||||
description: 'Skill "repo-init" (user) has a body of about 6223 tokens (712 lines), over the ~5000-token guidance for a skill body.',
|
|
||||||
evidence: 'body_tokens~6223; threshold=5000 tok; skill="repo-init"; source=user - this is the skill BODY which loads ON DEMAND only when the skill is invoked - NOT every turn like the always-loaded listing.',
|
|
||||||
});
|
|
||||||
const out = humanizeFinding(input);
|
|
||||||
assert.doesNotMatch(out.title, /listing budget/i,
|
|
||||||
`body finding must not be framed as a listing-budget cost, got: ${out.title}`);
|
|
||||||
assert.notEqual(out.title, 'A skill is using more of the listing budget than it should',
|
|
||||||
'body finding must have its own mapping, not the SKL _default');
|
|
||||||
assert.match(out.title, /body|on demand|when (it|the skill) runs/i,
|
|
||||||
`humanized title must convey the on-demand body cost, got: ${out.title}`);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('humanizeFinding passes through original strings when scanner prefix unknown', () => {
|
test('humanizeFinding passes through original strings when scanner prefix unknown', () => {
|
||||||
const input = makeFinding({ scanner: 'XXX', title: 'whatever' });
|
const input = makeFinding({ scanner: 'XXX', title: 'whatever' });
|
||||||
const out = humanizeFinding(input);
|
const out = humanizeFinding(input);
|
||||||
|
|
@ -205,13 +184,10 @@ test('humanizeFinding sets category Conflict for CNF/COL', () => {
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
test('humanizeFinding sets category Wasted tokens for TOK/CPS/SKL/AGT', () => {
|
test('humanizeFinding sets category Wasted tokens for TOK/CPS/SKL', () => {
|
||||||
// AGT (agent-listing budget) is an always-loaded per-turn token cost — the
|
for (const s of ['TOK', 'CPS', 'SKL']) {
|
||||||
// agent name+description is re-sent every turn in the listing — so it belongs
|
|
||||||
// in "Wasted tokens" alongside TOK/CPS/SKL, not the 'Other' fallback (M-BUG-17).
|
|
||||||
for (const s of ['TOK', 'CPS', 'SKL', 'AGT']) {
|
|
||||||
const out = humanizeFinding(makeFinding({ scanner: s }));
|
const out = humanizeFinding(makeFinding({ scanner: s }));
|
||||||
assert.equal(out.userImpactCategory, 'Wasted tokens', `${s} should map to Wasted tokens`);
|
assert.equal(out.userImpactCategory, 'Wasted tokens');
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
import { describe, it, beforeEach } from 'node:test';
|
import { describe, it, beforeEach } from 'node:test';
|
||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import { resolve, join, dirname } from 'node:path';
|
import { resolve, join } from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||||
import { tmpdir } from 'node:os';
|
import { tmpdir } from 'node:os';
|
||||||
|
|
@ -411,98 +411,3 @@ describe('GAP scanner — filter-before lever wiring (chatty hook fixture)', ()
|
||||||
assert.equal(lever, undefined, `expected no filter-before lever; got id=${lever?.id}`);
|
assert.equal(lever, undefined, `expected no filter-before lever; got id=${lever?.id}`);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('GAP scanner — test/demo data must not mask real gaps (M-BUG-13)', () => {
|
|
||||||
// Build a throwaway project (+ optional hermetic HOME settings) and run GAP
|
|
||||||
// exactly as posture --global does: includeGlobal discovery + scan().
|
|
||||||
async function runGap({ projectFiles = {}, homeSettings = null } = {}) {
|
|
||||||
const project = await mkdtemp(join(tmpdir(), 'config-audit-gap-mask-proj-'));
|
|
||||||
const home = await mkdtemp(join(tmpdir(), 'config-audit-gap-mask-home-'));
|
|
||||||
for (const [rel, content] of Object.entries(projectFiles)) {
|
|
||||||
const abs = join(project, rel);
|
|
||||||
await mkdir(dirname(abs), { recursive: true });
|
|
||||||
await writeFile(abs, content);
|
|
||||||
}
|
|
||||||
if (homeSettings) {
|
|
||||||
await mkdir(join(home, '.claude'), { recursive: true });
|
|
||||||
await writeFile(join(home, '.claude', 'settings.json'), JSON.stringify(homeSettings));
|
|
||||||
}
|
|
||||||
const original = process.env.HOME;
|
|
||||||
process.env.HOME = home;
|
|
||||||
try {
|
|
||||||
resetCounter();
|
|
||||||
const discovery = await discoverConfigFiles(project, { includeGlobal: true });
|
|
||||||
const result = await scan(project, discovery);
|
|
||||||
return result;
|
|
||||||
} finally {
|
|
||||||
process.env.HOME = original;
|
|
||||||
await rm(project, { recursive: true, force: true });
|
|
||||||
await rm(home, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const hasGap = (result, titleRe) =>
|
|
||||||
result.findings.some(f => f.scanner === 'GAP' && titleRe.test(f.title || ''));
|
|
||||||
|
|
||||||
it('a nested examples/ settings.json does NOT satisfy the outputStyle check (settings-key)', async () => {
|
|
||||||
const result = await runGap({
|
|
||||||
projectFiles: {
|
|
||||||
'.claude/CLAUDE.md': '# proj',
|
|
||||||
'examples/optimal-setup/.claude/settings.json': JSON.stringify({ outputStyle: 'Explanatory' }),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.ok(
|
|
||||||
hasGap(result, /output style/i),
|
|
||||||
`example settings.json must not mask the outputStyle gap; got: ${result.findings.map(f => f.title).join(' | ')}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('a nested examples/ keybindings.json does NOT satisfy the keybindings check (file-type)', async () => {
|
|
||||||
const result = await runGap({
|
|
||||||
projectFiles: {
|
|
||||||
'.claude/CLAUDE.md': '# proj',
|
|
||||||
'examples/optimal-setup/.claude/keybindings.json': JSON.stringify({}),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.ok(
|
|
||||||
hasGap(result, /keybinding/i),
|
|
||||||
`example keybindings.json must not mask the keybindings gap; got: ${result.findings.map(f => f.title).join(' | ')}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('a nested tests/fixtures/ settings.json does NOT satisfy the model check', async () => {
|
|
||||||
const result = await runGap({
|
|
||||||
projectFiles: {
|
|
||||||
'.claude/CLAUDE.md': '# proj',
|
|
||||||
'tests/fixtures/demo/.claude/settings.json': JSON.stringify({ model: 'opus' }),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.ok(
|
|
||||||
hasGap(result, /model config/i),
|
|
||||||
`fixture settings.json must not mask the model gap; got: ${result.findings.map(f => f.title).join(' | ')}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("the project's OWN real .claude/settings.json IS counted (not over-excluded)", async () => {
|
|
||||||
const result = await runGap({
|
|
||||||
projectFiles: {
|
|
||||||
'.claude/CLAUDE.md': '# proj',
|
|
||||||
'.claude/settings.json': JSON.stringify({ outputStyle: 'Explanatory' }),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.ok(
|
|
||||||
!hasGap(result, /output style/i),
|
|
||||||
`real project settings.json must satisfy outputStyle; got: ${result.findings.map(f => f.title).join(' | ')}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('the real ~/.claude/settings.json IS seen for settings-key checks (gotcha fix end-to-end)', async () => {
|
|
||||||
const result = await runGap({
|
|
||||||
homeSettings: { statusLine: { type: 'command', command: 'x' } },
|
|
||||||
});
|
|
||||||
assert.ok(
|
|
||||||
!hasGap(result, /status line/i),
|
|
||||||
`~/.claude/settings.json statusLine must be discovered; got: ${result.findings.map(f => f.title).join(' | ')}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
|
||||||
|
|
@ -1,102 +0,0 @@
|
||||||
/**
|
|
||||||
* optimize-lens-cli tests — payload scoping + candidate identity (M-BUG-11).
|
|
||||||
*
|
|
||||||
* The lens CLI feeds the precision-gate agent. Two correctness invariants it must
|
|
||||||
* hold, both established elsewhere in the codebase but originally missing here:
|
|
||||||
*
|
|
||||||
* 1. SCOPING — plugin-bundled CLAUDE.md (anything under `.claude/plugins/`:
|
|
||||||
* vendored plugin config + its bundled tests/fixtures + examples, active or
|
|
||||||
* stale) is NOT the user's authored config. The user can't act on a
|
|
||||||
* mechanism-fit suggestion against a file a plugin ships. So the lens must
|
|
||||||
* drop them — the M-BUG-2 `isPluginBundled` rule, applied to the lens.
|
|
||||||
*
|
|
||||||
* 2. IDENTITY — a candidate's `file` must uniquely name a readable file. The
|
|
||||||
* user-global `~/.claude/CLAUDE.md` and a repo-root `CLAUDE.md` both have
|
|
||||||
* relPath `CLAUDE.md`; labelling candidates by relPath collides them, and
|
|
||||||
* the agent's `Read(file)` then resolves the wrong one. Candidates carry an
|
|
||||||
* absolute path.
|
|
||||||
*
|
|
||||||
* Hermetic: a temp target with a real CLAUDE.md and a nested plugin-bundled one.
|
|
||||||
* `.claude/plugins/` is not in SKIP_DIRS, so the normal walk discovers it —
|
|
||||||
* no ~/.claude / --global needed.
|
|
||||||
*/
|
|
||||||
import { describe, it } from 'node:test';
|
|
||||||
import assert from 'node:assert/strict';
|
|
||||||
import { join, isAbsolute } from 'node:path';
|
|
||||||
import { mkdtemp, mkdir, writeFile, rm, readFile } from 'node:fs/promises';
|
|
||||||
import { tmpdir } from 'node:os';
|
|
||||||
import { execFile } from 'node:child_process';
|
|
||||||
import { promisify } from 'node:util';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
|
||||||
|
|
||||||
const execFileP = promisify(execFile);
|
|
||||||
const CLI = fileURLToPath(new URL('../../scanners/optimize-lens-cli.mjs', import.meta.url));
|
|
||||||
|
|
||||||
/** Build a temp target, run the lens CLI on it, return the parsed payload. */
|
|
||||||
async function runLens(files) {
|
|
||||||
const root = await mkdtemp(join(tmpdir(), 'ca-lens-cli-'));
|
|
||||||
try {
|
|
||||||
for (const [rel, content] of Object.entries(files)) {
|
|
||||||
const abs = join(root, rel);
|
|
||||||
await mkdir(join(abs, '..'), { recursive: true });
|
|
||||||
await writeFile(abs, content, 'utf-8');
|
|
||||||
}
|
|
||||||
const out = join(root, 'payload.json');
|
|
||||||
await execFileP('node', [CLI, root, '--output-file', out]);
|
|
||||||
return JSON.parse(await readFile(out, 'utf-8'));
|
|
||||||
} finally {
|
|
||||||
await rm(root, { recursive: true, force: true });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const REAL = '# Project\n\nNever commit secrets to the repo.\n';
|
|
||||||
const PLUGIN = '# Vendored plugin\n\nNever delete the plugin cache directory.\n';
|
|
||||||
|
|
||||||
describe('optimize-lens-cli — scoping (M-BUG-11)', () => {
|
|
||||||
it('excludes plugin-bundled CLAUDE.md from candidates', async () => {
|
|
||||||
const payload = await runLens({
|
|
||||||
'CLAUDE.md': REAL,
|
|
||||||
'.claude/plugins/cache/mkt/plug/1.0.0/CLAUDE.md': PLUGIN,
|
|
||||||
});
|
|
||||||
const bundled = payload.candidates.filter(
|
|
||||||
(c) => c.file.includes(`.claude/plugins/`) || c.file.includes(`/plugins/`),
|
|
||||||
);
|
|
||||||
assert.deepEqual(
|
|
||||||
bundled.map((c) => c.file),
|
|
||||||
[],
|
|
||||||
'no candidate should come from a file under .claude/plugins/',
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('still surfaces the user’s real CLAUDE.md', async () => {
|
|
||||||
const payload = await runLens({
|
|
||||||
'CLAUDE.md': REAL,
|
|
||||||
'.claude/plugins/cache/mkt/plug/1.0.0/CLAUDE.md': PLUGIN,
|
|
||||||
});
|
|
||||||
const real = payload.candidates.filter((c) => c.file.endsWith(`${join('', 'CLAUDE.md')}`));
|
|
||||||
assert.ok(real.length >= 1, 'the real CLAUDE.md never-instruction should survive scoping');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('excludes plugin-bundled CLAUDE.md from deterministic findings too', async () => {
|
|
||||||
const procedure =
|
|
||||||
'# Plugin release\n\n' +
|
|
||||||
Array.from({ length: 7 }, (_, i) => `${i + 1}. Do release step ${i + 1} and verify.`).join('\n') +
|
|
||||||
'\n';
|
|
||||||
const payload = await runLens({
|
|
||||||
'CLAUDE.md': REAL,
|
|
||||||
'.claude/plugins/cache/mkt/plug/1.0.0/CLAUDE.md': procedure,
|
|
||||||
});
|
|
||||||
const bundled = (payload.deterministic || []).filter((f) => String(f.file).includes('plugins/'));
|
|
||||||
assert.deepEqual(bundled, [], 'deterministic CA-OPT-001 must not fire on vendored plugin CLAUDE.md');
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe('optimize-lens-cli — candidate identity (M-BUG-11)', () => {
|
|
||||||
it('labels every candidate with an absolute, unique path', async () => {
|
|
||||||
const payload = await runLens({ 'CLAUDE.md': REAL });
|
|
||||||
assert.ok(payload.candidates.length >= 1, 'expected at least one candidate');
|
|
||||||
for (const c of payload.candidates) {
|
|
||||||
assert.ok(isAbsolute(c.file), `candidate file must be absolute, got: ${c.file}`);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
@ -1,7 +1,6 @@
|
||||||
import { describe, it } from 'node:test';
|
import { describe, it } from 'node:test';
|
||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import { resolve, dirname, join } from 'node:path';
|
import { resolve, dirname } from 'node:path';
|
||||||
import { tmpdir } from 'node:os';
|
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
import { execFile } from 'node:child_process';
|
import { execFile } from 'node:child_process';
|
||||||
import { promisify } from 'node:util';
|
import { promisify } from 'node:util';
|
||||||
|
|
@ -142,44 +141,4 @@ describe('posture humanizer wiring (Step 6)', () => {
|
||||||
'humanized stderr must differ from v5.0.0 verbatim stderr');
|
'humanized stderr must differ from v5.0.0 verbatim stderr');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// M-BUG-12: feature-gap.md/posture.md read findings from posture.mjs --output-file
|
|
||||||
// and group on humanizer fields (userActionLanguage etc.). Default-mode output-file
|
|
||||||
// must therefore humanize findings inside the nested scannerEnvelope; --raw stays raw.
|
|
||||||
describe('default mode --output-file (M-BUG-12: humanized findings)', () => {
|
|
||||||
it('writes humanized GAP findings (userActionLanguage defined) to the output file', async () => {
|
|
||||||
const tmp = join(tmpdir(), `ca-posture-outfile-${process.pid}.json`);
|
|
||||||
try {
|
|
||||||
await runPosture(['--output-file', tmp]);
|
|
||||||
const env = JSON.parse(await readFile(tmp, 'utf-8'));
|
|
||||||
const gap = env.scannerEnvelope.scanners.find(s => s.scanner === 'GAP');
|
|
||||||
assert.ok(gap, 'GAP scanner must be present in the output file');
|
|
||||||
assert.ok(gap.findings.length > 0, 'fixture must yield at least one GAP finding');
|
|
||||||
for (const f of gap.findings) {
|
|
||||||
assert.notEqual(f.userActionLanguage, undefined,
|
|
||||||
`${f.id}: default-mode output-file findings must carry userActionLanguage`);
|
|
||||||
assert.notEqual(f.userImpactCategory, undefined,
|
|
||||||
`${f.id}: default-mode output-file findings must carry userImpactCategory`);
|
|
||||||
}
|
|
||||||
} finally {
|
|
||||||
await unlink(tmp).catch(() => {});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
it('--raw --output-file keeps v5.0.0 raw finding shape (no humanizer fields)', async () => {
|
|
||||||
const tmp = join(tmpdir(), `ca-posture-outfile-raw-${process.pid}.json`);
|
|
||||||
try {
|
|
||||||
await runPosture(['--raw', '--output-file', tmp]);
|
|
||||||
const env = JSON.parse(await readFile(tmp, 'utf-8'));
|
|
||||||
for (const s of env.scannerEnvelope.scanners) {
|
|
||||||
for (const f of s.findings) {
|
|
||||||
assert.equal(f.userActionLanguage, undefined,
|
|
||||||
`${f.id}: --raw output-file must not carry userActionLanguage`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} finally {
|
|
||||||
await unlink(tmp).catch(() => {});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
|
||||||
|
|
@ -291,41 +291,3 @@ describe('RUL — user-global rule is not flagged "matches no files" (M-BUG-9 gu
|
||||||
assert.equal(dead.length, 0, `user-global rule wrongly flagged dead: ${dead.map(f => f.evidence).join(' | ')}`);
|
assert.equal(dead.length, 0, `user-global rule wrongly flagged dead: ${dead.map(f => f.evidence).join(' | ')}`);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('RUL — mid-pattern /**/ glob matches intermediate dirs (M-BUG-19)', () => {
|
|
||||||
// globToRegex restored the {{GLOBSTAR_SLASH}} placeholder to "(?:/.+/|/)"
|
|
||||||
// BEFORE the ? → [^/] replacement ran, corrupting the group opener "(?:"
|
|
||||||
// into "([^/]:" — so every pattern containing a mid-pattern "/**/" silently
|
|
||||||
// matched nothing but the zero-dir "|/" branch, and live rules like
|
|
||||||
// "posts/**/post.md" were flagged "matches no files".
|
|
||||||
let tmpRoot;
|
|
||||||
let result;
|
|
||||||
|
|
||||||
beforeEach(async () => {
|
|
||||||
resetCounter();
|
|
||||||
tmpRoot = await mkdtemp(join(tmpdir(), 'ca-rul-globstar-'));
|
|
||||||
await mkdir(join(tmpRoot, '.claude', 'rules'), { recursive: true });
|
|
||||||
await mkdir(join(tmpRoot, 'posts', '2026-01-23-slug'), { recursive: true });
|
|
||||||
await writeFile(join(tmpRoot, 'posts', '2026-01-23-slug', 'post.md'), '# Post\n', 'utf8');
|
|
||||||
await writeFile(
|
|
||||||
join(tmpRoot, '.claude', 'rules', 'post-scope.md'),
|
|
||||||
'---\npaths: "posts/**/post.md"\n---\n\n# Post rule\nbody\n',
|
|
||||||
'utf8',
|
|
||||||
);
|
|
||||||
const discovery = await discoverConfigFiles(tmpRoot);
|
|
||||||
result = await scan(tmpRoot, discovery);
|
|
||||||
});
|
|
||||||
|
|
||||||
afterEach(async () => {
|
|
||||||
if (tmpRoot) await rm(tmpRoot, { recursive: true, force: true });
|
|
||||||
});
|
|
||||||
|
|
||||||
it('does NOT flag a live posts/**/post.md rule as "matches no files"', () => {
|
|
||||||
const dead = result.findings.filter(f => f.title.includes('matches no files'));
|
|
||||||
assert.equal(
|
|
||||||
dead.length,
|
|
||||||
0,
|
|
||||||
`/**/ rule matched a real file but was flagged dead: ${dead.map(f => f.evidence).join(' | ')}`,
|
|
||||||
);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
|
||||||
|
|
@ -165,7 +165,7 @@
|
||||||
"id": "CA-GAP-003",
|
"id": "CA-GAP-003",
|
||||||
"scanner": "GAP",
|
"scanner": "GAP",
|
||||||
"severity": "low",
|
"severity": "low",
|
||||||
"title": "You haven't set up path-scoped rules yet",
|
"title": "Your rules all load on every conversation",
|
||||||
"description": "Path-scoped rules only load when you're working with files that match — keeps each conversation focused.",
|
"description": "Path-scoped rules only load when you're working with files that match — keeps each conversation focused.",
|
||||||
"file": null,
|
"file": null,
|
||||||
"line": null,
|
"line": null,
|
||||||
|
|
@ -309,7 +309,7 @@
|
||||||
"id": "CA-GAP-012",
|
"id": "CA-GAP-012",
|
||||||
"scanner": "GAP",
|
"scanner": "GAP",
|
||||||
"severity": "info",
|
"severity": "info",
|
||||||
"title": "You haven't set up subagent isolation yet",
|
"title": "Your subagents share Claude's main work folder",
|
||||||
"description": "Isolated subagents run in their own copy of the repo so they can't accidentally disturb your main work.",
|
"description": "Isolated subagents run in their own copy of the repo so they can't accidentally disturb your main work.",
|
||||||
"file": null,
|
"file": null,
|
||||||
"line": null,
|
"line": null,
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue