Kjell Tore Guttormsen
09f817977c
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
3.4 KiB
3.4 KiB
| name | description | argument-hint | allowed-tools | model |
|---|---|---|---|---|
| config-audit:plugin-health | Audit plugin configuration quality — validates structure, frontmatter, and cross-plugin coherence | [plugin-path] | Read, Glob, Grep, Bash | sonnet |
Config-Audit: Plugin Health
Audit Claude Code plugin structure and quality — validates plugin.json, CLAUDE.md, command/agent frontmatter, and detects cross-plugin conflicts.
Arguments
$ARGUMENTSmay contain a path to a specific plugin directory- If omitted: scans all plugins in the marketplace root
--raw: pass-through to the scanner; produces v5.0.0 verbatim envelope (bypasses the humanizer) for byte-stable diff tooling
Implementation
Step 1: Discover plugins and greet
If a specific path is given, scan only that plugin. Otherwise, find all plugins using Glob for **/.claude-plugin/plugin.json.
Tell the user:
## Plugin Health Check
Auditing {N} plugin(s) for structure, frontmatter quality, and cross-plugin conflicts...
Step 2: Run scanner
Run silently for each plugin. Default mode writes a humanized JSON payload to --output-file where each PLH finding carries userImpactCategory, userActionLanguage, and relevanceContext alongside the v5.0.0 fields. --raw is passed through verbatim when present, and prints the byte-stable v5.0.0 envelope on stdout instead.
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/plugin-health-scanner.mjs <path> --output-file /tmp/config-audit-plugin-health.json $RAW_FLAG 2>/dev/null; echo $?
Read /tmp/config-audit-plugin-health.json with the Read tool. Exit codes 0, 1 and 2 are normal; only 3 is a real error.
The payload carries three things the report needs:
plugins[]— one row per plugin:name,declaredName,commandCount,agentCount,findingCount,score,grade. Use these for the table; never estimate a grade yourself.cross_plugin_findings[]— the namespace-collision and shared-command-name findings, already separated from the per-plugin ones (they also carrycrossPlugin: trueinfindings).findings[]— every finding, humanized.
Step 3: Present results
### Plugin Health Report
| Plugin | Grade | Commands | Agents | Status |
|--------|-------|----------|--------|--------|
| {plugins[].name} | {plugins[].grade} ({plugins[].score}) | {plugins[].commandCount} | {plugins[].agentCount} | {Good/Issues found} |
| ... | ... | ... | ... | ... |
{If cross-plugin issues:}
#### Cross-Plugin Issues ({count})
| Issue | Plugins | Recommendation |
|-------|---------|----------------|
| ... | ... | ... |
{If findings:}
#### Findings by Plugin
**{plugin-name}** ({finding_count} findings):
1. [{userActionLanguage}] {humanized title} ({id}) — {humanized recommendation}
2. ...
Group findings within each plugin by userImpactCategory (e.g., "Configuration mistake", "Conflict") and lead each line with userActionLanguage ("Fix this now", "Fix soon", "Optional cleanup"). The humanizer already produced the plain-language title/recommendation strings — render them verbatim, do not paraphrase.
Step 4: Suggest next steps
### What's next
- Fix structural issues based on recommendations above
- `/config-audit posture` — Full configuration posture assessment
- `/config-audit fix` — Auto-fix deterministic issues