open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

fix(commands): stop assuming shell state survives between blocks

Browse source 
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).

The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:

  - `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
  - `$TMPFILE` read across blocks (tokens, manifest, whats-active,
    plugin-health) — each command could not read the file it had just written
  - `$GLOBAL_FLAG` across blocks (fix)
  - `$TODAY` never assigned in any block (campaign), passing
    `--reference-date ""` to a write CLI in six places
  - three `$$` temp paths handed to the Read tool (fix), which expands neither

All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.

Also fixed, all confirmed against ground truth rather than inferred:

  - `implement` printed a rollback ID it never captured (the timestamp lived only
    inside a command substitution) — the one message a user reads after a bad run
  - `plan` reported "No analysis results found" for valid sessions, because Read
    was pointed at a glob it cannot expand; now uses Glob and verifies the
    analysis report exists before spawning the agent
  - five phase commands wrote state.yaml with two of four required fields; since
    the agent writes all four, a follow-up write silently deleted the rest
  - `implement` promised rollback deletes created files; rollback deliberately
    leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
  - `implement` claimed a score delta with no pre-change measurement
  - `verifier-agent` was told to write a report it has no tool to write
  - dead `Task` tool name in always-loaded rule context; planner-agent template
    demonstrated the inline file content its own line 110 forbids

The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.

Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
This commit is contained in:
Kjell Tore Guttormsen 2026-08-01 20:12:17 +02:00
commit 09f817977c
17 changed files with 413 additions and 71 deletions
Download patch file Download diff file Expand all files Collapse all files

16
commands/fix.md
View file

@ -43,7 +43,7 @@ if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
# A placeholder in square brackets does not start with a dash, so the arg loop
# would take it as the scan/fix TARGET instead of a flag.
GLOBAL_FLAG=""
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs <path> --output-file /tmp/config-audit-fix-scan-$$.json $GLOBAL_FLAG $RAW_FLAG 2>/dev/null; echo $?
node ${CLAUDE_PLUGIN_ROOT}/scanners/scan-orchestrator.mjs <path> --output-file /tmp/config-audit-fix-scan.json $GLOBAL_FLAG $RAW_FLAG 2>/dev/null; echo $?
```
Exit code 3 → tell user: "Scanner error. Try `/config-audit posture` to check your configuration."
@ -53,12 +53,15 @@ Exit code 3 → tell user: "Scanner error. Try `/config-audit posture` to check
Run fix planner silently. The fix-cli emits humanized prose to stderr in default mode and v5.0.0-shape JSON to stdout when `--json` is set; we use `--json` here for structured data and let the humanizer-aware rendering layer (this command's prose output below) supply the plain-language wording from the scan envelope above:
```bash
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> $GLOBAL_FLAG --output-file /tmp/config-audit-fix-plan-$$.json 2>/dev/null; echo $?
# Re-assign here: each fenced block is its own Bash call, so the value
# set in Step 1 is empty by the time this block runs.
GLOBAL_FLAG="" # --global when the user asked for global scope
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> $GLOBAL_FLAG --output-file /tmp/config-audit-fix-plan.json 2>/dev/null; echo $?
```
Exit codes: 0 = plan produced, 2 = one or more fixes failed (apply step only), 3 = argument or tool error. On 3, show the stderr message — an unknown flag is rejected by design, not silently ignored.
Read `/tmp/config-audit-fix-plan-$$.json` using the Read tool. Cross-reference each fix-plan entry against the humanized scan envelope (`/tmp/config-audit-fix-scan-$$.json`) by finding ID to recover the humanized `title`/`description`/`recommendation` plus `userImpactCategory`/`userActionLanguage` for grouping.
Read `/tmp/config-audit-fix-plan.json` using the Read tool. Cross-reference each fix-plan entry against the humanized scan envelope (`/tmp/config-audit-fix-scan.json`) by finding ID to recover the humanized `title`/`description`/`recommendation` plus `userImpactCategory`/`userActionLanguage` for grouping.
### Step 3: Present fix plan
@ -104,10 +107,13 @@ AskUserQuestion:
If confirmed, apply:
```bash
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --apply $GLOBAL_FLAG --output-file /tmp/config-audit-fix-applied-$$.json 2>/dev/null; echo $?
# Re-assign here: each fenced block is its own Bash call, so the value
# set in Step 1 is empty by the time this block runs.
GLOBAL_FLAG="" # --global when the user asked for global scope
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --apply $GLOBAL_FLAG --output-file /tmp/config-audit-fix-applied.json 2>/dev/null; echo $?
```
Read `/tmp/config-audit-fix-applied-$$.json` with the Read tool to get applied/failed counts and the backup ID. Exit code 2 means at least one fix failed — report it; `failed[]` carries the reason per fix.
Read `/tmp/config-audit-fix-applied.json` with the Read tool to get applied/failed counts and the backup ID. Exit code 2 means at least one fix failed — report it; `failed[]` carries the reason per fix.
### Step 6: Show results