Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
206 lines
7.4 KiB
Markdown
206 lines
7.4 KiB
Markdown
---
|
|
name: config-audit:implement
|
|
description: Phase 5 - Execute action plan with backups and verification
|
|
allowed-tools: Read, Write, Edit, Bash, Agent, AskUserQuestion
|
|
model: opus
|
|
---
|
|
|
|
# Config-Audit: Implementation (Phase 5)
|
|
|
|
Execute the action plan with full backup, verification, and rollback support.
|
|
|
|
## Prerequisites
|
|
|
|
- Must have completed Phase 4 (plan)
|
|
- Action plan at `~/.claude/config-audit/sessions/{session-id}/action-plan.md`
|
|
|
|
## Arguments
|
|
|
|
- `$ARGUMENTS` may contain `--raw` to forward to the implementer-agent's instructions; in `--raw` mode the agent renders v5.0.0 verbatim severity prefiks instead of humanized `userActionLanguage` urgency phrasing.
|
|
|
|
## Implementation
|
|
|
|
### Step 1: Parse flags, load and verify
|
|
|
|
Check whether `$ARGUMENTS` contains `--raw`. Carry the answer yourself: the agent
|
|
prompt in Step 4 is **not** a shell, so a variable assigned in a bash block cannot
|
|
be referenced from it. Substitute `{mode}` literally with `--raw` or `humanized`.
|
|
|
|
Find the most recent session with a plan (use the **Glob tool** for
|
|
`~/.claude/config-audit/sessions/*/state.yaml`, then Read the newest match — Read
|
|
does not expand `*`). If none: "No action plan found. Run `/config-audit plan` first."
|
|
|
|
Use the Read tool on the action plan and count actions. Tell the user:
|
|
|
|
```
|
|
## Implementing Action Plan
|
|
|
|
Found {N} actions to execute across {M} files.
|
|
A backup will be created before any changes are made.
|
|
```
|
|
|
|
### Step 2: Get user approval
|
|
|
|
```
|
|
AskUserQuestion:
|
|
question: "Ready to implement {N} actions? Backup created automatically — you can roll back with one command."
|
|
options:
|
|
- "Yes, proceed"
|
|
- "Review plan first" (then show the plan file path)
|
|
- "Cancel"
|
|
```
|
|
|
|
### Step 3: Create backup
|
|
|
|
Create backup silently, and **print the backup ID** — Step 6 has to tell the user
|
|
how to roll back, and a timestamp that only ever existed inside a command
|
|
substitution cannot be quoted later. Shell state does not survive to the next
|
|
block, so capture the printed value and substitute it literally from here on:
|
|
|
|
```bash
|
|
BACKUP_ID=$(date +%Y%m%d_%H%M%S)
|
|
mkdir -p ~/.claude/config-audit/backups/"$BACKUP_ID"/files/ 2>/dev/null
|
|
echo "$BACKUP_ID"
|
|
```
|
|
|
|
Use the printed ID wherever `{backup-id}` appears below. Never invent or re-derive
|
|
it with a second `date` call — a run that straddles a second boundary would hand
|
|
the user a rollback ID that does not exist.
|
|
|
|
Copy each file to be modified. Generate `manifest.yaml` with checksums.
|
|
|
|
The manifest is what `/config-audit rollback` reads, so it MUST carry both lists:
|
|
|
|
```yaml
|
|
files: # pre-existing files this run will MODIFY
|
|
- backup: files/root/CLAUDE.md
|
|
original: /abs/path/CLAUDE.md
|
|
sha256: <sha256 of the pre-change content>
|
|
created: # files this run will CREATE (no backup can exist)
|
|
- /abs/path/.claude/rules/post-quality.md
|
|
```
|
|
|
|
Record every `create`-type action under `created:`. Rollback cannot restore a
|
|
file that never existed, but it must be able to tell the user which files it is
|
|
leaving behind — a half-restored target is only dangerous when it is silent.
|
|
|
|
Tell the user: **"Backup created. Implementing actions..."**
|
|
|
|
### Step 4: Execute actions
|
|
|
|
Group actions by dependencies. For each group, spawn implementer agents (batch of 3):
|
|
|
|
```
|
|
Agent(subagent_type: "config-audit:implementer-agent")
|
|
model: sonnet
|
|
prompt: |
|
|
Execute action: {action-id}
|
|
File: {file-path}, Type: {create|modify|delete}
|
|
Mode: {mode} ("humanized" = humanized progress prose; "--raw" = v5.0.0 verbatim)
|
|
Details: {changes}
|
|
Verify backup exists, make change, validate syntax.
|
|
When logging progress, use the humanized title/userActionLanguage
|
|
fields from the action plan (the planner already rendered them) —
|
|
do not re-derive severity prose. Append result to:
|
|
~/.claude/config-audit/sessions/{session-id}/implementation-log.md
|
|
Append with Bash `>>` (heredoc) — NEVER the Write tool on this log;
|
|
parallel agents share it and a full-file Write clobbers their entries.
|
|
```
|
|
|
|
Show progress between groups using the humanized titles already present in the action plan:
|
|
|
|
```
|
|
Action 1/N: {humanized title} — done
|
|
Action 2/N: {humanized title} — done
|
|
...
|
|
```
|
|
|
|
### Step 5: Verify results
|
|
|
|
Spawn verifier agent:
|
|
|
|
```
|
|
Agent(subagent_type: "config-audit:verifier-agent")
|
|
model: sonnet (note: using sonnet, not haiku)
|
|
prompt: |
|
|
Verify all changes from implementation:
|
|
1. Modified files exist and are syntactically valid
|
|
2. New files created correctly
|
|
3. No new conflicts introduced
|
|
Return your findings as your final message. Do NOT write them to a file —
|
|
this agent is read-only by design (tools: Read, Glob, Grep) and has no
|
|
write tool; instructing it to write a report is a contract it cannot keep.
|
|
```
|
|
|
|
Append the verifier's returned findings to the log yourself, with Bash `>>`
|
|
(heredoc) — never the Write tool, for the same reason as Step 4:
|
|
|
|
```bash
|
|
cat >> ~/.claude/config-audit/sessions/{session-id}/implementation-log.md <<'EOF'
|
|
## Verification
|
|
{verifier findings}
|
|
EOF
|
|
```
|
|
|
|
If verifier finds issues: one retry with implementer agent. If still failing: report and suggest rollback.
|
|
|
|
### Step 6: Present results
|
|
|
|
```markdown
|
|
### Implementation Complete
|
|
|
|
**{succeeded} succeeded** | {failed} failed | {skipped} skipped
|
|
|
|
{If failed > 0:}
|
|
{failed} action(s) couldn't be completed — see log for details.
|
|
|
|
**Backup location:** `~/.claude/config-audit/backups/{backup-id}/`
|
|
**Rollback:** `/config-audit rollback {backup-id}`
|
|
**Full log:** `~/.claude/config-audit/sessions/{session-id}/implementation-log.md`
|
|
```
|
|
|
|
**On reporting a score.** Only quote a grade *change* if the pre-change grade was
|
|
actually captured before Step 4 ran. Once the files are edited, only the new grade
|
|
is measurable — a delta computed after the fact has no source and must not be
|
|
invented. To offer one, measure first in Step 1 and again here:
|
|
|
|
```bash
|
|
node ${CLAUDE_PLUGIN_ROOT}/scanners/posture.mjs <target-path> --output-file /tmp/config-audit-implement-posture.json 2>/dev/null; echo $?
|
|
```
|
|
|
|
Then Read `/tmp/config-audit-implement-posture.json`. Both the `--output-file` and
|
|
the `2>/dev/null` are required by the output rules — a bare scanner call would put
|
|
diagnostic output in front of the user. If no pre-change grade was captured, report
|
|
the new grade alone and say nothing about a delta.
|
|
|
|
### Step 7: Update state
|
|
|
|
Update `state.yaml` with all four fields `.claude/rules/state-management.md` requires:
|
|
|
|
- `current_phase: "implement"`
|
|
- `completed_phases`: append `implement` to the existing array (read it first; never replace it)
|
|
- `next_phase: null`
|
|
- `updated_at`: current timestamp
|
|
|
|
A full-file Write that names only two of the four silently deletes the other two.
|
|
|
|
## Rollback
|
|
|
|
If the user requests rollback at any point:
|
|
1. Read `manifest.yaml` from backup
|
|
2. Restore each file and verify checksums
|
|
3. **Report — do not delete — the files this run created.** Rollback restores from
|
|
backup, and no backup can exist for a file that did not exist before. Those
|
|
paths stay on disk; `/config-audit rollback` lists them under "Left in place"
|
|
so the user can remove them deliberately. Promising deletion here would leave a
|
|
half-restored config that reads as a clean rollback.
|
|
4. Update state to `rolled_back`
|
|
|
|
## Error Handling
|
|
|
|
| Error | What happens |
|
|
|-------|-------------|
|
|
| Permission denied | Skip action, log it, continue with others |
|
|
| File not found | Skip action, log it, continue |
|
|
| Invalid syntax after edit | Rollback that single file, log, continue |
|
|
| Critical failure | Offer full rollback |
|