Kjell Tore Guttormsen
09f817977c
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).
The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:
- `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
- `$TMPFILE` read across blocks (tokens, manifest, whats-active,
plugin-health) — each command could not read the file it had just written
- `$GLOBAL_FLAG` across blocks (fix)
- `$TODAY` never assigned in any block (campaign), passing
`--reference-date ""` to a write CLI in six places
- three `$$` temp paths handed to the Read tool (fix), which expands neither
All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.
Also fixed, all confirmed against ground truth rather than inferred:
- `implement` printed a rollback ID it never captured (the timestamp lived only
inside a command substitution) — the one message a user reads after a bad run
- `plan` reported "No analysis results found" for valid sessions, because Read
was pointed at a glob it cannot expand; now uses Glob and verifies the
analysis report exists before spawning the agent
- five phase commands wrote state.yaml with two of four required fields; since
the agent writes all four, a follow-up write silently deleted the rest
- `implement` promised rollback deletes created files; rollback deliberately
leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
- `implement` claimed a score delta with no pre-change measurement
- `verifier-agent` was told to write a report it has no tool to write
- dead `Task` tool name in always-loaded rule context; planner-agent template
demonstrated the inline file content its own line 110 forbids
The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.
Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
94 lines
4.3 KiB
Markdown
94 lines
4.3 KiB
Markdown
---
|
|
name: config-audit:interview
|
|
description: Phase 3 - Interactive interview to gather user preferences
|
|
allowed-tools: Read, Write, Edit, AskUserQuestion, Bash
|
|
model: sonnet
|
|
---
|
|
|
|
# Config-Audit: Interview (Phase 3)
|
|
|
|
Gather user preferences to inform the action plan.
|
|
|
|
## IMPORTANT: Inline Execution Only
|
|
|
|
This command runs AskUserQuestion **directly in the main context** — NOT via an `Agent` subagent.
|
|
AskUserQuestion requires synchronous terminal interaction and does not work when delegated to an `Agent` subagent.
|
|
|
|
## Prerequisites
|
|
|
|
- Must have completed Phase 2 (analysis)
|
|
- Use the Read tool on the analysis at `~/.claude/config-audit/sessions/{session-id}/analysis-report.md`
|
|
|
|
## Arguments
|
|
|
|
- `$ARGUMENTS` may contain `--raw` — pass-through accepted for CLI surface consistency. Interview is interactive prose only (no scanner output, no findings prose), so `--raw` is a no-op here.
|
|
|
|
## Implementation Steps
|
|
|
|
0. **Parse flags**:
|
|
|
|
```bash
|
|
RAW_FLAG=""
|
|
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
|
```
|
|
|
|
1. **Resolve the session, then load its state**:
|
|
|
|
```
|
|
Glob: ~/.claude/config-audit/sessions/*/state.yaml
|
|
Sort by modification time — the most recently modified session wins
|
|
```
|
|
|
|
Every path below substitutes that session's id for `{session-id}`. Never guess
|
|
it: if the Glob returns nothing, say "No audit session found — run
|
|
`/config-audit discover` first" and exit. Read the session's `state.yaml` and
|
|
verify `completed_phases` contains `analyze`; if it doesn't, tell the user
|
|
analysis hasn't run yet and exit. Then read the analysis report for context.
|
|
2. **Conduct interview inline**: Use AskUserQuestion tool directly (never delegate it to a subagent via `Agent` — a subagent cannot hold the interactive turn). Adapt questions based on analysis findings.
|
|
3. **Save interview results**: Write to `~/.claude/config-audit/sessions/{session-id}/interview.md`
|
|
4. **Update state** (see state-management rule), with one bound specific to this
|
|
command: interview is optional and can be run against a session that already
|
|
moved past it. If `completed_phases` already contains a later phase (`plan`,
|
|
`implement`, `verify`), do **not** rewind `current_phase` and do not re-add a
|
|
phase already in `completed_phases` — append `interview` only if it is absent,
|
|
leave `current_phase`/`next_phase` pointing at the furthest phase reached, and
|
|
tell the user the preferences will apply the next time `/config-audit plan`
|
|
runs. Rewinding a finished session is how its progress gets lost. Always set
|
|
`updated_at` to the current timestamp, whichever branch above applies.
|
|
5. **Output summary**
|
|
|
|
## Interview Questions
|
|
|
|
Ask these using AskUserQuestion (skip questions that don't apply based on analysis). Where the analysis report references finding IDs, use the humanized title from the report rather than re-deriving prose:
|
|
|
|
1. **Config Style** — Centralized vs Distributed vs Hybrid organization
|
|
2. **Unused automation that runs at specific events** — Wire up, review individually, delete, or leave (only if the analysis report flagged one)
|
|
3. **Duplicate Permissions** — Remove from local, consolidate, or keep (only if found)
|
|
4. **Modular Rules** — Use .claude/rules/ pattern? Yes/No
|
|
5. **Path-Scoped Rules** — Which patterns (tests, src, config, docs) — only if Q4=Yes
|
|
6. **Conflict Resolution** — Per-conflict: global vs project vs custom value (only if conflicts found)
|
|
7. **Permission Audit** — Audit or keep (only if >30 patterns in settings.local.json)
|
|
8. **Project Inheritance** — Per-project: inherit or isolate (only if multiple projects)
|
|
|
|
## Adaptive Questioning
|
|
|
|
Skip questions that don't apply:
|
|
- No unused hooks question if all hooks are wired
|
|
- No duplicates question if no duplicates found
|
|
- No conflict questions if no conflicts detected
|
|
- No path-scoping if user said no to modular rules
|
|
- Fewer project questions if only one project
|
|
- No permission audit if <30 patterns
|
|
|
|
## Skip Interview Option
|
|
|
|
If user runs `/config-audit plan` without interview:
|
|
- Use sensible defaults (centralized, inherit, enable rules)
|
|
- Flag decisions in plan as "assumed"
|
|
|
|
## Error Handling
|
|
|
|
- If user selects "Other" for any question, ask follow-up with AskUserQuestion
|
|
- If interview is cancelled, save partial results
|
|
- If no analysis report found, report error and exit
|
|
- If AskUserQuestion fails, STOP — do not use alternative methods
|