Kjell Tore Guttormsen
1182f85767
Dogfooding `/config-audit drift` against the machine. Fasit written before the
run; 6/6 predictions plus both F7 arms confirmed, 0 deviations.
M-BUG-21 (both arms):
The arg loop ended in `else if (!arg.startsWith('-')) targetPath = arg` with no
unknown-flag branch, so an unrecognised flag was dropped silently and its VALUE
became the scan target. `--output-file /tmp/x.json` scanned /tmp/x.json — a path
that does not exist — and reported the near-empty scan as drift, forever. The
same silence was destructive for `--save --name` with the value omitted: the
name stayed `default` and an existing baseline was overwritten. And the flag
ux-rules rule 2 requires did not exist at all: commands/drift.md ran the CLI
under `2>/dev/null` while telling the agent to read stdout, but the default
report, the --save confirmation and --list all write to stderr. All three modes
captured nothing.
M-BUG-27 (found during the run, not predicted):
diff-engine never compared the baseline's stored target_path against the current
target. The machine's `default` baseline is anchored to a test fixture, so
`/config-audit drift` diffed two unrelated trees, marked all 20 baseline
findings resolved and all 15 current ones new, and reported trend "improving".
A reassuring, entirely false signal — and the default path.
Root cause is one thing, not three: the CLI validated neither its flags nor its
anchor. Same class as the rollback chunk's "nothing agreed where a backup lives".
Fix: unknown options and value-less --name/--baseline/--output-file exit 3;
--output-file follows the posture.mjs pattern; `_baselineAnchor` rides in the
default-mode payload (stderr alone is invisible under `2>/dev/null`) while
--json/--raw stdout stays v5.0.0-shaped.
Suite 1410/0 (+12). Frozen snapshots untouched; raw/json/default backcompat green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RGrL3noVdFSUhohaKTMSN
113 lines
4.5 KiB
Markdown
113 lines
4.5 KiB
Markdown
---
|
|
name: config-audit:drift
|
|
description: Compare current configuration against a saved baseline — shows new, resolved, and changed findings
|
|
argument-hint: "[path] [--baseline name] [--save]"
|
|
allowed-tools: Read, Write, Glob, Grep, Bash
|
|
model: sonnet
|
|
---
|
|
|
|
# Config-Audit: Drift Detection
|
|
|
|
Compare current configuration against a saved baseline to see what changed.
|
|
|
|
## Arguments
|
|
|
|
- `$ARGUMENTS` may contain:
|
|
- A target path (default: current working directory)
|
|
- `--save`: Save current state as baseline
|
|
- `--baseline <name>`: Compare against a specific named baseline (default: "default")
|
|
- `--raw`: Pass-through to the scanner; produces v5.0.0 verbatim diff output (bypasses the humanizer). Use when piping into v5.0.0-baseline diff tooling that depends on byte-stable output.
|
|
|
|
## Implementation
|
|
|
|
### Save a baseline
|
|
|
|
If `--save` is present:
|
|
|
|
Tell the user: **"Saving current configuration as baseline..."**
|
|
|
|
```bash
|
|
RAW_FLAG=""
|
|
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
|
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --save --name <baseline-name> --json $RAW_FLAG 2>/dev/null
|
|
```
|
|
|
|
`--save` writes its human confirmation to **stderr**, which `2>/dev/null` discards — pass `--json` so the `{saved, name, path}` object lands on stdout. Read stdout for confirmation. Tell the user:
|
|
|
|
```markdown
|
|
### Baseline Saved
|
|
|
|
Captured current state as baseline "{name}".
|
|
Run `/config-audit drift` anytime to see what changed since this point.
|
|
```
|
|
|
|
### Compare against baseline
|
|
|
|
Without `--save`:
|
|
|
|
Tell the user: **"Comparing current configuration against baseline..."**
|
|
|
|
```bash
|
|
RAW_FLAG=""
|
|
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
|
|
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --baseline <name> --output-file /tmp/config-audit-drift.json $RAW_FLAG 2>/dev/null; echo $?
|
|
```
|
|
|
|
Exit codes: `0` = stable/improving, `1` = degrading (both normal — present the result either way), `3` = a real error.
|
|
|
|
Then read `/tmp/config-audit-drift.json` with the **Read tool**. The default-mode report itself goes to stderr, so `--output-file` is the only way this command sees the diff at all.
|
|
|
|
**Check `_baselineAnchor` first.** If the baseline was saved from a different directory than the one being scanned, the diff is not a drift signal — every baseline finding shows as "resolved" and every current finding as "new", which renders as a falsely reassuring "improving" trend. When the anchor differs, say so plainly and offer to re-anchor with `/config-audit drift --save` instead of presenting the numbers as drift.
|
|
|
|
In default mode the diff sections are humanized — finding titles, descriptions, and recommendations have already been replaced with plain-language equivalents. New/resolved/changed finding lists carry `userImpactCategory`, `userActionLanguage`, and `relevanceContext` so you can group and prioritize without re-deriving severity prose. If `--raw` was passed, the v5.0.0 diff is verbatim — present it in a code block as-is.
|
|
|
|
If baseline not found, tell the user:
|
|
|
|
```
|
|
No baseline found. Save one first with:
|
|
/config-audit drift --save
|
|
```
|
|
|
|
Otherwise, parse and present the drift report. Use the Read tool on the captured stdout (or pipe it into a tmpfile first if you prefer):
|
|
|
|
```markdown
|
|
### Configuration Drift
|
|
|
|
**Trend:** {Improving|Degrading|Stable}
|
|
**Score:** {before} → {after} ({+/-delta} points)
|
|
|
|
{If new findings:}
|
|
#### New Issues ({count})
|
|
| ID | Action | Description |
|
|
|----|--------|-------------|
|
|
| {id} | {userActionLanguage — "Fix this now", "Fix soon", etc.} | {humanized title} |
|
|
|
|
{If resolved findings:}
|
|
#### Resolved ({count})
|
|
| ID | Description |
|
|
|----|-------------|
|
|
| {id} | {humanized title} |
|
|
|
|
{If area changes:}
|
|
#### Area Changes
|
|
| Area | Before | After | Change |
|
|
|------|--------|-------|--------|
|
|
| ... | ... | ... | ... |
|
|
```
|
|
|
|
When iterating new/resolved findings, prefer `userActionLanguage` over raw `severity` for the "Action" column — the humanizer already mapped severity to plain-language phrasing, and surfacing it consistently keeps the toolchain coherent. Mention `relevanceContext` when it isn't `affects-everyone` (the user wants to know if a fix touches shared config or just their machine).
|
|
|
|
### List baselines
|
|
|
|
If `$ARGUMENTS` contains `--list`:
|
|
|
|
```bash
|
|
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs --list 2>/dev/null
|
|
```
|
|
|
|
### What's next
|
|
|
|
After viewing drift:
|
|
- `/config-audit fix` — Auto-fix new findings
|
|
- `/config-audit posture` — Full posture assessment
|
|
- `/config-audit drift --save` — Update the baseline to current state
|