config-audit/commands/drift.md
Kjell Tore Guttormsen 0b763f25c1 fix(commands): router dogfood — five seam defects, plus the placeholder class
Dogfooding `/config-audit` (the router) against the repo, fasit written before
any run (docs/router-fasit.local.md, untouched). Every claim below is measured
behaviour, not a reading of the source.

1. Bare `<target-path>` inside the step-3 fence is a shell REDIRECTION, not an
   argument. Measured in zsh: both CLIs failed before starting, no output file
   was written, and the echoed status was 1 — inside the band the router's own
   gate calls "continue normally". Quoting makes an unsubstituted placeholder
   reach argv, so it fails in the CLI where the exit code means something.
   Swept the whole class: 30 sites across 12 further command files, since a
   defect in one file is a class until the opposite is measured. New guard:
   command-placeholder-shell-safety.test.mjs.

2. The orchestrator's exit code was discarded. Two commands on one line share a
   single trailing `echo $?`, which reports only the last: measured, an
   orchestrator exit 3 echoed as posture's 0, so the "3 -> stop" gate could
   never fire. Both statuses are now captured and echoed.

3. "Running 12 configuration scanners" — the orchestrator registers 16. The new
   test binds the narrated count to the registry so the next scanner added
   cannot re-stale it silently.

4. The Area Breakdown table hardcoded 7 rows; posture emits 9 quality areas.
   Token Efficiency (a B on this repo) and Plugin Hygiene never reached the
   user. Rows added, and the row set is now asserted against lib/scoring.mjs.
   Label aligned: "MCP Servers" -> "MCP", as posture emits it.

5. Step 6 rendered "the headline line from the humanized stderr scorecard" and
   forbade deriving a replacement — while step 3 sent posture's stderr to
   /dev/null, as UX rule 2 requires, and the prose is absent from the JSON
   payload (measured). The slot could only be improvised. posture's stderr now
   goes to a file in the session dir, as commands/posture.md already did; the
   user still never sees raw scanner output.

Also: `grep -q -- "--raw"` matched any argument CONTAINING --raw (measured on
`--rawdog` and on a path with --raw in it) — anchored to whole arguments.
SCOPE_FLAGS renamed SCOPE_FLAG, since zsh does not word-split and the plural
invited the M-BUG-45 shape.

command-shell-state-shape.test.mjs only recognised line-initial assignments, so
it reported the idiomatic `node …; STATUS=$?` capture as never assigned. Widened
to assignments after a separator; verified it still fails on a real cross-block
reference before trusting it.

Suite 1477 -> 1483, frozen v5.0.0 snapshots untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
2026-08-09 21:18:05 +02:00

4.9 KiB

name description argument-hint allowed-tools model
config-audit:drift Compare current configuration against a saved baseline — shows new, resolved, and changed findings [path] [--baseline name] [--save] Read, Write, Glob, Grep, Bash sonnet

Config-Audit: Drift Detection

Compare current configuration against a saved baseline to see what changed.

Arguments

  • $ARGUMENTS may contain:
    • A target path (default: current working directory)
    • --save: Save current state as baseline
    • --baseline <name>: Compare against a specific named baseline (default: "default")
    • --raw: Pass-through to the scanner; produces v5.0.0 verbatim diff output (bypasses the humanizer). Use when piping into v5.0.0-baseline diff tooling that depends on byte-stable output.

Implementation

Save a baseline

If --save is present:

Tell the user: "Saving current configuration as baseline..."

RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs "<path>" --save --name "<baseline-name>" --json $RAW_FLAG 2>/dev/null

--save writes its human confirmation to stderr, which 2>/dev/null discards — pass --json so the {saved, name, path} object lands on stdout. Read stdout for confirmation. Tell the user:

### Baseline Saved

Captured current state as baseline "{name}".
Run `/config-audit drift` anytime to see what changed since this point.

Compare against baseline

Without --save:

Tell the user: "Comparing current configuration against baseline..."

RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs "<path>" --baseline "<name>" --output-file /tmp/config-audit-drift.json $RAW_FLAG >/dev/null 2>/dev/null; echo $?

Exit codes: 0 = stable/improving, 1 = degrading (both normal — present the result either way), 3 = a real error.

Then read /tmp/config-audit-drift.json with the Read tool. The default-mode report itself goes to stderr, so --output-file is the only way this command sees the diff at all.

Check _baselineAnchor first. If the baseline was saved from a different directory than the one being scanned, the diff is not a drift signal — every baseline finding shows as "resolved" and every current finding as "new", which renders as a falsely reassuring "improving" trend. When the anchor differs, say so plainly and offer to re-anchor with /config-audit drift --save instead of presenting the numbers as drift.

In default mode the diff sections are humanized — finding titles, descriptions, and recommendations have already been replaced with plain-language equivalents. New/resolved/changed finding lists carry userImpactCategory, userActionLanguage, and relevanceContext so you can group and prioritize without re-deriving severity prose. If --raw was passed, the v5.0.0 diff is verbatim — present it in a code block as-is.

If baseline not found, tell the user:

No baseline found. Save one first with:
  /config-audit drift --save

Otherwise, parse and present the drift report. Use the Read tool on the captured stdout (or pipe it into a tmpfile first if you prefer):

### Configuration Drift

**Trend:** {Improving|Degrading|Stable}
**Score:** {before} → {after} ({+/-delta} points)

{If new findings:}
#### New Issues ({count})
| ID | Action | Description |
|----|--------|-------------|
| {id} | {userActionLanguage — "Fix this now", "Fix soon", etc.} | {humanized title} |

{If resolved findings:}
#### Resolved ({count})
| ID | Description |
|----|-------------|
| {id} | {humanized title} |

{If area changes:}
#### Area Changes
| Area | Before | After | Change |
|------|--------|-------|--------|
| ... | ... | ... | ... |

When iterating new/resolved findings, prefer userActionLanguage over raw severity for the "Action" column — the humanizer already mapped severity to plain-language phrasing, and surfacing it consistently keeps the toolchain coherent. Mention relevanceContext when it isn't affects-everyone (the user wants to know if a fix touches shared config or just their machine).

List baselines

If $ARGUMENTS contains --list:

node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs --list --output-file /tmp/config-audit-baselines.json 2>/dev/null; echo $?

The human-readable listing goes to stderr, which 2>/dev/null discards — read /tmp/config-audit-baselines.json with the Read tool and render the baselines array (name, findingCount, savedAt) as a table. If the array is empty, tell the user no baselines are saved yet and point at /config-audit drift --save.

What's next

After viewing drift:

  • /config-audit fix — Auto-fix new findings
  • /config-audit posture — Full posture assessment
  • /config-audit drift --save — Update the baseline to current state