fix(drift): validate the arguments and the baseline anchor drift never checked

Dogfooding `/config-audit drift` against the machine. Fasit written before the
run; 6/6 predictions plus both F7 arms confirmed, 0 deviations.

M-BUG-21 (both arms):
The arg loop ended in `else if (!arg.startsWith('-')) targetPath = arg` with no
unknown-flag branch, so an unrecognised flag was dropped silently and its VALUE
became the scan target. `--output-file /tmp/x.json` scanned /tmp/x.json — a path
that does not exist — and reported the near-empty scan as drift, forever. The
same silence was destructive for `--save --name` with the value omitted: the
name stayed `default` and an existing baseline was overwritten. And the flag
ux-rules rule 2 requires did not exist at all: commands/drift.md ran the CLI
under `2>/dev/null` while telling the agent to read stdout, but the default
report, the --save confirmation and --list all write to stderr. All three modes
captured nothing.

M-BUG-27 (found during the run, not predicted):
diff-engine never compared the baseline's stored target_path against the current
target. The machine's `default` baseline is anchored to a test fixture, so
`/config-audit drift` diffed two unrelated trees, marked all 20 baseline
findings resolved and all 15 current ones new, and reported trend "improving".
A reassuring, entirely false signal — and the default path.

Root cause is one thing, not three: the CLI validated neither its flags nor its
anchor. Same class as the rollback chunk's "nothing agreed where a backup lives".

Fix: unknown options and value-less --name/--baseline/--output-file exit 3;
--output-file follows the posture.mjs pattern; `_baselineAnchor` rides in the
default-mode payload (stderr alone is invisible under `2>/dev/null`) while
--json/--raw stdout stays v5.0.0-shaped.

Suite 1410/0 (+12). Frozen snapshots untouched; raw/json/default backcompat green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015RGrL3noVdFSUhohaKTMSN
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 18:20:04 +02:00
commit 1182f85767
5 changed files with 293 additions and 32 deletions

View file

@ -5,6 +5,34 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Fixed
- **`M-BUG-21``drift-cli.mjs` had no `--output-file`, and its argument loop turned the missing
flag into a wrong scan target.** The loop ended in `else if (!arg.startsWith('-')) targetPath = arg`
with no unknown-flag branch, so an unrecognised flag was dropped silently and its *value* fell
through to the scan target: `drift-cli.mjs . --output-file /tmp/x.json` scanned `/tmp/x.json`, a
path that does not exist, and reported the resulting near-empty scan as drift — permanently, and
without a warning. The same silence was destructive for `--save --name` with the value omitted:
`--name` was ignored, the name stayed `default`, and an existing baseline was **overwritten**.
Unknown options and value-less `--name`/`--baseline`/`--output-file` now exit `3`.
- **`M-BUG-21` (second arm) — `/config-audit drift` captured nothing at all.** `commands/drift.md`
ran the CLI under `2>/dev/null` and told the agent to "read stdout", but the default-mode report,
the `--save` confirmation, and the `--list` output all go to **stderr**. All three modes returned
empty. `--output-file` now writes the diff (humanized in default mode, raw under `--json`/`--raw`,
matching `posture.mjs`), and the command reads that file; `--save` passes `--json` for its
confirmation.
- **`M-BUG-27``drift` compared against baselines anchored to a different directory and called it
"improving".** `diff-engine` never checked the baseline's stored `target_path` against the current
scan target. Diffing a repo against a baseline saved elsewhere marked every baseline finding
"resolved" and every current finding "new" — a 100% phantom diff surfacing as a *reassuring* trend,
on the **default** baseline. The CLI now warns on stderr in every mode and carries
`_baselineAnchor {matches, baselineTarget, currentTarget}` in the default-mode payload, so a caller
running under `2>/dev/null` can still see it. `--json`/`--raw` stdout stays v5.0.0-shaped and the
frozen `drift.json` snapshot is untouched.
**1410** tests (+12). No count change (scanners **16**, agents **7**, commands **21**, hooks **4**).
## [5.13.0] - 2026-07-31
### Summary

View file

@ -12,7 +12,7 @@
![Commands](https://img.shields.io/badge/commands-21-green)
![Agents](https://img.shields.io/badge/agents-7-orange)
![Hooks](https://img.shields.io/badge/hooks-4-red)
![Tests](https://img.shields.io/badge/tests-1398-brightgreen)
![Tests](https://img.shields.io/badge/tests-1410-brightgreen)
![License](https://img.shields.io/badge/license-MIT-lightgrey)
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 16 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, cross-plugin collision, output-style, and always-loaded agent-listing-budget detection. Zero external dependencies.

View file

@ -29,10 +29,10 @@ Tell the user: **"Saving current configuration as baseline..."**
```bash
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --save --name <baseline-name> $RAW_FLAG 2>/dev/null
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --save --name <baseline-name> --json $RAW_FLAG 2>/dev/null
```
Read stdout for confirmation. Tell the user:
`--save` writes its human confirmation to **stderr**, which `2>/dev/null` discards — pass `--json` so the `{saved, name, path}` object lands on stdout. Read stdout for confirmation. Tell the user:
```markdown
### Baseline Saved
@ -50,10 +50,16 @@ Tell the user: **"Comparing current configuration against baseline..."**
```bash
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --baseline <name> $RAW_FLAG 2>/dev/null
node ${CLAUDE_PLUGIN_ROOT}/scanners/drift-cli.mjs <path> --baseline <name> --output-file /tmp/config-audit-drift.json $RAW_FLAG 2>/dev/null; echo $?
```
Read stdout. In default mode the diff sections are humanized — finding titles, descriptions, and recommendations have already been replaced with plain-language equivalents. New/resolved/changed finding lists carry `userImpactCategory`, `userActionLanguage`, and `relevanceContext` so you can group and prioritize without re-deriving severity prose. If `--raw` was passed, the v5.0.0 diff is verbatim — present it in a code block as-is.
Exit codes: `0` = stable/improving, `1` = degrading (both normal — present the result either way), `3` = a real error.
Then read `/tmp/config-audit-drift.json` with the **Read tool**. The default-mode report itself goes to stderr, so `--output-file` is the only way this command sees the diff at all.
**Check `_baselineAnchor` first.** If the baseline was saved from a different directory than the one being scanned, the diff is not a drift signal — every baseline finding shows as "resolved" and every current finding as "new", which renders as a falsely reassuring "improving" trend. When the anchor differs, say so plainly and offer to re-anchor with `/config-audit drift --save` instead of presenting the numbers as drift.
In default mode the diff sections are humanized — finding titles, descriptions, and recommendations have already been replaced with plain-language equivalents. New/resolved/changed finding lists carry `userImpactCategory`, `userActionLanguage`, and `relevanceContext` so you can group and prioritize without re-deriving severity prose. If `--raw` was passed, the v5.0.0 diff is verbatim — present it in a code block as-is.
If baseline not found, tell the user:

View file

@ -5,17 +5,22 @@
* Compare current configuration against a saved baseline.
* Usage:
* node drift-cli.mjs <path> --save [--name my-baseline]
* node drift-cli.mjs <path> [--baseline my-baseline] [--json]
* node drift-cli.mjs <path> [--baseline my-baseline] [--json] [--output-file path]
* node drift-cli.mjs --list
* Unknown options and value-less --name/--baseline/--output-file exit 3.
* Zero external dependencies.
*/
import { resolve } from 'node:path';
import { writeFile } from 'node:fs/promises';
import { runAllScanners } from './scan-orchestrator.mjs';
import { diffEnvelopes, formatDiffReport } from './lib/diff-engine.mjs';
import { saveBaseline, loadBaseline, listBaselines } from './lib/baseline.mjs';
import { humanizeFindings } from './lib/humanizer.mjs';
const BOOL_FLAGS = ['--save', '--list', '--json', '--raw', '--global'];
const VALUE_FLAGS = ['--name', '--baseline', '--output-file'];
async function main() {
const args = process.argv.slice(2);
let targetPath = '.';
@ -25,24 +30,39 @@ async function main() {
let jsonMode = false;
let rawMode = false;
let includeGlobal = false;
let outputFile = null;
// M-BUG-21: this loop used to end in `else if (!arg.startsWith('-')) targetPath = arg`,
// with no unknown-flag branch. An unrecognised flag was dropped silently and its
// VALUE fell through to targetPath — so `--output-file /tmp/x.json` scanned
// /tmp/x.json, a path that does not exist, yielding a near-empty scan and
// therefore permanent phantom drift. A missing value for --name was equally
// silent and destructive: it left baselineName at 'default' and OVERWROTE the
// default baseline. Both now fail loudly (exit 3) instead.
for (let i = 0; i < args.length; i++) {
if (args[i] === '--save') {
save = true;
} else if (args[i] === '--name' && args[i + 1]) {
baselineName = args[++i];
} else if (args[i] === '--baseline' && args[i + 1]) {
baselineName = args[++i];
} else if (args[i] === '--list') {
list = true;
} else if (args[i] === '--json') {
jsonMode = true;
} else if (args[i] === '--raw') {
rawMode = true;
} else if (args[i] === '--global') {
includeGlobal = true;
} else if (!args[i].startsWith('-')) {
targetPath = args[i];
const arg = args[i];
if (BOOL_FLAGS.includes(arg)) {
if (arg === '--save') save = true;
else if (arg === '--list') list = true;
else if (arg === '--json') jsonMode = true;
else if (arg === '--raw') rawMode = true;
else if (arg === '--global') includeGlobal = true;
} else if (VALUE_FLAGS.includes(arg)) {
const value = args[i + 1];
if (value === undefined || value.startsWith('-')) {
throw new Error(`Option ${arg} requires a value.`);
}
if (arg === '--name' || arg === '--baseline') baselineName = value;
else outputFile = value;
i++;
} else if (arg.startsWith('-')) {
throw new Error(
`Unknown option: ${arg}\n` +
`Valid options: ${[...BOOL_FLAGS, ...VALUE_FLAGS].join(' ')}`
);
} else {
targetPath = arg;
}
}
@ -106,6 +126,26 @@ async function main() {
process.exit(1);
}
// M-BUG-27: a baseline carries the path it was saved from, but nothing ever
// compared it against the current target. Diffing a repo against a baseline
// anchored elsewhere produced 100% phantom drift — every baseline finding
// "resolved", every current finding "new" — and reported it as trend
// "improving": a reassuring and entirely false signal, on the DEFAULT
// baseline. The warning goes to stderr in every mode; stdout stays
// byte-identical to the frozen v5.0.0 shape.
const baselineTarget = baseline._baseline?.target_path || '';
const currentTarget = resolve(targetPath);
const anchorMatches = !baselineTarget || baselineTarget === currentTarget;
if (baselineTarget && baselineTarget !== currentTarget) {
process.stderr.write(
`\nWarning: baseline "${baselineName}" was saved from a different target path.\n` +
` baseline: ${baselineTarget}\n` +
` current: ${currentTarget}\n` +
` The two scans cover different trees, so this diff is not a drift signal.\n` +
` Re-anchor with: drift-cli.mjs ${currentTarget} --save --name ${baselineName}\n\n`
);
}
// Run current scan
const current = await runAllScanners(targetPath, {
includeGlobal,
@ -115,22 +155,39 @@ async function main() {
// Diff
const diff = diffEnvelopes(baseline, current);
// Default mode: humanize finding-bearing diff fields before report rendering.
// `_baselineAnchor` rides here and NOT in the raw shape: commands/drift.md runs
// the CLI under `2>/dev/null`, so the stderr warning above never reaches the
// caller that has to act on it. --json/--raw stay v5.0.0-shaped.
const humanizedDiff = {
...diff,
_baselineAnchor: { matches: anchorMatches, baselineTarget, currentTarget },
newFindings: humanizeFindings(diff.newFindings || []),
resolvedFindings: humanizeFindings(diff.resolvedFindings || []),
unchangedFindings: humanizeFindings(diff.unchangedFindings || []),
movedFindings: humanizeFindings(diff.movedFindings || []),
};
if (jsonMode || rawMode) {
// --json and --raw both write the raw v5.0.0-shape diff (byte-identical).
process.stdout.write(JSON.stringify(diff, null, 2) + '\n');
} else {
// Default mode: humanize finding-bearing diff fields before report rendering.
const humanizedDiff = {
...diff,
newFindings: humanizeFindings(diff.newFindings || []),
resolvedFindings: humanizeFindings(diff.resolvedFindings || []),
unchangedFindings: humanizeFindings(diff.unchangedFindings || []),
movedFindings: humanizeFindings(diff.movedFindings || []),
};
const report = formatDiffReport(humanizedDiff);
process.stderr.write('\n' + report + '\n');
}
// ux-rules rule 2: every scanner Bash call uses `--output-file <path>` and the
// command reads the file with the Read tool. drift-cli had no such flag, and
// its default-mode report goes to stderr — which commands/drift.md discarded
// via `2>/dev/null` while instructing the agent to "read stdout". The command
// captured nothing. Matches posture.mjs: raw diff in --json/--raw, humanized
// otherwise; stdout is unaffected.
if (outputFile) {
const fileDiff = (jsonMode || rawMode) ? diff : humanizedDiff;
await writeFile(outputFile, JSON.stringify(fileDiff, null, 2), 'utf-8');
process.stderr.write(`\nResults written to ${outputFile}\n`);
}
// Exit code: 0=stable/improving, 1=degrading
if (diff.summary.trend === 'degrading') process.exit(1);
process.exit(0);

View file

@ -2,9 +2,11 @@ import { describe, it, afterEach } from 'node:test';
import assert from 'node:assert/strict';
import { resolve, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { execFileSync } from 'node:child_process';
import { execFileSync, spawnSync } from 'node:child_process';
import { mkdtempSync, readFileSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { deleteBaseline } from '../../scanners/lib/baseline.mjs';
import { hermeticEnv, withHermeticHome } from '../helpers/hermetic-home.mjs';
import { hermeticEnv, withHermeticHome, HERMETIC_HOME } from '../helpers/hermetic-home.mjs';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
const FIXTURES = resolve(__dirname, '../fixtures');
@ -81,3 +83,171 @@ describe('drift-cli compare', () => {
});
});
});
// ---------------------------------------------------------------------------
// M-BUG-21 — argument validation.
//
// The arg loop ended in `else if (!args[i].startsWith('-')) targetPath = args[i]`,
// so an UNKNOWN flag was dropped silently and its VALUE became the scan target.
// `drift-cli.mjs . --output-file /tmp/x.json` scanned /tmp/x.json — a path that
// does not exist — producing a near-empty scan and therefore phantom drift
// against the baseline, permanently and without a single warning.
// ---------------------------------------------------------------------------
/** Run the CLI capturing both streams regardless of exit code. */
function spawnCapture(args) {
const res = spawnSync('node', [DRIFT_CLI, ...args], RUN);
return { status: res.status, stdout: String(res.stdout || ''), stderr: String(res.stderr || '') };
}
/** Run the CLI expecting a non-zero exit; returns { status, stderr }. */
function runExpectingFailure(args) {
try {
execFileSync('node', [DRIFT_CLI, ...args], RUN);
return { status: 0, stderr: '' };
} catch (err) {
return { status: err.status, stderr: String(err.stderr || '') };
}
}
describe('drift-cli argument validation (M-BUG-21)', () => {
it('rejects an unknown flag instead of swallowing its value as the scan target', () => {
const { status, stderr } = runExpectingFailure([HEALTHY, '--bogus', 'some-value', '--json']);
assert.equal(status, 3, 'unknown flag must fail loudly, not scan "some-value"');
assert.match(stderr, /unknown option/i);
assert.match(stderr, /--bogus/);
});
it('rejects --name without a value instead of silently writing the default baseline', () => {
// The destructive case: `--save --name` (value missing) fell through to the
// default baseline name and OVERWROTE an existing baseline without warning.
const { status, stderr } = runExpectingFailure([HEALTHY, '--save', '--name']);
assert.equal(status, 3);
assert.match(stderr, /--name/);
assert.match(stderr, /requires a value/i);
});
it('rejects --baseline without a value', () => {
const { status, stderr } = runExpectingFailure([HEALTHY, '--baseline']);
assert.equal(status, 3);
assert.match(stderr, /--baseline/);
assert.match(stderr, /requires a value/i);
});
it('still accepts a bare path as the scan target', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE, '--json'], RUN);
const out = JSON.parse(
execFileSync('node', [DRIFT_CLI, HEALTHY, '--baseline', TEST_BASELINE, '--json'], RUN)
);
assert.equal(out.summary.trend, 'stable');
});
});
// ---------------------------------------------------------------------------
// ux-rules rule 2 — every scanner Bash call uses `--output-file <path>`.
// drift-cli had no such flag, and its default-mode report goes to STDERR, so
// commands/drift.md ("... 2>/dev/null" + "Read stdout") captured NOTHING.
// ---------------------------------------------------------------------------
describe('drift-cli --output-file (ux-rules rule 2)', () => {
const outDir = mkdtempSync(join(tmpdir(), 'drift-outfile-'));
it('writes the diff to the given path in --json mode', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const target = join(outDir, 'diff-json.json');
execFileSync('node', [DRIFT_CLI, HEALTHY, '--baseline', TEST_BASELINE, '--json', '--output-file', target], RUN);
assert.ok(existsSync(target), '--output-file must create the file');
const written = JSON.parse(readFileSync(target, 'utf-8'));
assert.ok('summary' in written);
assert.ok('newFindings' in written);
});
it('writes the diff to the given path in default mode', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const target = join(outDir, 'diff-default.json');
execFileSync('node', [DRIFT_CLI, HEALTHY, '--baseline', TEST_BASELINE, '--output-file', target], RUN);
assert.ok(existsSync(target), 'default mode must honour --output-file too');
const written = JSON.parse(readFileSync(target, 'utf-8'));
assert.ok('summary' in written);
});
it('does not treat the --output-file value as the scan target', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const target = join(outDir, 'not-a-scan-target.json');
const stdout = execFileSync(
'node', [DRIFT_CLI, HEALTHY, '--baseline', TEST_BASELINE, '--json', '--output-file', target], RUN
);
// Scanning HEALTHY against a baseline of HEALTHY is stable. If the output
// path had been swallowed as the target, this would be phantom drift.
assert.equal(JSON.parse(stdout).summary.trend, 'stable');
});
});
// ---------------------------------------------------------------------------
// M-BUG-27 — baseline anchor guard.
//
// diff-engine never compared the baseline's target_path against the current
// scan target. Comparing a repo against a baseline saved from an unrelated
// directory yielded 100% phantom drift reported as trend "improving" — a
// reassuring, entirely false signal, and the DEFAULT behaviour (--baseline
// default). The warning goes to stderr in every mode; --raw stdout stays
// byte-identical to the frozen v5.0.0 snapshot.
// ---------------------------------------------------------------------------
describe('drift-cli baseline anchor guard (M-BUG-27)', () => {
const OTHER = resolve(FIXTURES, 'conflict-project');
it('warns when the baseline was anchored to a different target path', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const res = spawnCapture([OTHER, '--baseline', TEST_BASELINE, '--json']);
assert.match(res.stderr, /different (target|path)/i, 'must warn about the anchor mismatch');
assert.match(res.stderr, /healthy-project/, 'warning should name the baseline target');
});
it('does not warn when the target paths match', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const res = spawnCapture([HEALTHY, '--baseline', TEST_BASELINE, '--json']);
assert.doesNotMatch(res.stderr, /different (target|path)/i);
});
it('records the anchor mismatch in the --output-file payload (default mode)', () => {
// The stderr warning alone is not enough: commands/drift.md runs the CLI
// under `2>/dev/null`, so a stderr-only warning is invisible to the very
// caller that needs it. The machine-readable flag must ride in the file.
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const target = join(mkdtempSync(join(tmpdir(), 'drift-anchor-')), 'diff.json');
spawnCapture([OTHER, '--baseline', TEST_BASELINE, '--output-file', target]);
const written = JSON.parse(readFileSync(target, 'utf-8'));
assert.ok(written._baselineAnchor, '_baselineAnchor must be present');
assert.equal(written._baselineAnchor.matches, false);
assert.match(written._baselineAnchor.baselineTarget, /healthy-project/);
assert.match(written._baselineAnchor.currentTarget, /conflict-project/);
});
it('marks the anchor as matching when targets agree', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const target = join(mkdtempSync(join(tmpdir(), 'drift-anchor-ok-')), 'diff.json');
execFileSync('node', [DRIFT_CLI, HEALTHY, '--baseline', TEST_BASELINE, '--output-file', target], RUN);
const written = JSON.parse(readFileSync(target, 'utf-8'));
assert.equal(written._baselineAnchor.matches, true);
});
it('keeps --raw stdout free of the warning (frozen v5.0.0 contract)', () => {
execFileSync('node', [DRIFT_CLI, HEALTHY, '--save', '--name', TEST_BASELINE], RUN);
const res = spawnCapture([OTHER, '--baseline', TEST_BASELINE, '--raw']);
const parsed = JSON.parse(res.stdout);
assert.ok(!('baselineAnchor' in parsed), '--raw stdout must stay v5.0.0-shaped');
assert.ok(!('targetMismatch' in parsed), '--raw stdout must stay v5.0.0-shaped');
});
});