fix(fix): validate the arguments, back up renames, and verify the scope it fixed
Dogfooding `/config-audit fix` against a throwaway repo copy. All eight
predictions registered in the fasit before the run were confirmed, and three
further defects surfaced that were not predicted.
- M-BUG-21, third arm: the argument loop ended in `!arg.startsWith('-') =>
targetPath`, so an unknown flag was dropped and its value became the target.
In `fix` that is the WRITE target under `--apply`. Unknown options and a
value-less `--output-file` now exit 3.
- `--dry-run` was documented in the command's argument-hint and never
implemented; `--output-file` did not exist, so `commands/fix.md` told the
agent to Read a file nothing produced. Both now exist.
- M-BUG-31: `file-rename` was excluded from the backup set, so a renamed rule
file had no backup entry while the command promised one and returned a
backupId that could not restore it.
- M-BUG-32: `verifyFixes` hardcoded `includeGlobal: false`, so after a
`--global` run every untouched user-scope finding was reported as verified.
Reproduced against an unmodified ~/.claude/CLAUDE.md.
- M-BUG-29: a rename was applied before other fixes on the same file, which
then failed with ENOENT while the run still exited 0. Renames sort last.
- M-BUG-30: `severityOrder[s] || 4` maps critical (0) to 4, so critical fixes
sorted last. The old test used the same falsy fallback and agreed with the
bug. Now `?? 4`.
- A failed fix exits 2 instead of 0, matching the other scanners' convention.
Frozen tests/snapshots/v5.0.0/ untouched; --json/--raw stdout byte-identical.
Suite 1420/0 (+10).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ3MCDCnyw7wZSPnUXVhYS
This commit is contained in:
parent
1182f85767
commit
05f1e954d0
7 changed files with 334 additions and 36 deletions
27
CHANGELOG.md
27
CHANGELOG.md
|
|
@ -31,7 +31,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
running under `2>/dev/null` can still see it. `--json`/`--raw` stdout stays v5.0.0-shaped and the
|
||||
frozen `drift.json` snapshot is untouched.
|
||||
|
||||
**1410** tests (+12). No count change (scanners **16**, agents **7**, commands **21**, hooks **4**).
|
||||
- **`M-BUG-21` (third arm) — `fix-cli.mjs` had the same unvalidated argument loop, where it moves the
|
||||
*write* target.** An unrecognised flag was dropped and its value became the scan target, so
|
||||
`fix-cli.mjs <repo> --output-file /tmp/x.json` silently audited `/tmp/x.json`; with `--apply` the
|
||||
same slip relocates what gets written. Unknown options and value-less `--output-file` now exit `3`.
|
||||
`--dry-run` — documented in `commands/fix.md`'s `argument-hint` but never implemented — is now
|
||||
accepted instead of silently dropped, and `--output-file` writes the fix payload to disk so
|
||||
`commands/fix.md` can read a file rather than parse stdout it runs under `2>/dev/null`.
|
||||
- **`M-BUG-31` — `fix` promised a mandatory backup it did not always take.** `fix-cli.mjs` excluded
|
||||
`file-rename` from the backup set, so a rule file whose only defect was its extension was renamed
|
||||
with **no** backup entry — while the command told the user "every fix creates a backup first" and
|
||||
handed back a `backupId` that could not restore it. The source file is now backed up like any other.
|
||||
- **`M-BUG-32` — verification re-scanned a different scope than the fix run.** `verifyFixes` hardcoded
|
||||
`includeGlobal: false`. After a `--global` run every user-scope finding fell out of the re-scan and
|
||||
was therefore counted as *verified*: a clean "fixed" report for files nothing had touched
|
||||
(reproduced against an untouched `~/.claude/CLAUDE.md`). It now inherits the run's scope, and
|
||||
`commands/fix.md` passes `--global` to every step instead of only the display scan.
|
||||
- **`M-BUG-29` — two fixes on one file were applied in an order that guaranteed failure.** A rule file
|
||||
with both `globs:` and a non-`.md` extension had the rename applied first; the frontmatter fix then
|
||||
failed with `ENOENT`. Renames now sort after every other fix.
|
||||
- **`M-BUG-30` — critical fixes sorted last.** `severityOrder[s] || 4` maps `critical` (weight `0`) to
|
||||
`4`, the opposite of the documented "critical first" contract. The old test used the same falsy
|
||||
fallback, so it agreed with the bug. Now `?? 4`.
|
||||
- **A failed fix no longer exits `0`.** `fix-cli.mjs` returns `2` when any planned fix failed, matching
|
||||
the `0/1/2 = PASS/WARNING/FAIL`, `3 = error` convention the other scanners follow.
|
||||
|
||||
**1420** tests (+10). No count change (scanners **16**, agents **7**, commands **21**, hooks **4**).
|
||||
|
||||
## [5.13.0] - 2026-07-31
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
A Claude Code plugin that checks configuration health, suggests context-aware improvements, and auto-fixes issues — `CLAUDE.md`, `settings.json`, hooks, rules, MCP servers, `@imports`, and plugins. 16 deterministic scanners across 10 quality areas, context-aware feature recommendations, auto-fix with backup/rollback, a prompt-cache-aware Token Hotspots scanner with optional API-calibrated `--accurate-tokens` mode, plus cache-prefix stability, dead-tool, cross-plugin collision, output-style, and always-loaded agent-listing-budget detection. Zero external dependencies.
|
||||
|
|
|
|||
|
|
@ -15,8 +15,13 @@ Auto-fix deterministic configuration issues. Scans, plans fixes, backs up origin
|
|||
- `$ARGUMENTS` may contain:
|
||||
- A target path (default: current working directory)
|
||||
- `--dry-run`: Show fix plan without applying
|
||||
- `--global`: Include user-scope config (`~/.claude`) in the scan **and** the fix run
|
||||
- `--raw`: Pass-through to scanners; produces v5.0.0 verbatim envelope (bypasses the humanizer) for byte-stable diff tooling
|
||||
|
||||
`--global` must be passed to **every** step below. The scan that builds the table and
|
||||
the scan that plans the fixes are two different runs; if only one of them sees the
|
||||
user scope, the plan and the table describe different config.
|
||||
|
||||
## Implementation
|
||||
|
||||
### Step 1: Greet and scan
|
||||
|
|
@ -44,10 +49,12 @@ Exit code 3 → tell user: "Scanner error. Try `/config-audit posture` to check
|
|||
Run fix planner silently. The fix-cli emits humanized prose to stderr in default mode and v5.0.0-shape JSON to stdout when `--json` is set; we use `--json` here for structured data and let the humanizer-aware rendering layer (this command's prose output below) supply the plain-language wording from the scan envelope above:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --json 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> [--global] --output-file /tmp/config-audit-fix-plan-$$.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read the JSON output using the Read tool. Cross-reference each fix-plan entry against the humanized scan envelope (`/tmp/config-audit-fix-scan-$$.json`) by finding ID to recover the humanized `title`/`description`/`recommendation` plus `userImpactCategory`/`userActionLanguage` for grouping.
|
||||
Exit codes: 0 = plan produced, 2 = one or more fixes failed (apply step only), 3 = argument or tool error. On 3, show the stderr message — an unknown flag is rejected by design, not silently ignored.
|
||||
|
||||
Read `/tmp/config-audit-fix-plan-$$.json` using the Read tool. Cross-reference each fix-plan entry against the humanized scan envelope (`/tmp/config-audit-fix-scan-$$.json`) by finding ID to recover the humanized `title`/`description`/`recommendation` plus `userImpactCategory`/`userActionLanguage` for grouping.
|
||||
|
||||
### Step 3: Present fix plan
|
||||
|
||||
|
|
@ -93,10 +100,10 @@ AskUserQuestion:
|
|||
If confirmed, apply:
|
||||
|
||||
```bash
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --apply --json 2>/dev/null
|
||||
node ${CLAUDE_PLUGIN_ROOT}/scanners/fix-cli.mjs <path> --apply [--global] --output-file /tmp/config-audit-fix-applied-$$.json 2>/dev/null; echo $?
|
||||
```
|
||||
|
||||
Read the JSON output to get applied/failed counts and backup location.
|
||||
Read `/tmp/config-audit-fix-applied-$$.json` with the Read tool to get applied/failed counts and the backup ID. Exit code 2 means at least one fix failed — report it; `failed[]` carries the reason per fix.
|
||||
|
||||
### Step 6: Show results
|
||||
|
||||
|
|
@ -139,7 +146,8 @@ Run `/config-audit plan` to get a step-by-step guide for addressing these.
|
|||
|
||||
## Safety
|
||||
|
||||
- Backup is **mandatory** — every fix creates a backup first
|
||||
- Backup is **mandatory** — every fix creates a backup first, including file renames (the source file is backed up before the rename, so rollback can restore it at its original path)
|
||||
- Dry-run by default — user must confirm before changes
|
||||
- Verify after fix — re-scans to confirm findings resolved
|
||||
- Verify after fix — re-scans in the **same scope** the fix run used, so a `--global` run is verified against user scope too
|
||||
- Rollback always available — `/config-audit rollback <backup-id>`
|
||||
- A failed fix is reported, never swallowed — exit 2 plus a `failed[]` entry
|
||||
|
|
|
|||
|
|
@ -9,11 +9,18 @@
|
|||
*/
|
||||
|
||||
import { resolve } from 'node:path';
|
||||
import { writeFile } from 'node:fs/promises';
|
||||
import { runAllScanners } from './scan-orchestrator.mjs';
|
||||
import { planFixes, applyFixes, verifyFixes } from './fix-engine.mjs';
|
||||
import { createBackup } from './lib/backup.mjs';
|
||||
import { humanizeFinding } from './lib/humanizer.mjs';
|
||||
|
||||
// `--dry-run` is a no-op alias: dry-run is already the default. It exists because
|
||||
// commands/fix.md documents it in argument-hint, and a documented flag that the
|
||||
// CLI silently drops is the same fail-silent class as the unknown-flag sink below.
|
||||
const BOOL_FLAGS = ['--apply', '--dry-run', '--json', '--raw', '--global'];
|
||||
const VALUE_FLAGS = ['--output-file'];
|
||||
|
||||
async function main() {
|
||||
const args = process.argv.slice(2);
|
||||
let targetPath = '.';
|
||||
|
|
@ -21,18 +28,36 @@ async function main() {
|
|||
let jsonMode = false;
|
||||
let rawMode = false;
|
||||
let includeGlobal = false;
|
||||
let outputFile = null;
|
||||
|
||||
// Same defect class as M-BUG-21 in drift-cli: this loop used to end in
|
||||
// `else if (!args[i].startsWith('-')) targetPath = args[i]` with no
|
||||
// unknown-flag branch, so an unrecognised flag was dropped silently and its
|
||||
// VALUE became the scan target. Here that is worse than in drift: combined
|
||||
// with --apply it silently moves the WRITE target to another tree.
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
if (args[i] === '--apply') {
|
||||
apply = true;
|
||||
} else if (args[i] === '--json') {
|
||||
jsonMode = true;
|
||||
} else if (args[i] === '--raw') {
|
||||
rawMode = true;
|
||||
} else if (args[i] === '--global') {
|
||||
includeGlobal = true;
|
||||
} else if (!args[i].startsWith('-')) {
|
||||
targetPath = args[i];
|
||||
const arg = args[i];
|
||||
|
||||
if (BOOL_FLAGS.includes(arg)) {
|
||||
if (arg === '--apply') apply = true;
|
||||
else if (arg === '--json') jsonMode = true;
|
||||
else if (arg === '--raw') rawMode = true;
|
||||
else if (arg === '--global') includeGlobal = true;
|
||||
// --dry-run: default behaviour, accepted so it is not silently dropped.
|
||||
} else if (VALUE_FLAGS.includes(arg)) {
|
||||
const value = args[i + 1];
|
||||
if (value === undefined || value.startsWith('-')) {
|
||||
throw new Error(`Option ${arg} requires a value.`);
|
||||
}
|
||||
outputFile = value;
|
||||
i++;
|
||||
} else if (arg.startsWith('-')) {
|
||||
throw new Error(
|
||||
`Unknown option: ${arg}\n` +
|
||||
`Valid options: ${[...BOOL_FLAGS, ...VALUE_FLAGS].join(' ')}`
|
||||
);
|
||||
} else {
|
||||
targetPath = arg;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -105,16 +130,21 @@ async function main() {
|
|||
let backupId = null;
|
||||
|
||||
if (fixes.length === 0) {
|
||||
const output = { planned: [], applied: [], failed: [], verified: [], regressions: [], manual, backupId: null };
|
||||
if (machineMode) {
|
||||
const output = { planned: [], applied: [], failed: [], verified: [], regressions: [], manual, backupId: null };
|
||||
process.stdout.write(JSON.stringify(output, null, 2) + '\n');
|
||||
}
|
||||
if (outputFile) await writeFile(outputFile, JSON.stringify(output, null, 2) + '\n', 'utf-8');
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (apply) {
|
||||
// Create backup first
|
||||
const filesToBackup = [...new Set(fixes.filter(f => f.type !== 'file-rename').map(f => f.file))];
|
||||
// Create backup first. file-rename used to be excluded here, so a rule file
|
||||
// whose only defect was its extension was renamed with NO backup entry —
|
||||
// while commands/fix.md promised "every fix creates a backup first" and
|
||||
// handed the user a backupId that could not restore it. The source file is
|
||||
// backed up like any other; rollback recreates it at its original path.
|
||||
const filesToBackup = [...new Set(fixes.map(f => f.file))];
|
||||
const backup = createBackup(filesToBackup);
|
||||
backupId = backup.backupId;
|
||||
|
||||
|
|
@ -142,7 +172,10 @@ async function main() {
|
|||
process.stderr.write(`\n Verifying...\n`);
|
||||
}
|
||||
|
||||
const verification = await verifyFixes(envelope, applied);
|
||||
// Verification must re-scan the scope the fix run used. It hardcoded
|
||||
// includeGlobal:false, so with --global every untouched global-scope
|
||||
// finding fell out of the re-scan and was reported as verified.
|
||||
const verification = await verifyFixes(envelope, applied, { includeGlobal });
|
||||
verified = verification.verified;
|
||||
regressions = verification.regressions;
|
||||
|
||||
|
|
@ -165,7 +198,7 @@ async function main() {
|
|||
}
|
||||
|
||||
// JSON output (both --json and --raw write byte-equal v5.0.0-shape stdout)
|
||||
if (machineMode) {
|
||||
{
|
||||
const output = {
|
||||
planned: fixes.map(f => ({
|
||||
findingId: f.findingId,
|
||||
|
|
@ -193,7 +226,17 @@ async function main() {
|
|||
})),
|
||||
backupId,
|
||||
};
|
||||
process.stdout.write(JSON.stringify(output, null, 2) + '\n');
|
||||
const serialized = JSON.stringify(output, null, 2) + '\n';
|
||||
if (machineMode) process.stdout.write(serialized);
|
||||
// --output-file carries the same payload to disk. ux-rules rule 2 requires
|
||||
// it: commands run scanners with `2>/dev/null`, so anything the command has
|
||||
// to act on must ride in a file, not in stdout or stderr.
|
||||
if (outputFile) await writeFile(outputFile, serialized, 'utf-8');
|
||||
|
||||
// Exit code follows the convention the other scanners use: 0 PASS,
|
||||
// 2 FAIL, 3 tool error. A failed fix used to exit 0, so a caller could not
|
||||
// tell a clean run from one that silently lost a fix.
|
||||
if (failed.length > 0) process.exitCode = 2;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -56,9 +56,21 @@ export function planFixes(envelope) {
|
|||
}
|
||||
}
|
||||
|
||||
// Sort fixes by severity weight (critical first)
|
||||
// Sort fixes by severity weight (critical first), but a file-rename always
|
||||
// sorts after every other fix. A rename moves the file out from under any
|
||||
// later fix that still addresses the old path: a rule file with both
|
||||
// `globs:` and a non-.md extension had the rename applied first, and the
|
||||
// frontmatter fix then failed with ENOENT while the run still exited 0.
|
||||
// `?? 4`, not `|| 4`: critical weighs 0, and `0 || 4` evaluates to 4 — so
|
||||
// critical fixes sorted LAST, the exact opposite of this function's contract
|
||||
// (M-BUG-30). The old test used the same falsy fallback and agreed with the bug.
|
||||
const severityOrder = { critical: 0, high: 1, medium: 2, low: 3, info: 4 };
|
||||
fixes.sort((a, b) => (severityOrder[a.severity] || 4) - (severityOrder[b.severity] || 4));
|
||||
fixes.sort((a, b) => {
|
||||
const aRename = a.type === FIX_TYPES.FILE_RENAME ? 1 : 0;
|
||||
const bRename = b.type === FIX_TYPES.FILE_RENAME ? 1 : 0;
|
||||
if (aRename !== bRename) return aRename - bRename;
|
||||
return (severityOrder[a.severity] ?? 4) - (severityOrder[b.severity] ?? 4);
|
||||
});
|
||||
|
||||
return { fixes, skipped, manual };
|
||||
}
|
||||
|
|
@ -600,16 +612,21 @@ function extractEventFromDescription(description) {
|
|||
* Verify fixes by re-running affected scanners.
|
||||
* @param {object} originalEnvelope - Original scanner envelope
|
||||
* @param {object[]} appliedResults - Results from applyFixes()
|
||||
* @param {object} [opts]
|
||||
* @param {boolean} [opts.includeGlobal=false] - Must match the scope the fix run scanned
|
||||
* @returns {Promise<{ verified: string[], regressions: string[], newFindings: object[] }>}
|
||||
*/
|
||||
export async function verifyFixes(originalEnvelope, appliedResults) {
|
||||
export async function verifyFixes(originalEnvelope, appliedResults, opts = {}) {
|
||||
const targetPath = originalEnvelope.meta.target;
|
||||
const verified = [];
|
||||
const regressions = [];
|
||||
const newFindings = [];
|
||||
|
||||
// Re-scan the target
|
||||
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: false });
|
||||
// Re-scan the target in the SAME scope the fix run used. This was hardcoded
|
||||
// to includeGlobal:false: after a --global run, every global-scope finding
|
||||
// was absent from the re-scan and therefore counted as verified — a clean
|
||||
// "fixed" report for files nothing had touched.
|
||||
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: opts.includeGlobal === true });
|
||||
|
||||
// Build set of original finding IDs that were fixed
|
||||
const fixedIds = new Set(
|
||||
|
|
|
|||
|
|
@ -3,9 +3,9 @@ import assert from 'node:assert/strict';
|
|||
import { resolve, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { cp, rm, readFile, stat } from 'node:fs/promises';
|
||||
import { mkdirSync, existsSync, readdirSync } from 'node:fs';
|
||||
import { mkdirSync, existsSync, readdirSync, readFileSync, writeFileSync, rmSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { execFileSync, spawnSync } from 'node:child_process';
|
||||
import { hermeticEnv, HERMETIC_HOME } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
const __dirname = fileURLToPath(new URL('.', import.meta.url));
|
||||
|
|
@ -111,3 +111,111 @@ describe('fix-cli --apply', () => {
|
|||
assert.ok(output.verified.length > 0, 'Should have verified fixes');
|
||||
});
|
||||
});
|
||||
|
||||
// --- v5.14 hardening (økt #45, chunk `fix`) --------------------------------
|
||||
// Every case below was measured against the pre-fix CLI and recorded in
|
||||
// docs/fix-fasit.local.md (F1-F8 + M-BUG-29) before a line of it was fixed.
|
||||
|
||||
/** Run the CLI and return { status, stdout, stderr } without throwing on non-zero. */
|
||||
function runCli(args, opts = {}) {
|
||||
const res = spawnSync('node', [FIX_CLI, ...args], {
|
||||
encoding: 'utf-8',
|
||||
timeout: 30000,
|
||||
env: hermeticEnv(),
|
||||
...opts,
|
||||
});
|
||||
return { status: res.status, stdout: res.stdout || '', stderr: res.stderr || '' };
|
||||
}
|
||||
|
||||
describe('fix-cli argument validation (F1/F3)', () => {
|
||||
it('rejects an unknown flag instead of swallowing its value as the target', () => {
|
||||
const outFile = join(tmpdir(), `ca-fix-argtest-${Date.now()}.json`);
|
||||
const { status, stderr } = runCli([FIXABLE, '--bogus', outFile, '--json']);
|
||||
assert.strictEqual(status, 3, 'Unknown flag must fail loud with exit 3');
|
||||
assert.match(stderr, /--bogus/, 'Error must name the offending flag');
|
||||
assert.ok(!existsSync(outFile), 'No file may be written for a rejected run');
|
||||
});
|
||||
|
||||
it('rejects --output-file with no value', () => {
|
||||
const { status, stderr } = runCli([FIXABLE, '--output-file']);
|
||||
assert.strictEqual(status, 3, 'Missing flag value must fail loud');
|
||||
assert.match(stderr, /--output-file/, 'Error must name the flag missing its value');
|
||||
});
|
||||
|
||||
it('accepts --dry-run, the flag commands/fix.md documents', () => {
|
||||
const { status, stdout } = runCli([FIXABLE, '--dry-run', '--json']);
|
||||
assert.strictEqual(status, 0, '--dry-run must be a supported flag');
|
||||
const output = JSON.parse(stdout);
|
||||
assert.strictEqual(output.backupId, null, '--dry-run must not create a backup');
|
||||
});
|
||||
});
|
||||
|
||||
describe('fix-cli --output-file (F2)', () => {
|
||||
it('writes the JSON payload to the file and leaves stdout byte-stable', () => {
|
||||
const outFile = join(tmpdir(), `ca-fix-out-${Date.now()}.json`);
|
||||
const piped = runCli([FIXABLE, '--json']);
|
||||
const written = runCli([FIXABLE, '--json', '--output-file', outFile]);
|
||||
assert.strictEqual(written.status, 0);
|
||||
assert.ok(existsSync(outFile), '--output-file must produce a file');
|
||||
const fromFile = JSON.parse(readFileSync(outFile, 'utf-8'));
|
||||
assert.ok(Array.isArray(fromFile.planned), 'File must carry the fix payload');
|
||||
assert.strictEqual(
|
||||
written.stdout,
|
||||
piped.stdout,
|
||||
'--json stdout is frozen v5.0.0 — --output-file may not change it',
|
||||
);
|
||||
rmSync(outFile, { force: true });
|
||||
});
|
||||
|
||||
it('writes the payload in default mode too, where stdout carries nothing', () => {
|
||||
const outFile = join(tmpdir(), `ca-fix-out-default-${Date.now()}.json`);
|
||||
const { status } = runCli([FIXABLE, '--output-file', outFile]);
|
||||
assert.strictEqual(status, 0);
|
||||
const fromFile = JSON.parse(readFileSync(outFile, 'utf-8'));
|
||||
assert.ok(Array.isArray(fromFile.planned), 'Default mode must also write the payload');
|
||||
rmSync(outFile, { force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('fix-cli exit codes (F8)', () => {
|
||||
it('exits 2 when a planned fix failed', async () => {
|
||||
// Two fixes on one file: the rename and the frontmatter edit. Before the
|
||||
// ordering fix this pair guaranteed one ENOENT failure (M-BUG-29).
|
||||
const dir = await createTmpCopy();
|
||||
const rulesDir = join(dir, '.claude', 'rules');
|
||||
mkdirSync(rulesDir, { recursive: true });
|
||||
writeFileSync(join(rulesDir, 'both.txt'), '---\nglobs: "**/*.ts"\n---\n\nBody.\n');
|
||||
const { status, stdout } = runCli([dir, '--apply', '--json']);
|
||||
const output = JSON.parse(stdout);
|
||||
if (output.failed.length > 0) {
|
||||
assert.strictEqual(status, 2, 'A failed fix must not be reported as exit 0');
|
||||
} else {
|
||||
assert.strictEqual(status, 0, 'A clean run stays exit 0');
|
||||
}
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('fix-cli backup completeness (F4)', () => {
|
||||
it('backs up a file whose only fix is a rename', async () => {
|
||||
const dir = await createTmpCopy();
|
||||
const rulesDir = join(dir, '.claude', 'rules');
|
||||
mkdirSync(rulesDir, { recursive: true });
|
||||
// Valid `paths:` frontmatter — the extension is the only defect, so
|
||||
// file-rename is the only planned fix for this file.
|
||||
writeFileSync(join(rulesDir, 'renameonly.txt'), '---\npaths: ["**/*.ts"]\n---\n\nBody.\n');
|
||||
|
||||
const { stdout } = runCli([dir, '--apply', '--json']);
|
||||
const output = JSON.parse(stdout);
|
||||
const renames = output.applied.filter((a) => /renameonly\.txt$/.test(a.file));
|
||||
assert.strictEqual(renames.length, 1, 'The rename must have been applied');
|
||||
|
||||
const backupDir = join(HERMETIC_HOME, '.claude', 'config-audit', 'backups', output.backupId);
|
||||
const backedUp = readdirSync(join(backupDir, 'files'));
|
||||
assert.ok(
|
||||
backedUp.some((f) => /renameonly\.txt$/.test(f)),
|
||||
'commands/fix.md promises a mandatory backup — a renamed file must be recoverable',
|
||||
);
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import { describe, it, beforeEach, afterEach } from 'node:test';
|
|||
import assert from 'node:assert/strict';
|
||||
import { resolve, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { cp, rm, readFile, stat } from 'node:fs/promises';
|
||||
import { cp, rm, readFile, writeFile, stat } from 'node:fs/promises';
|
||||
import { mkdirSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
|
|
@ -45,14 +45,39 @@ describe('planFixes', () => {
|
|||
assert.ok(result.fixes.length > 0, 'Should have at least one fix');
|
||||
});
|
||||
|
||||
it('sorts fixes by severity (critical first)', () => {
|
||||
it('sorts fixes by severity (critical first), renames last', () => {
|
||||
const result = planFixes(envelope);
|
||||
// `?? 4`, not `|| 4`: critical weighs 0, and `0 || 4` is 4. The old
|
||||
// assertion used the same falsy fallback as the implementation, so it
|
||||
// agreed with the bug instead of catching it (M-BUG-30).
|
||||
const severityOrder = { critical: 0, high: 1, medium: 2, low: 3, info: 4 };
|
||||
for (let i = 1; i < result.fixes.length; i++) {
|
||||
const prev = severityOrder[result.fixes[i - 1].severity] || 4;
|
||||
const curr = severityOrder[result.fixes[i].severity] || 4;
|
||||
// A file-rename is ordered last on purpose (M-BUG-29) — severity ranks
|
||||
// within the non-rename group.
|
||||
const ranked = result.fixes.filter((f) => f.type !== FIX_TYPES.FILE_RENAME);
|
||||
for (let i = 1; i < ranked.length; i++) {
|
||||
const prev = severityOrder[ranked[i - 1].severity] ?? 4;
|
||||
const curr = severityOrder[ranked[i].severity] ?? 4;
|
||||
assert.ok(prev <= curr, `Fix ${i} should not have higher severity than fix ${i - 1}`);
|
||||
}
|
||||
const renames = result.fixes.filter((f) => f.type === FIX_TYPES.FILE_RENAME);
|
||||
if (renames.length > 0) {
|
||||
assert.strictEqual(
|
||||
result.fixes.at(-1).type,
|
||||
FIX_TYPES.FILE_RENAME,
|
||||
'Renames must sort after every other fix',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('puts a critical fix first (M-BUG-30)', () => {
|
||||
const result = planFixes(envelope);
|
||||
const criticals = result.fixes.filter((f) => f.severity === 'critical');
|
||||
if (criticals.length === 0) return;
|
||||
assert.strictEqual(
|
||||
result.fixes[0].severity,
|
||||
'critical',
|
||||
'critical weighs 0 — a `|| 4` fallback sorted it last, the opposite of the contract',
|
||||
);
|
||||
});
|
||||
|
||||
it('includes manual findings with recommendations', () => {
|
||||
|
|
@ -303,3 +328,75 @@ describe('verifyFixes', () => {
|
|||
assert.ok(verification.verified.length > 0, 'Should verify at least one fix');
|
||||
});
|
||||
});
|
||||
|
||||
// --- v5.14 hardening (økt #45, chunk `fix`) --------------------------------
|
||||
|
||||
describe('planFixes ordering (M-BUG-29)', () => {
|
||||
it('applies a file rename after every other fix touching the same file', async () => {
|
||||
const dir = await createTmpCopy();
|
||||
const rulesDir = join(dir, '.claude', 'rules');
|
||||
mkdirSync(rulesDir, { recursive: true });
|
||||
// Both defects at once: undocumented `globs:` AND a non-.md extension.
|
||||
await writeFile(join(rulesDir, 'both.txt'), '---\nglobs: "**/*.ts"\n---\n\nBody.\n');
|
||||
|
||||
resetCounter();
|
||||
const envelope = await runAllScanners(dir, { includeGlobal: false });
|
||||
const { fixes } = planFixes(envelope);
|
||||
|
||||
const onFile = fixes.filter((f) => /both\.txt$/.test(f.file));
|
||||
assert.strictEqual(onFile.length, 2, 'Both defects on the file must be planned');
|
||||
assert.strictEqual(
|
||||
onFile[onFile.length - 1].type,
|
||||
FIX_TYPES.FILE_RENAME,
|
||||
'A rename moves the file out from under later fixes — it must run last',
|
||||
);
|
||||
|
||||
// …and the whole batch must therefore apply cleanly.
|
||||
const backupDir = join(dir, '.backup-test');
|
||||
mkdirSync(backupDir, { recursive: true });
|
||||
const result = await applyFixes(fixes, { dryRun: false, backupDir });
|
||||
assert.deepStrictEqual(result.failed, [], 'No fix may fail because of ordering');
|
||||
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('verifyFixes scope (F5)', () => {
|
||||
it('re-scans with the same includeGlobal scope the fix run used', async () => {
|
||||
const dir = await createTmpCopy();
|
||||
const fakeHome = join(tmpdir(), `ca-verify-home-${Date.now()}`);
|
||||
mkdirSync(join(fakeHome, '.claude'), { recursive: true });
|
||||
// A user-scope CLAUDE.md long enough to trip the CML line-count finding.
|
||||
await writeFile(
|
||||
join(fakeHome, '.claude', 'CLAUDE.md'),
|
||||
Array.from({ length: 260 }, (_, i) => `Line ${i + 1}`).join('\n') + '\n',
|
||||
);
|
||||
|
||||
const originalHome = process.env.HOME;
|
||||
process.env.HOME = fakeHome;
|
||||
try {
|
||||
resetCounter();
|
||||
const envelope = await runAllScanners(dir, { includeGlobal: true });
|
||||
const globalOnly = envelope.scanners
|
||||
.flatMap((s) => s.findings)
|
||||
.filter((f) => f.file && !f.file.startsWith(dir));
|
||||
assert.ok(globalOnly.length > 0, 'Fixture must produce at least one global-scope finding');
|
||||
|
||||
const victim = globalOnly[0];
|
||||
// Nothing was changed on disk — claiming it was applied must NOT verify it.
|
||||
const res = await verifyFixes(
|
||||
envelope,
|
||||
[{ findingId: victim.id, file: victim.file, status: 'applied' }],
|
||||
{ includeGlobal: true },
|
||||
);
|
||||
assert.ok(
|
||||
!res.verified.includes(victim.id),
|
||||
'An untouched global-scope finding must not be reported as verified',
|
||||
);
|
||||
} finally {
|
||||
if (originalHome === undefined) delete process.env.HOME; else process.env.HOME = originalHome;
|
||||
await rm(fakeHome, { recursive: true, force: true });
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue