open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

fix(fix): validate the arguments, back up renames, and verify the scope it fixed

Browse source 
Dogfooding `/config-audit fix` against a throwaway repo copy. All eight
predictions registered in the fasit before the run were confirmed, and three
further defects surfaced that were not predicted.

- M-BUG-21, third arm: the argument loop ended in `!arg.startsWith('-') =>
  targetPath`, so an unknown flag was dropped and its value became the target.
  In `fix` that is the WRITE target under `--apply`. Unknown options and a
  value-less `--output-file` now exit 3.
- `--dry-run` was documented in the command's argument-hint and never
  implemented; `--output-file` did not exist, so `commands/fix.md` told the
  agent to Read a file nothing produced. Both now exist.
- M-BUG-31: `file-rename` was excluded from the backup set, so a renamed rule
  file had no backup entry while the command promised one and returned a
  backupId that could not restore it.
- M-BUG-32: `verifyFixes` hardcoded `includeGlobal: false`, so after a
  `--global` run every untouched user-scope finding was reported as verified.
  Reproduced against an unmodified ~/.claude/CLAUDE.md.
- M-BUG-29: a rename was applied before other fixes on the same file, which
  then failed with ENOENT while the run still exited 0. Renames sort last.
- M-BUG-30: `severityOrder[s] || 4` maps critical (0) to 4, so critical fixes
  sorted last. The old test used the same falsy fallback and agreed with the
  bug. Now `?? 4`.
- A failed fix exits 2 instead of 0, matching the other scanners' convention.

Frozen tests/snapshots/v5.0.0/ untouched; --json/--raw stdout byte-identical.
Suite 1420/0 (+10).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJ3MCDCnyw7wZSPnUXVhYS
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 18:41:27 +02:00
commit 05f1e954d0
7 changed files with 334 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

27
scanners/fix-engine.mjs
View file

@ -56,9 +56,21 @@ export function planFixes(envelope) {
}
}
// Sort fixes by severity weight (critical first)
// Sort fixes by severity weight (critical first), but a file-rename always
// sorts after every other fix. A rename moves the file out from under any
// later fix that still addresses the old path: a rule file with both
// `globs:` and a non-.md extension had the rename applied first, and the
// frontmatter fix then failed with ENOENT while the run still exited 0.
// `?? 4`, not `|| 4`: critical weighs 0, and `0 || 4` evaluates to 4 — so
// critical fixes sorted LAST, the exact opposite of this function's contract
// (M-BUG-30). The old test used the same falsy fallback and agreed with the bug.
const severityOrder = { critical: 0, high: 1, medium: 2, low: 3, info: 4 };
fixes.sort((a, b) => (severityOrder[a.severity] || 4) - (severityOrder[b.severity] || 4));
fixes.sort((a, b) => {
const aRename = a.type === FIX_TYPES.FILE_RENAME ? 1 : 0;
const bRename = b.type === FIX_TYPES.FILE_RENAME ? 1 : 0;
if (aRename !== bRename) return aRename - bRename;
return (severityOrder[a.severity] ?? 4) - (severityOrder[b.severity] ?? 4);
});
return { fixes, skipped, manual };
}
@ -600,16 +612,21 @@ function extractEventFromDescription(description) {
* Verify fixes by re-running affected scanners.
* @param {object} originalEnvelope - Original scanner envelope
* @param {object[]} appliedResults - Results from applyFixes()
* @param {object} [opts]
* @param {boolean} [opts.includeGlobal=false] - Must match the scope the fix run scanned
* @returns {Promise<{ verified: string[], regressions: string[], newFindings: object[] }>}
*/
export async function verifyFixes(originalEnvelope, appliedResults) {
export async function verifyFixes(originalEnvelope, appliedResults, opts = {}) {
const targetPath = originalEnvelope.meta.target;
const verified = [];
const regressions = [];
const newFindings = [];
// Re-scan the target
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: false });
// Re-scan the target in the SAME scope the fix run used. This was hardcoded
// to includeGlobal:false: after a --global run, every global-scope finding
// was absent from the re-scan and therefore counted as verified — a clean
// "fixed" report for files nothing had touched.
const newEnvelope = await runAllScanners(targetPath, { includeGlobal: opts.includeGlobal === true });
// Build set of original finding IDs that were fixed
const fixedIds = new Set(