open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

test(snapshots): make byte/snapshot tests hermetic + re-seed baseline

Browse source 
The COL collision-scanner and the CLAUDE.md cascade resolve ~/.claude from
process.env.HOME (active-config-reader). Snapshot/byte CLIs were spawned with
the developer's real HOME, so they picked up installed plugins/skills and the
user CLAUDE.md — making the v5.0.0 + default-output snapshots machine- and
time-dependent. They were seeded 2026-05-01 with COL=1 (a real ~/.claude skill
collision) and drifted to COL=0 after the polyrepo split: 26 pre-existing
failures unrelated to Batch 1.

Fix (test-only, no production change):
- tests/helpers/hermetic-home.mjs — empty temp HOME, mirroring the pattern
  collision.test.mjs already uses for the COL unit test.
- 7 harnesses spawn CLIs (or call lint()) under the hermetic HOME, so output
  depends only on committed fixtures. Determinism verified across runs.
- Re-seeded all snapshots under hermetic HOME via SEED_SNAPSHOT/UPDATE_SNAPSHOT
  (added a SEED guard to the frozen v5.0.0 byte tests). Snapshots now reflect
  the fixture alone (COL=0, fixture-only activeConfig counts).
- Also re-seeded the unused env-aware snapshots (manifest/whats-active/
  plugin-health), which had baked dozens of real ~/.claude skill/plugin names
  into the committed repo — privacy cleanup.

Full suite: 812/812 green, stable across 3 runs.
This commit is contained in:
Kjell Tore Guttormsen 2026-06-18 12:26:00 +02:00
commit 8216fb4175
20 changed files with 326 additions and 3386 deletions
Download patch file Download diff file Expand all files Collapse all files

40
tests/helpers/hermetic-home.mjs Normal file
View file

@ -0,0 +1,40 @@
/**
* Hermetic HOME for snapshot / byte-stability tests.
*
* The COL collision-scanner and the CLAUDE.md cascade resolve ~/.claude from
* process.env.HOME (see scanners/lib/active-config-reader.mjs — enumeratePlugins,
* enumerateSkills, walkClaudeMdCascade). A CLI spawned with the developer's real
* HOME picks up their installed plugins/skills and user CLAUDE.md, which makes
* byte-snapshots machine- and time-dependent: the snapshots seeded on one machine
* drift the moment another machine (or the same machine, later) has a different
* ~/.claude. Pointing HOME at an empty temp dir isolates every snapshot/byte test
* from user state, exactly as tests/scanners/collision.test.mjs already does for
* the COL unit test.
*
* Writes any CLI may make under HOME (sessions, baselines) land in the temp dir,
* never in the repo.
*/
import { mkdtempSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
export const HERMETIC_HOME = mkdtempSync(join(tmpdir(), 'config-audit-hermetic-home-'));
/** Subprocess env with HOME/USERPROFILE redirected to the empty hermetic dir. */
export function hermeticEnv(extra = {}) {
return { ...process.env, HOME: HERMETIC_HOME, USERPROFILE: HERMETIC_HOME, ...extra };
}
/** Run an in-process callback with process.env.HOME pointed at the hermetic dir. */
export async function withHermeticHome(fn) {
const originalHome = process.env.HOME;
const originalProfile = process.env.USERPROFILE;
process.env.HOME = HERMETIC_HOME;
process.env.USERPROFILE = HERMETIC_HOME;
try {
return await fn();
} finally {
if (originalHome === undefined) delete process.env.HOME; else process.env.HOME = originalHome;
if (originalProfile === undefined) delete process.env.USERPROFILE; else process.env.USERPROFILE = originalProfile;
}
}