open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

test(snapshots): make byte/snapshot tests hermetic + re-seed baseline

Browse source 
The COL collision-scanner and the CLAUDE.md cascade resolve ~/.claude from
process.env.HOME (active-config-reader). Snapshot/byte CLIs were spawned with
the developer's real HOME, so they picked up installed plugins/skills and the
user CLAUDE.md — making the v5.0.0 + default-output snapshots machine- and
time-dependent. They were seeded 2026-05-01 with COL=1 (a real ~/.claude skill
collision) and drifted to COL=0 after the polyrepo split: 26 pre-existing
failures unrelated to Batch 1.

Fix (test-only, no production change):
- tests/helpers/hermetic-home.mjs — empty temp HOME, mirroring the pattern
  collision.test.mjs already uses for the COL unit test.
- 7 harnesses spawn CLIs (or call lint()) under the hermetic HOME, so output
  depends only on committed fixtures. Determinism verified across runs.
- Re-seeded all snapshots under hermetic HOME via SEED_SNAPSHOT/UPDATE_SNAPSHOT
  (added a SEED guard to the frozen v5.0.0 byte tests). Snapshots now reflect
  the fixture alone (COL=0, fixture-only activeConfig counts).
- Also re-seeded the unused env-aware snapshots (manifest/whats-active/
  plugin-health), which had baked dozens of real ~/.claude skill/plugin names
  into the committed repo — privacy cleanup.

Full suite: 812/812 green, stable across 3 runs.
This commit is contained in:
Kjell Tore Guttormsen 2026-06-18 12:26:00 +02:00
commit 8216fb4175
20 changed files with 326 additions and 3386 deletions
Download patch file Download diff file Expand all files Collapse all files

64
tests/snapshots/default-output/token-hotspots.json
View file

@ -5,7 +5,7 @@
"status": "ok",
"files_scanned": 2,
"duration_ms": 0,
"total_estimated_tokens": 1809,
"total_estimated_tokens": 809,
"hotspots": [
{
"source": "mcp:memory (.mcp.json)",
@ -15,61 +15,45 @@
"Review whether this source needs to load on every turn."
]
},
{
"source": "mcp:sadhguru-wisdom (plugin:sadhguru-wisdom)",
"estimated_tokens": 500,
"rank": 2,
"recommendations": [
"Review whether this source needs to load on every turn."
]
},
{
"source": "mcp:vegnorm-rag (plugin:vegnormalene)",
"estimated_tokens": 500,
"rank": 3,
"recommendations": [
"Review whether this source needs to load on every turn."
]
},
{
"source": "CLAUDE.md",
"estimated_tokens": 116,
"rank": 4,
"rank": 2,
"recommendations": [
"Move volatile top-of-file content to the bottom or extract to an @import-ed file.",
"Split overlong CLAUDE.md into focused @imports (≤200 lines each)."
],
"path": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/config-audit/tests/fixtures/marketplace-medium/CLAUDE.md"
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/CLAUDE.md"
},
{
"source": "hooks/hooks.json",
"estimated_tokens": 81,
"rank": 3,
"recommendations": [
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
"Move rarely-used permissions to a project-local override."
],
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/hooks/hooks.json"
},
{
"source": ".claude/settings.json",
"estimated_tokens": 59,
"rank": 4,
"recommendations": [
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
"Move rarely-used permissions to a project-local override."
],
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/.claude/settings.json"
},
{
"source": ".mcp.json",
"estimated_tokens": 53,
"rank": 5,
"recommendations": [
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
"Move rarely-used permissions to a project-local override."
],
"path": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/config-audit/tests/fixtures/marketplace-medium/hooks/hooks.json"
},
{
"source": ".claude/settings.json",
"estimated_tokens": 59,
"rank": 6,
"recommendations": [
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
"Move rarely-used permissions to a project-local override."
],
"path": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/config-audit/tests/fixtures/marketplace-medium/.claude/settings.json"
},
{
"source": ".mcp.json",
"estimated_tokens": 53,
"rank": 7,
"recommendations": [
"Deduplicate overlapping entries — each duplicate inflates the per-turn schema payload.",
"Move rarely-used permissions to a project-local override."
],
"path": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/config-audit/tests/fixtures/marketplace-medium/.mcp.json"
"path": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/.mcp.json"
}
],
"findings": [