test(snapshots): make byte/snapshot tests hermetic + re-seed baseline
The COL collision-scanner and the CLAUDE.md cascade resolve ~/.claude from process.env.HOME (active-config-reader). Snapshot/byte CLIs were spawned with the developer's real HOME, so they picked up installed plugins/skills and the user CLAUDE.md — making the v5.0.0 + default-output snapshots machine- and time-dependent. They were seeded 2026-05-01 with COL=1 (a real ~/.claude skill collision) and drifted to COL=0 after the polyrepo split: 26 pre-existing failures unrelated to Batch 1. Fix (test-only, no production change): - tests/helpers/hermetic-home.mjs — empty temp HOME, mirroring the pattern collision.test.mjs already uses for the COL unit test. - 7 harnesses spawn CLIs (or call lint()) under the hermetic HOME, so output depends only on committed fixtures. Determinism verified across runs. - Re-seeded all snapshots under hermetic HOME via SEED_SNAPSHOT/UPDATE_SNAPSHOT (added a SEED guard to the frozen v5.0.0 byte tests). Snapshots now reflect the fixture alone (COL=0, fixture-only activeConfig counts). - Also re-seeded the unused env-aware snapshots (manifest/whats-active/ plugin-health), which had baked dozens of real ~/.claude skill/plugin names into the committed repo — privacy cleanup. Full suite: 812/812 green, stable across 3 runs.
This commit is contained in:
parent
4b94da0f11
commit
8216fb4175
20 changed files with 326 additions and 3386 deletions
|
|
@ -2,6 +2,19 @@
|
|||
"newFindings": [],
|
||||
"resolvedFindings": [],
|
||||
"unchangedFindings": [
|
||||
{
|
||||
"id": "CA-CML-001",
|
||||
"scanner": "CML",
|
||||
"severity": "low",
|
||||
"title": "Missing recommended sections",
|
||||
"description": "CLAUDE.md is missing: Project overview, Architecture",
|
||||
"file": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/CLAUDE.md",
|
||||
"line": null,
|
||||
"evidence": "Present sections: Marketplace Medium, Plugins, Commands, Conventions",
|
||||
"category": null,
|
||||
"recommendation": "Add sections for: Project overview, Architecture",
|
||||
"autoFixable": false
|
||||
},
|
||||
{
|
||||
"id": "CA-GAP-001",
|
||||
"scanner": "GAP",
|
||||
|
|
@ -242,39 +255,12 @@
|
|||
"severity": "low",
|
||||
"title": "Tool listed in both permissions.deny and permissions.allow",
|
||||
"description": ".claude/settings.json contains 1 tool present in both deny and allow lists. The deny list wins — the allow entries are dead config but still load on every turn and may confuse future readers about intent.",
|
||||
"file": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/config-audit/tests/fixtures/marketplace-medium/.claude/settings.json",
|
||||
"file": "/Users/ktg/repos/ktg-plugin-marketplace/config-audit/tests/fixtures/marketplace-medium/.claude/settings.json",
|
||||
"line": null,
|
||||
"evidence": "Read: allow=\"Read(src/**)\" + deny=\"Read(./.env)\"",
|
||||
"category": "permissions-hygiene",
|
||||
"recommendation": "Remove the redundant allow entries. If you actually want this tool enabled, remove it from the deny list instead. Settings should express intent clearly.",
|
||||
"autoFixable": false
|
||||
},
|
||||
{
|
||||
"id": "CA-COL-001",
|
||||
"scanner": "COL",
|
||||
"severity": "low",
|
||||
"title": "Skill name \"okr-offentlig-sektor\" used by multiple plugins",
|
||||
"description": "2 plugins (okr, okr) expose a skill named \"okr-offentlig-sektor\". Even when invocation is namespaced via /plugin:skill, shared names create ambiguity in error messages, search results, and the plugin-skills enumeration.",
|
||||
"file": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/okr/skills/okr-offentlig-sektor/SKILL.md",
|
||||
"line": null,
|
||||
"evidence": "name=\"okr-offentlig-sektor\"; plugins=okr,okr",
|
||||
"category": "plugin-hygiene",
|
||||
"recommendation": "Coordinate naming across plugins, or rename one to clarify intent. The shared name forces every reader to disambiguate by source.",
|
||||
"autoFixable": false,
|
||||
"details": {
|
||||
"namespaces": [
|
||||
{
|
||||
"source": "plugin:okr",
|
||||
"name": "okr-offentlig-sektor",
|
||||
"path": "/Users/ktg/.claude/plugins/marketplaces/ktg-plugin-marketplace/plugins/okr/skills/okr-offentlig-sektor/SKILL.md"
|
||||
},
|
||||
{
|
||||
"source": "plugin:okr",
|
||||
"name": "okr-offentlig-sektor",
|
||||
"path": "/Users/ktg/.claude/plugins/marketplaces/ktg-privat/plugins/okr/skills/okr-offentlig-sektor/SKILL.md"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"movedFindings": [],
|
||||
|
|
@ -293,11 +279,11 @@
|
|||
{
|
||||
"name": "CLAUDE.md",
|
||||
"before": {
|
||||
"score": 100,
|
||||
"score": 90,
|
||||
"grade": "A"
|
||||
},
|
||||
"after": {
|
||||
"score": 100,
|
||||
"score": 90,
|
||||
"grade": "A"
|
||||
},
|
||||
"delta": 0
|
||||
|
|
@ -401,11 +387,11 @@
|
|||
{
|
||||
"name": "Plugin Hygiene",
|
||||
"before": {
|
||||
"score": 90,
|
||||
"score": 100,
|
||||
"grade": "A"
|
||||
},
|
||||
"after": {
|
||||
"score": 90,
|
||||
"score": 100,
|
||||
"grade": "A"
|
||||
},
|
||||
"delta": 0
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue