test(config-audit): add Opus 4.7 pattern fixtures (cache, redundant, imports, sonnet-era)
This commit is contained in:
parent
350cebc39c
commit
94ce70186c
11 changed files with 255 additions and 0 deletions
7
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/.claude/settings.json
vendored
Normal file
7
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
{
|
||||||
|
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||||
|
"permissions": {
|
||||||
|
"allow": ["Bash(npm test)"],
|
||||||
|
"deny": []
|
||||||
|
}
|
||||||
|
}
|
||||||
47
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/CLAUDE.md
vendored
Normal file
47
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,47 @@
|
||||||
|
# Cache-Breaking Volatile Top
|
||||||
|
|
||||||
|
> Last Run: {timestamp} — volatile placeholder that changes on every invocation
|
||||||
|
> Session: {uuid} — another volatile field
|
||||||
|
> Now: {date}
|
||||||
|
|
||||||
|
## Recent Activity
|
||||||
|
|
||||||
|
- 2026-04-19T12:00:00Z — User A edited file X
|
||||||
|
- 2026-04-19T11:45:00Z — User B pushed commit Y
|
||||||
|
- 2026-04-19T11:30:00Z — CI run Z completed
|
||||||
|
- 2026-04-19T11:15:00Z — Review comment added
|
||||||
|
- 2026-04-19T11:00:00Z — Deployment triggered
|
||||||
|
- 2026-04-19T10:45:00Z — Log rotation ran
|
||||||
|
- 2026-04-19T10:30:00Z — Backup verified
|
||||||
|
- 2026-04-19T10:15:00Z — Cache cleared
|
||||||
|
- 2026-04-19T10:00:00Z — Session started
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
The status widget above renews on every turn, pushing the stable-prefix
|
||||||
|
content further down the file. Under Opus 4.7 prompt caching, any change
|
||||||
|
within the first block invalidates the cache-prefix, forcing a full
|
||||||
|
recomputation each turn and inflating token cost per session.
|
||||||
|
|
||||||
|
## Stable Content (cache target)
|
||||||
|
|
||||||
|
Below this line is content that rarely changes — the project overview,
|
||||||
|
conventions, and shared rules. But because the volatile header sits ABOVE
|
||||||
|
this stable section, it cannot benefit from caching.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
A fixture designed to trip the Opus 4.7 TOK scanner's cache-breaking
|
||||||
|
detector (CA-TOK-001). The first 30 lines contain volatile-looking
|
||||||
|
patterns (timestamps, session ids, running activity logs) that would
|
||||||
|
break prompt-cache reuse on every turn.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
- `npm run build`
|
||||||
|
- `npm test`
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- Conventional Commits
|
||||||
|
- TypeScript strict
|
||||||
7
plugins/config-audit/tests/fixtures/opus-47/deep-imports/.claude/settings.json
vendored
Normal file
7
plugins/config-audit/tests/fixtures/opus-47/deep-imports/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
{
|
||||||
|
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||||
|
"permissions": {
|
||||||
|
"allow": [],
|
||||||
|
"deny": []
|
||||||
|
}
|
||||||
|
}
|
||||||
10
plugins/config-audit/tests/fixtures/opus-47/deep-imports/CLAUDE.md
vendored
Normal file
10
plugins/config-audit/tests/fixtures/opus-47/deep-imports/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,10 @@
|
||||||
|
# Deep Import Chain
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Fixture designed to trip the TOK scanner's import-depth detector
|
||||||
|
(CA-TOK-003). The top-level CLAUDE.md imports layer1, layer1 imports
|
||||||
|
layer2, and layer2 imports layer3 — a 3-deep @import chain where each
|
||||||
|
hop fragments the cache prefix.
|
||||||
|
|
||||||
|
@layer1.md
|
||||||
46
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer1.md
vendored
Normal file
46
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer1.md
vendored
Normal file
|
|
@ -0,0 +1,46 @@
|
||||||
|
# Layer 1
|
||||||
|
|
||||||
|
First layer of the deep import chain. Each layer contributes substantive
|
||||||
|
content so the chain is not trivially dismissable as "all short stubs".
|
||||||
|
|
||||||
|
## Section A
|
||||||
|
|
||||||
|
Coding conventions for the fictitious project that exists solely to
|
||||||
|
exercise the TOK scanner's import-depth detection logic.
|
||||||
|
|
||||||
|
- Prefer async/await over raw Promises.
|
||||||
|
- Annotate return types even when TypeScript can infer them.
|
||||||
|
- Keep functions under 40 lines where practical.
|
||||||
|
- Use `readonly` generously on types.
|
||||||
|
- Dependency-inject side effects at module boundaries.
|
||||||
|
- Validate inputs at system boundaries only.
|
||||||
|
|
||||||
|
## Section B
|
||||||
|
|
||||||
|
Error handling patterns:
|
||||||
|
|
||||||
|
- Subclass `Error` with typed messages.
|
||||||
|
- Never swallow errors silently.
|
||||||
|
- Prefer `Result<T, E>` return types in business logic.
|
||||||
|
- Log at boundaries, never inside pure functions.
|
||||||
|
|
||||||
|
## Section C
|
||||||
|
|
||||||
|
Testing patterns:
|
||||||
|
|
||||||
|
- Test fixtures live under `tests/fixtures/{name}/`.
|
||||||
|
- Each fixture has a single shape it exercises.
|
||||||
|
- Use `describe(...)` + `it(...)` from `node:test`.
|
||||||
|
- Prefer table-driven tests for permutations.
|
||||||
|
- Keep test setup idempotent.
|
||||||
|
|
||||||
|
## Section D
|
||||||
|
|
||||||
|
CI/CD conventions:
|
||||||
|
|
||||||
|
- Conventional Commits for every commit.
|
||||||
|
- PR-free single-branch workflow on Forgejo.
|
||||||
|
- Lint and typecheck on every push.
|
||||||
|
- Release tags follow `{plugin}-v{semver}`.
|
||||||
|
|
||||||
|
@layer2.md
|
||||||
43
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer2.md
vendored
Normal file
43
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer2.md
vendored
Normal file
|
|
@ -0,0 +1,43 @@
|
||||||
|
# Layer 2
|
||||||
|
|
||||||
|
Second layer of the chain. This file is imported by layer1.md and imports
|
||||||
|
layer3.md in turn, forming a 3-deep @import chain that the TOK scanner
|
||||||
|
should flag.
|
||||||
|
|
||||||
|
## Architecture Notes
|
||||||
|
|
||||||
|
The ficticious project uses a layered architecture split across:
|
||||||
|
|
||||||
|
- Application layer (HTTP adapters, CLI adapters)
|
||||||
|
- Domain layer (pure business logic)
|
||||||
|
- Infrastructure layer (databases, caches, external APIs)
|
||||||
|
|
||||||
|
## Dependency Rules
|
||||||
|
|
||||||
|
- Application depends on domain but not infrastructure.
|
||||||
|
- Infrastructure implements ports defined in the domain.
|
||||||
|
- Domain never imports from application or infrastructure.
|
||||||
|
- Cross-cutting concerns (logging, tracing) live as ports.
|
||||||
|
|
||||||
|
## Observability
|
||||||
|
|
||||||
|
- Structured logs with correlation ids.
|
||||||
|
- Metrics scraped from a `/metrics` endpoint.
|
||||||
|
- Trace spans around domain service boundaries.
|
||||||
|
- Health checks separate from metrics endpoints.
|
||||||
|
|
||||||
|
## Data Access
|
||||||
|
|
||||||
|
- Repositories return domain objects, never ORM entities.
|
||||||
|
- Database migrations numbered sequentially.
|
||||||
|
- Idempotent migrations where feasible.
|
||||||
|
- Readonly replicas for analytical queries.
|
||||||
|
|
||||||
|
## Caching
|
||||||
|
|
||||||
|
- Read-through cache for hot entities.
|
||||||
|
- TTL chosen per entity class.
|
||||||
|
- Invalidate on write, not on read miss.
|
||||||
|
- Measure hit-rate per cache bucket.
|
||||||
|
|
||||||
|
@layer3.md
|
||||||
45
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer3.md
vendored
Normal file
45
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer3.md
vendored
Normal file
|
|
@ -0,0 +1,45 @@
|
||||||
|
# Layer 3
|
||||||
|
|
||||||
|
Third layer of the chain — terminal leaf. Imported by layer2.md, imports
|
||||||
|
nothing further. Exists solely to make the chain 3-deep, tripping the
|
||||||
|
TOK scanner's import-depth detector.
|
||||||
|
|
||||||
|
## Release Checklist
|
||||||
|
|
||||||
|
- Version bumped in plugin manifest and package.json.
|
||||||
|
- CHANGELOG.md has an entry for the new version.
|
||||||
|
- README badges reflect the new version.
|
||||||
|
- All tests green on a clean working tree.
|
||||||
|
- Tag pushed to Forgejo alongside the main branch.
|
||||||
|
|
||||||
|
## Supported Platforms
|
||||||
|
|
||||||
|
- macOS Intel (primary development)
|
||||||
|
- macOS Apple Silicon (untested, considered compatible)
|
||||||
|
- Linux x86_64 (CI target)
|
||||||
|
- Windows (partial — managed-settings path missing)
|
||||||
|
|
||||||
|
## Known Gaps
|
||||||
|
|
||||||
|
- Windows managed-settings support deferred to a future release.
|
||||||
|
- Prompt-cache hit-rate measurement requires runtime telemetry.
|
||||||
|
- Token-cost calibration pending authoritative research.
|
||||||
|
|
||||||
|
## Hook Safety
|
||||||
|
|
||||||
|
- All hooks run in bounded time (timeout declared).
|
||||||
|
- Non-zero exit blocks the operation.
|
||||||
|
- Hook scripts never write outside the repository root.
|
||||||
|
- Hook scripts never modify `.git/hooks/` or shell configs.
|
||||||
|
|
||||||
|
## Rule Prioritisation
|
||||||
|
|
||||||
|
- Project rules override user rules.
|
||||||
|
- Rules with narrower paths win over broader ones.
|
||||||
|
- Conflicts trigger a CNF finding at high severity.
|
||||||
|
- Deprecated rule fields are rewritten on load.
|
||||||
|
|
||||||
|
## Closing Note
|
||||||
|
|
||||||
|
This terminal layer rounds out the chain to ensure the TOK scanner sees a
|
||||||
|
meaningful volume of imported content at each depth, not just a stub.
|
||||||
16
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/.claude/settings.json
vendored
Normal file
16
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,16 @@
|
||||||
|
{
|
||||||
|
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||||
|
"permissions": {
|
||||||
|
"allow": [
|
||||||
|
"Read",
|
||||||
|
"Read(**)",
|
||||||
|
"Read(src/**)",
|
||||||
|
"Bash",
|
||||||
|
"Bash(*)",
|
||||||
|
"Bash(npm *)",
|
||||||
|
"Bash(npm run *)",
|
||||||
|
"Bash(npm test)"
|
||||||
|
],
|
||||||
|
"deny": []
|
||||||
|
}
|
||||||
|
}
|
||||||
8
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/CLAUDE.md
vendored
Normal file
8
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
# Redundant Tool Declarations
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Fixture designed to trip the TOK scanner's redundant-permissions detector
|
||||||
|
(CA-TOK-002). The `.claude/settings.json` contains overlapping tool
|
||||||
|
patterns that inflate the tool-schema payload sent to Opus 4.7 on every
|
||||||
|
turn without adding expressive power.
|
||||||
7
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/.claude/settings.json
vendored
Normal file
7
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
||||||
|
{
|
||||||
|
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||||
|
"permissions": {
|
||||||
|
"allow": ["Bash(npm run *)"],
|
||||||
|
"deny": ["Read(./.env)"]
|
||||||
|
}
|
||||||
|
}
|
||||||
19
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/CLAUDE.md
vendored
Normal file
19
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,19 @@
|
||||||
|
# Sonnet-Era Clean Config
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Negative-control fixture for the TOK scanner. Contains no cache-breaking
|
||||||
|
volatility, no redundant permissions, no deep import chains. Represents
|
||||||
|
a clean, Sonnet-era configuration that pre-dates Opus 4.7 features — the
|
||||||
|
TOK scanner should emit zero medium/high severity findings here (info
|
||||||
|
severity is acceptable for pattern D).
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
- `npm test`
|
||||||
|
- `npm run build`
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- TypeScript strict
|
||||||
|
- Conventional Commits
|
||||||
Loading…
Add table
Add a link
Reference in a new issue