test(config-audit): add Opus 4.7 pattern fixtures (cache, redundant, imports, sonnet-era)
This commit is contained in:
parent
350cebc39c
commit
94ce70186c
11 changed files with 255 additions and 0 deletions
7
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/.claude/settings.json
vendored
Normal file
7
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"allow": ["Bash(npm test)"],
|
||||
"deny": []
|
||||
}
|
||||
}
|
||||
47
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/CLAUDE.md
vendored
Normal file
47
plugins/config-audit/tests/fixtures/opus-47/cache-breaking/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
# Cache-Breaking Volatile Top
|
||||
|
||||
> Last Run: {timestamp} — volatile placeholder that changes on every invocation
|
||||
> Session: {uuid} — another volatile field
|
||||
> Now: {date}
|
||||
|
||||
## Recent Activity
|
||||
|
||||
- 2026-04-19T12:00:00Z — User A edited file X
|
||||
- 2026-04-19T11:45:00Z — User B pushed commit Y
|
||||
- 2026-04-19T11:30:00Z — CI run Z completed
|
||||
- 2026-04-19T11:15:00Z — Review comment added
|
||||
- 2026-04-19T11:00:00Z — Deployment triggered
|
||||
- 2026-04-19T10:45:00Z — Log rotation ran
|
||||
- 2026-04-19T10:30:00Z — Backup verified
|
||||
- 2026-04-19T10:15:00Z — Cache cleared
|
||||
- 2026-04-19T10:00:00Z — Session started
|
||||
|
||||
## Current State
|
||||
|
||||
The status widget above renews on every turn, pushing the stable-prefix
|
||||
content further down the file. Under Opus 4.7 prompt caching, any change
|
||||
within the first block invalidates the cache-prefix, forcing a full
|
||||
recomputation each turn and inflating token cost per session.
|
||||
|
||||
## Stable Content (cache target)
|
||||
|
||||
Below this line is content that rarely changes — the project overview,
|
||||
conventions, and shared rules. But because the volatile header sits ABOVE
|
||||
this stable section, it cannot benefit from caching.
|
||||
|
||||
## Project Overview
|
||||
|
||||
A fixture designed to trip the Opus 4.7 TOK scanner's cache-breaking
|
||||
detector (CA-TOK-001). The first 30 lines contain volatile-looking
|
||||
patterns (timestamps, session ids, running activity logs) that would
|
||||
break prompt-cache reuse on every turn.
|
||||
|
||||
## Commands
|
||||
|
||||
- `npm run build`
|
||||
- `npm test`
|
||||
|
||||
## Conventions
|
||||
|
||||
- Conventional Commits
|
||||
- TypeScript strict
|
||||
7
plugins/config-audit/tests/fixtures/opus-47/deep-imports/.claude/settings.json
vendored
Normal file
7
plugins/config-audit/tests/fixtures/opus-47/deep-imports/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"allow": [],
|
||||
"deny": []
|
||||
}
|
||||
}
|
||||
10
plugins/config-audit/tests/fixtures/opus-47/deep-imports/CLAUDE.md
vendored
Normal file
10
plugins/config-audit/tests/fixtures/opus-47/deep-imports/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
# Deep Import Chain
|
||||
|
||||
## Overview
|
||||
|
||||
Fixture designed to trip the TOK scanner's import-depth detector
|
||||
(CA-TOK-003). The top-level CLAUDE.md imports layer1, layer1 imports
|
||||
layer2, and layer2 imports layer3 — a 3-deep @import chain where each
|
||||
hop fragments the cache prefix.
|
||||
|
||||
@layer1.md
|
||||
46
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer1.md
vendored
Normal file
46
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer1.md
vendored
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
# Layer 1
|
||||
|
||||
First layer of the deep import chain. Each layer contributes substantive
|
||||
content so the chain is not trivially dismissable as "all short stubs".
|
||||
|
||||
## Section A
|
||||
|
||||
Coding conventions for the fictitious project that exists solely to
|
||||
exercise the TOK scanner's import-depth detection logic.
|
||||
|
||||
- Prefer async/await over raw Promises.
|
||||
- Annotate return types even when TypeScript can infer them.
|
||||
- Keep functions under 40 lines where practical.
|
||||
- Use `readonly` generously on types.
|
||||
- Dependency-inject side effects at module boundaries.
|
||||
- Validate inputs at system boundaries only.
|
||||
|
||||
## Section B
|
||||
|
||||
Error handling patterns:
|
||||
|
||||
- Subclass `Error` with typed messages.
|
||||
- Never swallow errors silently.
|
||||
- Prefer `Result<T, E>` return types in business logic.
|
||||
- Log at boundaries, never inside pure functions.
|
||||
|
||||
## Section C
|
||||
|
||||
Testing patterns:
|
||||
|
||||
- Test fixtures live under `tests/fixtures/{name}/`.
|
||||
- Each fixture has a single shape it exercises.
|
||||
- Use `describe(...)` + `it(...)` from `node:test`.
|
||||
- Prefer table-driven tests for permutations.
|
||||
- Keep test setup idempotent.
|
||||
|
||||
## Section D
|
||||
|
||||
CI/CD conventions:
|
||||
|
||||
- Conventional Commits for every commit.
|
||||
- PR-free single-branch workflow on Forgejo.
|
||||
- Lint and typecheck on every push.
|
||||
- Release tags follow `{plugin}-v{semver}`.
|
||||
|
||||
@layer2.md
|
||||
43
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer2.md
vendored
Normal file
43
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer2.md
vendored
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
# Layer 2
|
||||
|
||||
Second layer of the chain. This file is imported by layer1.md and imports
|
||||
layer3.md in turn, forming a 3-deep @import chain that the TOK scanner
|
||||
should flag.
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
The ficticious project uses a layered architecture split across:
|
||||
|
||||
- Application layer (HTTP adapters, CLI adapters)
|
||||
- Domain layer (pure business logic)
|
||||
- Infrastructure layer (databases, caches, external APIs)
|
||||
|
||||
## Dependency Rules
|
||||
|
||||
- Application depends on domain but not infrastructure.
|
||||
- Infrastructure implements ports defined in the domain.
|
||||
- Domain never imports from application or infrastructure.
|
||||
- Cross-cutting concerns (logging, tracing) live as ports.
|
||||
|
||||
## Observability
|
||||
|
||||
- Structured logs with correlation ids.
|
||||
- Metrics scraped from a `/metrics` endpoint.
|
||||
- Trace spans around domain service boundaries.
|
||||
- Health checks separate from metrics endpoints.
|
||||
|
||||
## Data Access
|
||||
|
||||
- Repositories return domain objects, never ORM entities.
|
||||
- Database migrations numbered sequentially.
|
||||
- Idempotent migrations where feasible.
|
||||
- Readonly replicas for analytical queries.
|
||||
|
||||
## Caching
|
||||
|
||||
- Read-through cache for hot entities.
|
||||
- TTL chosen per entity class.
|
||||
- Invalidate on write, not on read miss.
|
||||
- Measure hit-rate per cache bucket.
|
||||
|
||||
@layer3.md
|
||||
45
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer3.md
vendored
Normal file
45
plugins/config-audit/tests/fixtures/opus-47/deep-imports/layer3.md
vendored
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
# Layer 3
|
||||
|
||||
Third layer of the chain — terminal leaf. Imported by layer2.md, imports
|
||||
nothing further. Exists solely to make the chain 3-deep, tripping the
|
||||
TOK scanner's import-depth detector.
|
||||
|
||||
## Release Checklist
|
||||
|
||||
- Version bumped in plugin manifest and package.json.
|
||||
- CHANGELOG.md has an entry for the new version.
|
||||
- README badges reflect the new version.
|
||||
- All tests green on a clean working tree.
|
||||
- Tag pushed to Forgejo alongside the main branch.
|
||||
|
||||
## Supported Platforms
|
||||
|
||||
- macOS Intel (primary development)
|
||||
- macOS Apple Silicon (untested, considered compatible)
|
||||
- Linux x86_64 (CI target)
|
||||
- Windows (partial — managed-settings path missing)
|
||||
|
||||
## Known Gaps
|
||||
|
||||
- Windows managed-settings support deferred to a future release.
|
||||
- Prompt-cache hit-rate measurement requires runtime telemetry.
|
||||
- Token-cost calibration pending authoritative research.
|
||||
|
||||
## Hook Safety
|
||||
|
||||
- All hooks run in bounded time (timeout declared).
|
||||
- Non-zero exit blocks the operation.
|
||||
- Hook scripts never write outside the repository root.
|
||||
- Hook scripts never modify `.git/hooks/` or shell configs.
|
||||
|
||||
## Rule Prioritisation
|
||||
|
||||
- Project rules override user rules.
|
||||
- Rules with narrower paths win over broader ones.
|
||||
- Conflicts trigger a CNF finding at high severity.
|
||||
- Deprecated rule fields are rewritten on load.
|
||||
|
||||
## Closing Note
|
||||
|
||||
This terminal layer rounds out the chain to ensure the TOK scanner sees a
|
||||
meaningful volume of imported content at each depth, not just a stub.
|
||||
16
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/.claude/settings.json
vendored
Normal file
16
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Read",
|
||||
"Read(**)",
|
||||
"Read(src/**)",
|
||||
"Bash",
|
||||
"Bash(*)",
|
||||
"Bash(npm *)",
|
||||
"Bash(npm run *)",
|
||||
"Bash(npm test)"
|
||||
],
|
||||
"deny": []
|
||||
}
|
||||
}
|
||||
8
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/CLAUDE.md
vendored
Normal file
8
plugins/config-audit/tests/fixtures/opus-47/redundant-tools/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
# Redundant Tool Declarations
|
||||
|
||||
## Overview
|
||||
|
||||
Fixture designed to trip the TOK scanner's redundant-permissions detector
|
||||
(CA-TOK-002). The `.claude/settings.json` contains overlapping tool
|
||||
patterns that inflate the tool-schema payload sent to Opus 4.7 on every
|
||||
turn without adding expressive power.
|
||||
7
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/.claude/settings.json
vendored
Normal file
7
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/.claude/settings.json
vendored
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"allow": ["Bash(npm run *)"],
|
||||
"deny": ["Read(./.env)"]
|
||||
}
|
||||
}
|
||||
19
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/CLAUDE.md
vendored
Normal file
19
plugins/config-audit/tests/fixtures/opus-47/sonnet-era/CLAUDE.md
vendored
Normal file
|
|
@ -0,0 +1,19 @@
|
|||
# Sonnet-Era Clean Config
|
||||
|
||||
## Overview
|
||||
|
||||
Negative-control fixture for the TOK scanner. Contains no cache-breaking
|
||||
volatility, no redundant permissions, no deep import chains. Represents
|
||||
a clean, Sonnet-era configuration that pre-dates Opus 4.7 features — the
|
||||
TOK scanner should emit zero medium/high severity findings here (info
|
||||
severity is acceptable for pattern D).
|
||||
|
||||
## Commands
|
||||
|
||||
- `npm test`
|
||||
- `npm run build`
|
||||
|
||||
## Conventions
|
||||
|
||||
- TypeScript strict
|
||||
- Conventional Commits
|
||||
Loading…
Add table
Add a link
Reference in a new issue