The SC asks two things of an effort: high run: that it discovered at least
one dimension AND that the dimension list in the output brief is a TRUE
SUPERSET of the interview-derived ones. activationCheck computed
dimensions - dimensions_baseline >= 1 and returned ok on that alone. A
count delta says nothing about membership: a run that dropped two
interview dimensions and appended three discovered ones is +1 and passed
the check while violating the second half outright. Supersetness was
asserted only by Phase 4.5's prose contract that discovery appends -
nothing read it.
The stats record cannot carry the dimension names that would show it
directly: names are free prose, and lib/exporters/field-allowlist.mjs
denies prose by omission (question, project_dir, brief_path are already
excluded for that reason). So the run attests membership with
dimensions_baseline_preserved, a boolean set in a new Phase 4.5 step 5,
and the gate refuses to call activation OK without it. An ABSENT
attestation fails rather than falling back to the old count-only rule -
otherwise legacy rows would keep passing on the defect.
Producer side wired end to end: the record format, the measurement-fields
prose (five fields -> six), the exporter allowlist, the jsonl-schemas
fixture row, and the --activation-check comment in
docs/storm-measurement.md.
Review finding d2786604458207a5a73478cdcb6a54bbdb92141d (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
allowTurn() appends BEFORE the turn runs, so during granted turn N the
ledger holds N records. The hook denied at `used >= budget`, which blocked
every tool call of the FINAL granted turn: the primitive granted B turns
and the harness permitted B-1. Worse, an exhausted run therefore always
terminated through an exit-2 tool denial instead of the graceful "cap
exhausted" exit at commands/trekresearch.md - and the prose says in as
many words that exit 2 is not exit 1, so the model was pushed out through
the one exit it is told NOT to treat as a cap.
The review recommended denying at `used > budget`. Taken alone that fixes
the count and breaks the hook: once the O_EXCL claim (previous commit)
makes a breached ledger impossible, `granted > budget` can no longer fire,
and the case this hook exists for - the loop consults the gate, is denied,
and issues the tool call anyway - would be allowed. A deny branch that
cannot be reached is a dead security claim, which is the same thing S82
removed two of rather than leave standing.
So the denial itself became a record. allowTurn() appends a tombstone
{runId, exhausted: true} when it denies for budget, and the hook denies on
the tombstone. Both properties now hold at once:
granted == budget, no tombstone -> turn B is in flight -> ALLOW
tombstone present -> the gate already said no -> DENY
granted > budget -> breached, any cause -> DENY
A tombstone is not a turn: readLedger reports {granted, exhausted}
separately so it can never consume budget. allowTurn short-circuits on an
existing tombstone, so a hammered gate neither re-walks every slot nor
grows the ledger. The tombstone write is best effort on purpose - the
denial is already the correct answer, so a ledger that cannot take the
record must not turn a denial into a grant.
The parallel-boundary test now asserts GRANTED turns rather than raw
ledger lines, because the denied callers legitimately add tombstones.
Review finding 8eb53458ac3efec778094f9f03b09e1cc1077a09 (MINOR).
Operator decision: tombstone over the literal recommended_action.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
countTurns-then-appendFileSync is read-then-write. N callers that all
observe used == budget-1 all decide to grant, and the bound is exceeded
by N-1. The comment above allowTurn asserted "Append-only: never
read-modify-write" and named the concurrent case - Phase 4.5/5 may spawn
several agents in a single message - as the reason it had to be. The
decision path was exactly what the comment denied, so the concurrency
claim had nothing under it.
Each grant now creates <data root>/trekresearch-loop-claims/<runId>-<slot>.claim
with flag 'wx' (O_CREAT|O_EXCL) before appending. The kernel picks the
winner per slot, slot numbers are bounded by the budget, and each can be
created exactly once - so total grants for a run cannot exceed the budget
however many callers arrive together. The ledger count now only says
where to start looking for a free slot.
Two of the three tests are deterministic and do not race anything: they
assert the invariant directly by pre-creating claims, including the state
a mid-append competitor leaves behind (ledger 7, slots 1-8 claimed, budget
8 -> deny). That matters because the third test - six real concurrent shim
processes at the boundary - passed even BEFORE the fix, since process
startup jitter serialised them. A race test that passes by luck is not
evidence, so it ships as a real-world regression guard next to the two
that are.
Stated rather than left to be discovered: claim files are empty, at most
budget per run, and never cleaned - the same standing as the ledger, which
also grows for the life of the data root. Reusing a runId across runs, or
two runIds colliding after filename sanitisation, both deny a turn, which
is the safe direction for a budget control.
Review finding 3994491ef1fdba6e0e3645b5b713cbdbdeb2b328 (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
An unreadable ledger returned 0 from countTurns in BOTH the primitive and
the hook, so a run whose ledger existed but could not be read (EISDIR,
EACCES, EIO) was handed the full budget again on every call - unbounded.
research-loop-cap.mjs argues against exactly that three lines above the
code that did it, and its missing-DIRECTORY case already failed closed.
The unreadable-FILE case now agrees with it.
Only ENOENT still counts as zero turns spent: that is the legitimate
first-turn state, and the reason this cannot just throw on any read
failure.
The hook no longer carries its own countTurns. It imports the primitive's
exported readLedger(), the same way it already resolves the data root
through resolveDataRoot() - a reader and a writer with private copies of
the counting rule is how a hook ends up enforcing a different bound than
the gate it backs. In scope + cannot count now exits 2 with a message
that says counting failed, not that the budget is spent.
Fail-closed stays scoped to the loop: a test pins that an unreadable
ledger in an OUT-of-scope session still exits 0, because a PreToolUse
hook that over-blocks bricks every session on the box.
Also dropped the existsSync pre-check before the read - readFileSync's
own ENOENT carries the same information without a second syscall that
can disagree with the read that follows it.
Review finding 5e1c6230f48ead38fa77cd8f4b06bfdc2b5b7bbf (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
A fractional TREKRESEARCH_MAX_CONV_TURNS below 1 cleared the `n <= 0`
guard on its raw value and only then floored, so '0.5' and '0.9' became
0 and the budget became 0 x MAX_TOTAL_DIMENSIONS = 0: every turn denied,
the loop silently dead rather than bounded.
README.md:229 and docs/architecture.md:15 both promise that invalid
values fall back to 3. docs/command-modes.md:42 enumerated "empty,
non-numeric, zero, or negative" and happened to sidestep the case; the
enumeration is now exhaustive about it.
Measured before: '0.5' -> 0, '0.9' -> 0, '2.7' -> 2, '' / 'abc' / '-2'
/ '0' -> 3. Measured after: '0.5' -> 3, '0.9' -> 3, '2.7' -> 2, and
'Infinity' -> 3 (it is not a cap either).
A cap of 0 is not a narrower cap, it is an off switch. The tests pin
both directions: the fraction falls back, and allowTurn cannot report a
budget of 0 under it.
Review finding fc516799e6042e246a4b62d81903ac27c2efab84 (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
CLAUDE_PLUGIN_DATA is empty in the Bash tool's process env, and the Phase 5
bash snippet is the cap's only caller. resolveLedgerPath() returned null there
and allowTurn() failed closed, so the budget gate denied turn 1 of every real
run: the loop this delivery exists to bound could never spend a turn, and the
pre-registered measurement could not be run at all.
resolveDataRoot() is now the single root for everything the loop writes --
CLAUDE_PLUGIN_DATA when the harness sets it, ~/.claude/voyage when it does
not. Three consumers resolve through it, which is the point: the cap ledger,
the PreToolUse hook's scope-marker lookup, and the command's bash snippets.
A writer and a reader that resolved the root separately are what made the
enforcement hook allow unconditionally in every real run while CLAUDE.md and
docs/architecture.md called it enforcing.
Same root cause, same commit:
- Marker write and remove now share ONE absolute-path guard and one root; the
write requires a non-empty CLAUDE_CODE_SESSION_ID before composing the path
(unset, the marker was named `.json`, which no lookup matches and no TTL
sweep cleans up).
- The per-turn gates resolve VOYAGE_ROOT with a plugin-cache fallback and
reserve exit 2 for "gate could not run". Interpolating an empty
${CLAUDE_PLUGIN_ROOT} ran `node /lib/...` -> exit 1, which the contract read
as "privacy gate says no" -- an unsatisfiable rewrite loop no query could
clear.
Two now-unreachable deny branches are removed rather than left as dead safety
claims (allowTurn's no_plugin_data_dir; the hook's uncountable-ledger deny).
The fail-closed stance stays where it is still real: a ledger that cannot be
WRITTEN denies the turn.
Verified end-to-end through the real bash snippets and the real hook with both
variables stripped and HOME sandboxed: marker written under the fallback root,
8 turns spent, 9th denied, hook exits 2, and exits 0 again after removal.
Note: the fallback exit-2 branch fires against the installed v5.9.1 cache,
which predates lib/util/research-loop-cap.mjs -- correct behaviour, and it
clears when the plugin is reinstalled.
Review findings 2670c10a, fbd6d534, 93550dfb.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vPSXe88qp5aqWUqbDNWoF
Review finding 91e21c1f (MAJOR emitted, catalogue tier BLOCKER). Live in every
session regardless of the STORM flag.
The hard-block tier is the one thing no operator flag unlocks - not `strict`,
not `--soft`, not VOYAGE_QUERY_PRIVACY_ALLOW=1 - so a format it misses is a
secret leaving the machine in an outbound query with no second gate behind it.
Two patterns keyed on a run of consecutive alphanumerics, which a `-` or `_`
inside the token body breaks:
- /\bsk-[A-Za-z0-9]{20,}\b/ was commented "OpenAI/Anthropic-style" but the run
ends after `api03` (3 chars) in an Anthropic Console key, so
sk-ant-api03-<~95> passed through. Measured by execution, not read.
- /\bghp_[A-Za-z0-9]{36,}\b/ covered only the classic prefix: github_pat_<...>
and gho_<36> passed through; ghp_<36> was blocked.
Widened with one pattern per real-world format rather than one loose pattern,
so each stays readable and its length floor stays honest:
- sk-ant-<scheme>-<20+ base64url> (covers api03 and oat01)
- gh[pousr]_<36+> (classic PAT, OAuth, user, server, refresh)
- github_pat_<20+ incl. underscore> (fine-grained, real format is <22>_<59>)
Patterns whose body class includes `-`/`_` carry no trailing \b - it would not
fire on a non-word final character. The existing sk-/AKIA/xox/PEM patterns are
unchanged; the AWS comment is accurate as written, so it was left alone.
Formats verified against GitHub's token-format documentation and Anthropic key
anatomy before the patterns were written, not from memory:
- github.blog/engineering/platform-security/behind-githubs-new-authentication-token-formats/
- gh[pousr]_ + 36 chars; github_pat_ + <22>_<59> = 93 total
- sk-ant-api03- + ~95 base64url chars (base64url includes _ and -)
6 new tests: three table rows for the missed formats, and three that pin each
one blocked with `strict: false` AND the opt-in env var set at once - the
property that makes this tier meaningful.
Suite 952 (950/0/2, baseline 937 + 15 across both Track A fixes).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013zNqxP8qTWgJhn3wMYUFEh
The documented `brief-validator.mjs <brief.md>` invocation (no flags) always
bailed to Usage/exit 2. Root cause: when --min-version is absent, minIdx is -1,
so the skip index minIdx+1 was 0 — excluding argv index 0, exactly where the
file positional sits in the no-flag case. Any leading flag (--soft, --json)
pushed the file to index >=1 and masked the bug, so the function-level tests
never caught it.
Guard the skip index to -1 when --min-version is absent. Add two CLI regression
tests (execFileSync, matching the next-session-prompt-validator pattern):
no-flag invocation reaches validation, and --min-version still skips its value
token to find the file. Suite 822 -> 824 (822/0/2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WH1krHamUehZh6JqVqs85t
Scores committed agent-run fixtures against the golden corpus at
(file, rule_key) granularity, building on the deterministic coordinator
contract (4a). Offline: committed reviewer payloads, no live agent spawn,
no LLM, no network (the LLM-in-the-loop grading is the separate 4c tier).
- lib/review/gold-scorer.mjs: scoreFindings (precision/recall/f1 at
(file,rule_key) granularity, line+severity ignored) + scoreVerdict; pure,
with documented vacuous-set conventions.
- tests/fixtures/bakeoff-rich/runs/run-perfect.json: committed run that
reproduces all 5 seeded gold findings through runContract.
- tests/lib/gold-eval.test.mjs: the scoring RUN (precision/recall/f1 = 1.0,
verdict == expected_verdict BLOCK, nothing suppressed/skipped).
- lib/util/test-census.mjs: third census category (goldEval) — a scoring run
is neither behavior coverage nor a doc-pin; honest-count invariant now 3-way.
- docs/eval-corpus/README.md: 4b moved from Future hardening to implemented.
Suite 809 -> 822 (820/0/2). gold-scorer covers TP+FP+FN+degenerate paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJQYC5vpkJWxndS55vQQZ6
The economy profile's cross-tier Jaccard floor (0.55) rests on parked-synthetic
fixtures; empirical Step-17 calibration is v4.2-budget-gated ($60–120,
unauthorized). Fork resolved as label-not-calibrate: the experimental status,
previously prose-only in docs/profiles.md, now lives in the profile data and is
machine-checked. No new user-facing capability — honest labeling + a guard.
- lib/profiles/economy.yaml: add `experimental: true` (with rationale comment).
- lib/validators/profile-validator.mjs: recognize `experimental` as an OPTIONAL
boolean; non-boolean → PROFILE_INVALID_ENUM. Absent ⇒ tier is stable
(premium/balanced unaffected, profile_version stays 1.0 — additive).
- README.md + docs/operations.md + docs/profiles.md: flag the `economy` table
row "⚠ Experimental (uncalibrated Jaccard floor)".
- tests/synthetic/profile-jaccard-calibration.md + analysis §6/§10 + backlog
plan §S34: cross-reference the marker; mark V30/S34 done.
+5 tests (739 → 744, 742/2/0): economy declares experimental:true; premium and
balanced do not; validator rejects non-boolean experimental; every profile-doc
economy row is flagged; the flag tracks the calibration's parked-synthetic
status (must drop in the same change that lands real calibration).
Closes the balance backlog (4/4, S31–S34). claude plugin validate green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
The plan template (templates/plan-template.md:21) emits plan_version ONLY in
the prose blockquote metadata line:
> Generated by trekplan v{version} on {YYYY-MM-DD} — `plan_version: 1.7`
It carries no frontmatter plan_version (the frontmatter block is trekreview-only
and holds source_findings). But PLAN_VERSION_REGEX in lib/parsers/plan-schema.mjs
was `^`-anchored (/m), matching ONLY line-start (frontmatter). The backtick-
wrapped prose form never matched → extractPlanVersion returned null → every real
generated plan got a spurious PLAN_NO_VERSION warning from plan-validator.
Not caught earlier because no test ran the parser against the actual template;
the synthetic fixtures all carry frontmatter plan_version and parse fine.
Internal source contradiction surfaced: the parser comment documents intent
"frontmatter OR doc body", planning-orchestrator.md:228 + the template place it
in the prose line, while HANDOVER-CONTRACTS.md calls it a frontmatter field.
Operator-chosen fix (parser, not template): relax the regex to
`/(?:^|`)plan_version:.../m` so it matches line-start (frontmatter) OR the
backtick-prefixed prose form. Honors the parser's own documented contract,
fixes all existing + future plans, changes no plan output, touches one code
file — vs the template-fix which would alter every plan's shape, contradict
planning-orchestrator.md, and force a 3-file doc reconciliation (scope creep).
TDD (red first): added prose-form + canonical-template regression pins in
plan-schema.test.mjs and a no-PLAN_NO_VERSION pin in plan-validator.test.mjs;
all 3 red on the `^`-anchored regex, green after the relax. The template pin
ties the parser directly to the real generated artifact.
S22 defect closed in docs/S22-happy-path-dogfood.md (§Pipeline defects + log).
Tests: 728 (726 pass / 2 skip / 0 fail; live `node --test`, = 725 baseline +3).
plugin validate passes (1 accepted CLAUDE.md-at-root warning).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Phase 9's dedup hand-off was broken on two layers, both surfaced by the S22
dogfood: (a) plan-critic + scope-guardian (Read/Glob/Grep, no Write) were told
to write /tmp/*-out.json the dedup helper reads — they cannot; (b) even with
Write, their Output format emits markdown, not the helper's
{agent,findings:[{file,line,rule_key,text}]} schema. readJsonOrNull then
swallowed the absent files -> a silent empty merge that discarded every finding.
Fix (operator-chosen A'): keep the reviewers read-only; make the hand-off run.
- plan-review-dedup.mjs gains a --stdin mode reading {plan_critic,scope_guardian};
malformed stdin exits non-zero so a broken hand-off surfaces loudly instead of
collapsing into a silent empty merge. File mode + its tests are untouched.
- plan-critic.md + scope-guardian.md now emit a trailing machine-readable `json`
findings block (the inline hand-off; no Write tool needed).
- trekplan.md + planning-orchestrator.md Phase 9 rewritten in lockstep: extract
both blocks, pipe via heredoc into --stdin. No temp files, portable, no
pathguard dependency.
TDD: malformed-stdin test failed first (CLI ignored stdin -> exit 0 = the bug),
green after impl. New S23 doc-pin asserts both docs use --stdin (not the dead
/tmp paths) and both agents declare the json block. Suite 724 (722/2/0); live
HEAD baseline was 720, not the stale 705 STATE carried forward.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
S21 = devil's-advocate blind spot #2 (security/safety), operator scope
"security-core": fix genuine defects, honestly record the rest.
SSRF fix (CWE-918). validateOtlpEndpoint classified the host by literal
string match. Decimal/hex/octal/trailing-dot encodings are already caught
(the WHATWG URL parser canonicalizes them), but IPv4-mapped IPv6 literals
(::ffff:127.0.0.1, ::ffff:192.168.x, ::ffff:169.254.169.254) render as
::ffff:HHHH:HHHH and matched neither the loopback set, the RFC-1918 regex,
link-local, nor HARD_BLOCKED_HOSTS — they passed over https and would reach
loopback / private / cloud-metadata, defeating the comment's "PERMANENTLY
block metadata" promise. Added mappedV4() to decode the embedded IPv4
(dotted + hex-pair forms) and classify on it. TDD: 4 tests failing-first —
mapped loopback/RFC-1918/metadata rejected (metadata stays HARD_BLOCKED even
with VOYAGE_OTEL_ALLOW_PRIVATE=1), mapped public ::ffff:8.8.8.8 still valid
(no over-block). Threat model narrow (endpoint is operator-set env; export
opt-in; a brief cannot set env).
Survivor #18 honest-residual. T2-bakeoff §3 "Classifier interference: 0"
read as satisfied, but the auto/bypass-mode re-run that matters for headless
trekreview was never run (mode is operator-set, not settable in-session) —
footnoted, not gating. Per recommendation #9, moved to an OPEN RESIDUAL
(untested), guarded by a new doc-consistency pin.
Audit record. ## S21 resolution block in devils-advocate-results.md
dispositions all four sub-questions: SSRF (fixed), hooks-block (verified;
advisory-rail residuals: Write-only matcher, Bash-redirect, regex gaps — by
design), malicious-brief-headless (catastrophe-blocked, exfil NOT blocked —
inherent limit). S21b flagged (NOT done): operations.md:15 mis-describes
autonomy-gate.mjs state machine + --gates table.
Test count: real baseline 700 (698 pass / 2 skip), NOT 715 — the node:test
headline carried in S20 commit msgs was stale/miscounted (census figures
were correct). Now 705 (703 pass / 2 skip / 0 fail); census behavior
601->605, doc-pins 71->72, total 672->677.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Closes devils-advocate audit §Top changes #8 (MINOR×MED). Reports the
behavior-test count separately from the doc-consistency prose-pin count so the
cited total no longer oversells behavior coverage, and removes the clearest
reword-fragile redundancies. Conservative scope (operator-chosen): no genuine
regression guard dropped.
1. Suite census (report separately). lib/util/test-census.mjs +
tests/lib/test-census.test.mjs walk tests/**/*.test.mjs and bucket top-level
test() declarations into behavior vs doc-consistency-pins (bucket per file,
regex PIN_FILE_RE), asserting the two sum to the total so neither can drift
silently. Split emitted as a t.diagnostic: behavior=601 doc-consistency-pins=69.
Metric = top-level declarations; node:test's runtime total counts subtests
too and is therefore >= it.
2. Conservative prune (3 tests, each provably subsumed):
- "trekexecute.md still parses v1.7 plan schema" — tautological OR-chain;
real coverage = the plan_version:1.7 template pin + plan-validator /
plan-schema behavior tests.
- "CLAUDE.md mentions all six pipeline commands" — hardcoded six-string list
subsumed by the filesystem-driven "commands table mentions every
commands/*.md file" structural pin.
- "CLAUDE.md mentions /trekcontinue command" — same subsumption.
Each removal leaves an in-place note (why + where coverage lives).
3. doc-consistency.test.mjs header now documents the structural-invariant vs
prose/existence-pin distinction so new pins land in the right kind.
TDD: census test written failing-first (ERR_MODULE_NOT_FOUND before
lib/util/test-census.mjs existed). Suite 699->698 (696 pass / 2 skip / 0 fail):
-3 prune +2 census. plugin validate passes (1 accepted warning).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Closes devils-advocate audit #4–#6 (MAJOR×MED). Four additive, non-breaking
changes to the v5.5 framing-alignment machinery:
1. --min-brief-version <ver> gate (audit #4). Opt-in version floor on
/trekplan + /trekresearch, forwarded to brief-validator as --min-version.
New opts.minBriefVersion warns BRIEF_VERSION_BELOW_MINIMUM (never blocks)
when a brief declares a version below the floor; trekreview exempt; absent
opt = no check. CLI shim parses --min-version and skips its value token in
filePath detection.
2. memory_alignment.status field (audit #5). brief-reviewer now emits
status: verified | n_a | contradictions so a score-5 N/A (no memory) is
distinguishable from a score-5 verified-aligned brief — the score≥4 gate
passes in both, status reveals whether the wrong-premise defense ran.
3. Document pre-2.2 = zero framing enforcement (audit #4). HANDOVER-CONTRACTS
§Handover 1 now states the producer-elective hole + two remedies. Also
fixes a stale "current is 2.1" line (current is 2.2).
4. Soften flagship overselling (audit #6). CLAUDE.md Context-Engineering
principle now hedges that main-context relief is asserted-by-design, not
measured (T1 PoC found Δ≈0); README carried no false claim to fix.
TDD: 8 new tests written failing-first (5 validator, 1 trekbrief status pin,
2 doc-consistency cross-file pins). Suite 691→699 (697 pass / 2 skip / 0 fail).
No flagship prose-pin added (deliberate, per S19 anti-bloat).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Make the bake-off-validated Workflow substrate (Arm B) reachable behind an
opt-in --workflow flag for /trekreview. Default Phase 5-6 path stays prose
to preserve the lower portability floor; --workflow raises the consumer
floor to Claude Code 2.1.154+ (the Workflow tool).
- arg-parser: --workflow added to trekreview boolean flags
- commands/trekreview.md: flag row + Phase 5 substrate-routing gate +
new section 'Phase 5-6 via the Workflow substrate' (invocation contract,
S10 gotchas, bake-off citation, auto/bypass residual as Known limitation)
- docs/command-modes.md: --workflow row in /trekreview table
- routes to existing scripts/trekreview-armB.workflow.mjs (byte-identical to
the S10 part-B POSITIVE build); integration is pure routing, no script change
TDD: 8 new tests (arg-parser flag recognition + combine; command/doc prose
pins for route, opt-in posture, 2.1.154+ floor, bake-off evidence).
Suite 662 -> 670 (668 pass / 2 skip / 0 fail). plugin validate clean modulo
known root-CLAUDE.md warning. Resolves W1-narrow-wins-plan.md S11.
Build the prose-vs-Workflow bake-off machinery for /trekreview Phase 5-6 and
run a 1-run/arm smoke to de-risk before the full measurement (operator posture:
build + smoke, then pause for go/no-go on the full >=3-runs/arm run).
New:
- lib/review/fidelity-diff.mjs (+ tests) — the PRIMARY metric: parse two
review.md (or two structured arm outputs) and compare verdict + jaccard over
(file,line,rule_key)-IDs + per-finding severity/rule_key. Reuses jaccard +
frontmatter + NW1 findings-schema + finding-id. fidelityDiffStructured avoids
rendering review.md per run.
- scripts/trekreview-armB.workflow.mjs — Arm B: Phase 5-6 as a Workflow
(parallel([conformance, correctness]) schema-forced -> JS dedup-by-triplet ->
agent(review-coordinator) verdict schema). Path-based input via args (reviewers
carry Read). Inlines dedup + the 12-key rule_key enum (scripts have no imports).
- tests/fixtures/bakeoff/ — committable fixture: real diff of b149538 (NW1) +
brief reconstructed from plan S9. Both arms review the same pinned input.
- docs/T2-bakeoff-results.md — smoke results + verdict + go/no-go recommendation.
Smoke result: SMOKE PASS. Arm B runs the full pipeline (3 agents) with ZERO
classifier interference; fidelity EQUIVALENT to Arm A at the verdict level
(both ALLOW; jaccard 1.0). Caveat: the clean TDD'd fixture yielded ~0 findings,
so finding-SET fidelity was not stressed (only verdict fidelity proven). A
reviewer-level divergence appeared (Arm B raised 1 raw finding, coordinator
filtered it; Arm A raised 0) — to be quantified in the full run on a
richer-finding-surface fixture. NOT the T2 §5 POSITIVE/NEGATIVE verdict.
Suite 647 -> 662 (660 pass / 2 skip / 0 fail; +15 fidelity-diff). claude plugin
validate clean (known root-CLAUDE.md warning only). Plan: docs/W1-narrow-wins-plan.md S10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Codify the /trekreview Phase 5 reviewer JSON contract as a validated schema so
main validates each reviewer's output instead of merely JSON.parse-ing it, and
re-asks on schema failure (not just parse failure). Retires the fragility at
trekreview.md:202-204. No Workflow dependency (S8 tier 1, ships regardless).
New lib/review/findings-schema.mjs (3-layer Content -> Raw-text -> CLI shim,
reusing result.mjs error-shape + rule-catalogue RULE_KEYS/SEVERITY_VALUES):
- validateFindings(payload): hard errors on load-bearing fields the dedup
triplet + verdict depend on — file, rule_key (in catalogue), severity (enum),
line (integer >= 0); accumulates all errors; per-finding location.
- extractFindingsBlock(text): last fenced ```json block (the :202 contract).
- validateReviewerOutput(text): extract + parse + schema, unifying parse and
schema failures under one bounded re-ask path.
Stable codes: FINDINGS_NOT_OBJECT/_NOT_ARRAY/_NO_JSON_BLOCK/_PARSE_ERROR (top),
FINDING_MISSING_FILE/_MISSING_RULE_KEY/_UNKNOWN_RULE_KEY/_BAD_SEVERITY/_BAD_LINE
(per-finding). Descriptive fields + unknown keys tolerated (forward-compat).
Phase 5 prose: replace "parse last json block; on parse error re-emit" with
"validate against findings-schema; on failure re-ask conforming JSON, bounded
N=2; never feed unvalidated findings to the coordinator".
TDD: 27 failing tests first, then minimal code to pass. Suite 606 -> 647
(645 pass / 2 skip / 0 fail). claude plugin validate clean (only the known
root-CLAUDE.md warning). Plan: docs/W1-narrow-wins-plan.md S9.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Implements the CLAUDE.md cross-cutting invariant "brief framing must match
operator intent" as a controlled brief_version 2.1->2.2 bump (operator option A1).
Three defense layers, version-gated at >=2.2 so existing 2.0/2.1 briefs stay
valid (forward + backward compatible), mirroring the phase_signals >=2.1 gate:
- L1 framing: enum field (preserve|refine|replace|new-direction). Enum-checked
on any version when present (BRIEF_INVALID_FRAMING); missing at >=2.2 ->
BRIEF_MISSING_FRAMING. /trekbrief Phase 2.5 collects it BEFORE any brief prose
(non-skippable, even in --quick).
- L2 memory alignment: new brief-reviewer dimension 6 comparing brief Intent/Goal
+ framing against operator memory for explicit contradictions; degrades to
score 5 (N/A) when no memory context is supplied. Wired into Phase 4e gate
(memory_alignment.score >= 4).
- L3 obligatory ## TL;DR (<=5 content lines) at >=2.2; soft cap ->
BRIEF_TLDR_TOO_LONG warning.
trekreview briefs are exempt from the framing/TL;DR gate. Handover 1 PUBLIC
CONTRACT doc, README "What's new", and the CLAUDE.md invariant + agents table
(brief-reviewer 5->6 dimensions) updated to 2.2 (schema axis only; plugin
version badge + CHANGELOG remain S10).
Iron Law followed: validator tests red->green first. Tests 586 -> 606
(+20, 604 pass / 2 skip). claude plugin validate passes (pre-existing
CLAUDE.md root-context warning unchanged).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
S4 of the 2.1.181 upgrade — implementation, not a gate. TDD: failing test
written first for the resolver gate, then the fix; suite green throughout.
- Resolver MAJOR (FIX): lib/profiles/phase-signal-resolver.mjs now imports
BASE_ALLOWED_MODELS from profile-validator and gates `model`
(if 'model' in entry && BASE_ALLOWED_MODELS.includes(entry.model)),
mirroring the EFFORT_LEVELS gate one line up. Out-of-allowlist models
(gpt-4, haiku) are dropped instead of handed to an agent spawn —
defense-in-depth behind brief-validator's validation-time check. No
circular import (brief-validator already imports the same symbol).
+2 tests (drops-invalid / keeps-valid).
- Native effort: (SHIP, static additive): effort: frontmatter on 8 agents —
retrieval (task-finder, git-historian, dependency-tracer,
architecture-mapper) = medium; adversarial-reasoning (plan-critic,
risk-assessor, contrarian-researcher, review-coordinator) = high. The
other 15 stay unset -> inherit Opus-4.8 default (high). This per-spawn
REASONING effort is a different axis from brief phase_signals.effort
(ORCHESTRATION shape) per the S3 decision.
- Doc-truth + axis distinction: new canonical docs/profiles.md
§Model & effort axes (opus->Opus 4.8 default-high; orchestration vs
reasoning effort table; native-effort precedence; per-agent levels).
Short notes in CLAUDE.md (after Agents table) and README.md (Cost
profile), both pointing to profiles.md.
- Open (non-blocking, unchanged): only STATIC effort shipped — the
verified-safe minimum. Profile-driven DYNAMIC effort still needs
verification of the per-spawn effort param or env-var injection.
Matrix: new "S4 resolutions" section. Tests 582 total / 580 pass / 0 fail /
2 skip (was 578 pass; +2). claude plugin validate passes (only pre-existing
root-CLAUDE.md warning).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
v5.1.0 shipped with an unquoted brief_version: 2.1 in trekbrief-template.md.
parseScalar coerced it to Number 2.1, and the sequencing gate guarded on
typeof === 'string', silently bypassing BRIEF_V51_MISSING_SIGNALS.
Three-part atomic fix:
- brief-validator.mjs:87+149 now accepts both string and number forms via
String(fm.brief_version) coercion.
- trekbrief-template.md quotes the value so new briefs parse as String.
- doc-consistency.test.mjs pins the QUOTED form going forward.
Three regression tests added in brief-validator.test.mjs.
The v4.2/v4.3 bespoke playground SPA (~388 KB), the /trekrevise command,
Handover 8 (annotation → revision), the supporting lib/ modules
(anchor-parser, annotation-digest, markdown-write, revision-guard), the
Playwright e2e suite, and the @playwright/test / @axe-core/playwright
devDeps are removed. A browser walkthrough found the playground borderline
unusable, and it duplicated the official /playground plugin's
document-critique / diff-review templates.
In their place: scripts/render-artifact.mjs — a small, zero-dependency
renderer that turns a brief/plan/review .md into a self-contained,
design-system-styled, zero-network .html (frontmatter folded into a
<details> block). /trekbrief, /trekplan, and /trekreview call it on their
last step and print the file:// link; to annotate, run /playground
(document-critique) on the .md and paste the generated prompt back.
Resolves the v4.3.1-deferred findings as moot (their target files are
deleted). npm test green: 509 tests, 507 pass, 0 fail, 2 skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pure module computing deterministic 16-char SHA-256 prefix for annotation set.
Canonicalization: sort by id, fixed field order (id|target_artifact|target_anchor|intent|comment|timestamp), \n-join, sha256, take first 16 hex.
Brief SC4 specifies sha256-prefix; research-05 said sha1 — brief wins per Hard Rule "Brief-driven".
6 tests pass: empty digest, order-independence, intent-sensitivity, format invariant, golden value, undefined-vs-empty equivalence.
Found by simulert v4.1 smoke — doc/code-drift in v4.1 ship:
docs/observability.md claims "Cloud metadata endpoints (169.254.169.254)
are permanently blocked" but the validator allowed them when
VOYAGE_OTEL_ALLOW_PRIVATE=1. Cloud metadata services expose IAM
credentials and instance secrets — operator-trust extended to
RFC-1918 home-lab access does NOT extend here, because the
blast-radius (cloud-account compromise) is qualitatively different.
New HARD_BLOCKED_HOSTS set checked BEFORE the link-local opt-in path:
- 169.254.169.254 (AWS / GCP / Azure metadata)
- 100.100.100.200 (AliCloud metadata)
- metadata.google.internal
- metadata.azure.com
New error code ENDPOINT_HARD_BLOCKED. Existing test for
ENDPOINT_LINK_LOCAL_REJECTED on 169.254.169.254 updated to assert
the new code; 3 new tests verify the hard-block holds even with
VOYAGE_OTEL_ALLOW_PRIVATE=1, plus AliCloud + GCP-hostname coverage.
Tests: 487 → 490 pass + 2 skipped.
Step 20 of v4.1 — implements drift detection in plan-validator.mjs per
brief Assumptions block 7: "Mismatch (e.g. korrupt manuell endring)
emitterer MANIFEST_PROFILE_DRIFT-warning fra plan-validator i --strict-modus."
Logic (after validateAllManifests in validatePlanContent):
1. Strict-mode only — soft mode never emits drift warnings.
2. Plan frontmatter must declare 'profile: <name>' to establish baseline.
3. For each step manifest, if profile_used is set AND differs from plan
profile, emit warning (NOT error) with code MANIFEST_PROFILE_DRIFT
and location 'step N: profile_used = X, plan profile = Y'.
Forward-compat preserved: drift is a warning, plan remains valid:true.
Operators see the drift in --strict mode without parsing breaking.
New files:
tests/validators/plan-validator-profile-drift.test.mjs — 4 tests
tests/fixtures/plan-profile-drift.md — drift fixture
Tests verify:
1. drift detected in strict mode → MANIFEST_PROFILE_DRIFT in warnings
2. drift NOT detected in soft mode → strict gate honored
3. matching profile → no drift warning
4. no plan-level profile → drift detection silent (no baseline)
Tests: 479 pass + 2 skipped (Docker not installed).
