Offline structure tests for evals/ (the suite itself needs headless runs):
- no case reaches the steg 1 intent gate (PM: 0 traces touched it). The
new case's scaffold is in: its brief passes brief-validator --soft
(2.1 WITH phase_signals) and --check gives BRIEF_INTENT_NOT_APPROVED —
that test is green, so the fixture reaches the gate.
- review-requires-project: trekreview.md composes 'Error: --project <dir>
is required.' in prose (8/10 in the PM run, backticks broke the regex),
and its arg-parser line passes "$@", which the Bash tool never has.
- no-error-code misses REVIEW_WRONG_TYPE; PASS/FAIL graders are raw
substrings; no-write cannot see a write through Bash.
7 tests, 6 red.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 7 of /trekbrief and Phase 12 of /trekplan say "skip silently" when
CLAUDE_PLUGIN_DATA is unset, and the Bash tool env never carries it — the
same silent skip that left brief-approved at 0 records. Økt 2's countable
form (brief-approved + intent_approved: true + a trekplan record with the
same slug) cannot be counted until those records land where the yardstick
reads. 7 tests, 7 red: no stats-append line in either command.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`origin` is a public mirror. Comments in tracked test files named the internal
review mechanism by its role rather than the measurement they came from; the
same class as the frozen file's overclaim fixed in 83d82f1. Replaced with what
they actually are: a re-measurement on a named commit, with the date.
The one remaining occurrence is the assertion that FORBIDS those phrases in
tests/fixtures/end-state-frozen.json - a guard has to spell out what it guards.
Suite 1161 (1159/0/2). Comment-only change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured 2026-09-18: nine ordinary shapes of a plan's `## Verification`
section parsed to zero criteria - an untagged fence, a ```text fence, a
markdown table, `## Verification (acceptance)`, `## Verification:`,
`### Verification`, an unclosed fence earlier in the document. Every one came
out as `0 of 0`, NOT OK, exit 1, and Phase 7 then forbade `result: completed`
without anyone being told that the FORMAT, not the code, was the problem.
"The section is empty" and "I cannot read this format" are different facts.
Three changes, one hole:
- The runner reports `NO_CRITERIA` with a source line (`plan.md:NN`) when the
section is there and nothing in it parsed, and names the two forms it does
read. Same for a brief's `## Success Criteria`, so the evidence block the
conformance reviewer gets says which of the two it is looking at rather than
showing an empty table.
- Phase 7 says it out loud instead of failing silently: report the source line
and the two forms, and say that the plan is what failed there, not the run.
- `/trekplan` now pins what it produces to what the runner reads: the heading
is exactly `## Verification`, the criteria are a bullet whose first
backticked span is the command or a shell-tagged fence, and the command must
be one the allowlist runs. A doc-consistency test holds the writer and the
reader together, so a runner that learns a new form must update the source.
Honest about the round trip: the two round-trip tests were GREEN on arrival -
the template already writes the bullet form the runner reads. What was missing
was not the format but the PIN: `/trekplan` mandated neither the heading string
nor the format, so a plan could satisfy the command's own instructions and
still parse to nothing. The tests now hold that.
Red first: 5 of the 7 new tests failed before the change (3 NO_CRITERIA, 2
doc-consistency); the 2 round-trip tests are guards, and said so above.
Suite 1161 (1159/0/2). Gate unchanged: defects 0 of 7, intact, exit 1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A denylist in front of /bin/sh is whack-a-mole. Measured 2026-09-18, end to
end through both screens: 5 of 11 named evasions ran with real effect - a
`command` prefix reached git, an escaped `rm` inside a shell fence deleted a
directory, `find -delete` deleted a file, `>|` and `tee` wrote outside the
working tree, a python one-liner deleted the whole tree - and 19 of 28 got
past the refusal list on its own. Every quoting, aliasing and indirection form
of the shell is another mole.
So the screen is now an ALLOWLIST. A criterion runs only when its first word
is a known test runner (npm test, npm run <script package.json declares>,
node --test, vitest, jest, pytest, python -m pytest, uv run pytest,
cargo test, go test, make test, bash <script under tests/>, a read-only git
subcommand) AND the command carries no shell operator and no newline.
Everything else is NOT RUN with the reason said out loud: never run, and never
reported as a failure either - an absent measurement is not a finding. That
also closes the smaller hole in the same file: a bare word a sentence merely
names (`whoami`, `login`, `package.json`) is no longer executed, because it is
not a runner.
REFUSED_BY_POLICY is gone with the list that produced it; a command outside
the allowlist is `unrunnable`, which in plan mode still fells the run and in
brief mode is reported to the reviewer as an absent measurement.
What the allowlist deliberately does NOT do, said in the file and in the
reviewer's rubric: it is not a sandbox. `npm test`, `npm run <script>` and
`make test` run whatever the repo's own package.json/Makefile says they run,
including a script that pushes - that is the repo's responsibility. And it
rejects honest commands too: an env prefix, a project's own binary, anything
piped. A check that needs one of those is declared through
`bash tests/<script>.sh`, the documented way in.
Red first: 6 of the new tests fail against the previous runner (measured with
an always-allow shim so the module still loads), including the end-to-end one
where the canary directory was deleted and files were written outside the
tree. The fixtures move from `true`/`false` to two allowlisted shell fixtures,
because `false` is no longer a runner - the fail case must still be a real
non-zero exit, not an unrun criterion.
Suite 1148 (1146/0/2).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/trekreview passed --cwd and trekexecute Phase 7 did not, so the same criterion
could resolve two ways in the two phases (PM checkpoint 2026-09-18, MINOR).
It stopped being cosmetic with the refusal list: --cwd is the boundary a write
is measured against, so leaving it unset silently moves that boundary to
whatever the process cwd happens to be. Both phases now resolve
CRITERIA_CWD="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
- one form, with a fallback, so a repo-less checkout falls back instead of
passing an empty --cwd. Pinned in tests/lib/doc-consistency.test.mjs.
Red first: the new pin failed on both phases before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/trekreview now runs the commands a BRIEF declares, and a brief is an artifact
that can arrive from outside the repo. Measured 2026-09-18 on 6cafb4c: the
executor denylist stopped a download piped into a shell, but the remote-writing
git subcommand and a recursive delete of a path both RAN. The denylist screens
catastrophe (root deletion, fork bombs, mkfs); it was never meant to screen an
artifact under review.
A second screen, in the runner and ahead of the denylist, refuses four classes:
- a remote-writing git subcommand. The subcommand is found by walking git's own
options (`-C`, `-c`, `--git-dir`, ... take a value), so `git status` and
`git log` still run and `git -C sub push` does not.
- a recursive delete: any `rm` carrying `-r`/`-rf`/`--recursive`. A plain
`rm build/artifact.txt` still runs.
- a download piped straight into a shell (also caught by the denylist; pinned
here so the runner does not depend on another file for it).
- a write outside the working tree. `/dev/null`-class devices are fine, and so
is anything under the working tree; `~/...`, an absolute path elsewhere, and
a target carrying an unexpanded `$VAR` are refused - the runner cannot know
where a variable points, and guessing is how a screen stops screening.
A refusal is its own outcome, REFUSED_BY_POLICY: the command never reaches a
shell, and `summary.ok` is false in both plan and brief mode. For the reviewer,
REFUSED is like NOT RUN - the absence of a measurement, never on its own a
finding - and the rubric and the evidence block both say so.
Chosen deliberately, and it is stricter than today's habit: writing scratch to
/tmp is refused too. The repo's own example plan does `> /tmp/out`. Verification
output belongs in the working tree; exempting the whole system temp dir would
have made the rule unstatable, since a working tree created under /tmp then
contains its own escape hatch.
NOT covered, stated rather than implied:
- other writing git subcommands (tag, remote, config, gc) - only push is listed
- writes through a wrapper: `sh -c '...'`, `xargs`, `find -exec`, a Makefile
target, a script the criterion invokes. The screen reads the command it is
given, not what that command goes on to do.
- `>` inside a quoted string reads as a redirect, so a criterion echoing a
literal `>` is refused. Fail-closed, on purpose.
- the whole surface still runs with the invoking process's permissions; this is
a refusal list, not a sandbox.
The denylist-layer test now uses a stand-in command with a screen double: the
refusal list catches a recursive delete first, so naming one there would have
stopped exercising the denylist layer at all.
Red first: the 6 new tests failed before this change (`refuseCommand` did not
exist), and the fixture brief's four writes ran.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"The first backtick span is the command" is right for a plan, whose template
puts the command first, and wrong for a brief, whose criterion usually opens
by NAMING the thing under discussion. Measured 2026-09-18 on the repo's own
example brief: 5 of 6 criteria FAILED, 3 of them parse artifacts - `--verbose`
run as a command gave exit 2 ("invalid option"), `tests/` gave exit 126 ("is a
directory"). The rubric reads a FAILED result as decisive, so each one became
a BROKEN_SUCCESS_CRITERION BLOCKER about prose.
looksLikeCommand() screens the span by SHAPE only - no filesystem lookup, so a
span parses the same everywhere. Refused: a leading flag, a directory, a token
carrying quotes/braces/prose, and a lone relative path with a slash (an
explicit ./, ../, / or ~/ still runs, as do env-var prefixes). A refused span
is `unrunnable` with reason `not-a-command` - its own outcome, never FAILED,
and it never reaches a shell.
It deliberately does NOT scan on to a later span. "The first span that LOOKS
like a command" invents commands out of prose: in that same example brief it
would have run `whoami` and `login`, two real binaries a sentence happens to
name. An absent measurement is honest; a guessed one is not.
The shape check applies to prose spans only. Inside a shell-tagged fence the
author has already declared shell, so `[ -f x ] || exit 1` still runs.
The rubric follows: a NOT RUN result is never on its own a finding. The
Partial row now describes half-built DELIVERED CODE, and the reviewer gets a
table of the three reason strings - no-command, placeholder, not-a-command -
with what each says about the sentence rather than about the code.
Not covered, stated for the record: a multi-token span whose first token is a
non-executable file (`tests/golden/login.stdout --check`) still runs, and a
criterion whose command is real but whose binary is absent still reports the
shell's exit 127 - that is a true measurement of a missing binary, not a
parse artifact.
Red first: 4 runner tests + 1 doc-consistency pin failed before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The runner read bullet lines only. The repo's own example plan writes its
whole acceptance run as a fenced bash block, so `## Verification` parsed to
ZERO criteria, the runner exited 1, and Phase 7 forbade `result: completed`
- a correct plan felled every single-session run. Measured 2026-09-18 on
6cafb4c: none of the repo's plan artifacts exited 0.
Fences are now read twice over, for two opposite reasons:
- a `## ` heading INSIDE a fence is quoted text and no longer opens a
section. examples/02-real-cli/REGENERATED.md is a report that quotes a
plan outline inside one fence; it used to yield an empty section that
read as "0 of 0", and now yields the honest NO_VERIFICATION_SECTION.
- a shell-tagged fence inside the section holds the commands. The tag list
is closed (bash/sh/shell/zsh/console/shell-session): an untagged fence is
more often expected OUTPUT than input, and inventing a criterion from
output is the failure this file exists to prevent.
Blank and comment-only lines inside the block declare nothing. A `$`/`>`
console prompt is stripped; a `#` root prompt is NOT, because it cannot be
told from a comment and running a comment is the worse mistake.
Measured after the fix (parse only - one example names a fictional CLI):
examples/01 6 criteria, plan-template 2 (both placeholders, correctly NOT
RUN), the two runner fixtures 2 each, plan-run-C 1, REGENERATED.md 0 with
NO_VERIFICATION_SECTION.
Divergence from the order's premise, stated for the record: it said 2 of 3
example plans write `## Verification` as a fenced bash block. Ground truth
is 1 of 3 - REGENERATED.md has no section of its own at all - and the
example plan's block holds 6 command lines, not 5.
Red first: 4 of the 6 new tests failed before this change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rubric required `brief-conformance-reviewer` to classify a Success
Criterion as Full only when "its verification command/test exists and passes".
Its tools are `Read`, `Glob`, `Grep`. It cannot run anything, so "passes" was
either guessed from the command's mere existence or quietly downgraded to
"exists" — a BLOCKER-tier rule key resting on an impression.
The reviewer stays read-only — a reviewer that executes the code it reviews is
not an independent reviewer. The command does the running instead:
- `/trekreview` Phase 4.5 runs the brief's `## Success Criteria` commands
through `lib/verification/criteria-runner.mjs --brief --evidence` and captures
the block as `sc_evidence_block`, pasted verbatim into the reviewer prompt in
Phase 5. The exit code does not stop the review — a failing criterion is
exactly what the review exists to find.
- `formatCriteriaEvidence` builds that block in code: one row per criterion with
the command, the exit code and the first output line. Chose a code-built block
over an orchestrator-written summary so the orchestrator cannot narrate a pass
that never happened.
- The rubric now judges the supplied result: `PASS` supports Full, `FAILED` /
`BLOCKED` is `Broken` with the exit code cited, and `NOT RUN` is the absence
of a measurement — never evidence in either direction.
- Phase 4.5 is skipped in `quick` mode: that mode does not launch the
conformance reviewer, so there is nobody to hand the result to.
Red first: seven tests in `tests/lib/criteria-runner.test.mjs` against a
committed brief fixture whose three criteria pass, fail, and are prose-only.
The two doc pins were verified red against the pre-fix files (rubric asked
"exists and passes"; no Phase 4.5; the block reached nobody).
Suite: 1117 (1115/0/2), up 9.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A trekplan's `## Verification` section is where the brief's success criteria
land. Phase 7 opened with "**Skip for trekplans.**", and only the multi-session
wave path (Phase 2.6 Step 3) ran master verification. A plan executed in ONE
session therefore reported `completed` without ever running the criteria it was
measured against — the executor's own belief was the only evidence.
The check now exists as code, not as an instruction:
- `lib/verification/criteria-runner.mjs` parses the criteria an artifact
DECLARES (a plan's `## Verification`, a brief's `## Success Criteria`), runs
each command, and returns a verdict built from exit codes. Fail-closed
throughout: a placeholder, a prose-only criterion, or an unavailable screen
is `unrunnable`/`blocked`, never `passed`. A plan with no `## Verification`
section exits 1 — a plan that promises no end-to-end check cannot be reported
as verified.
- Every command is screened through the plugin's own PreToolUse denylist
(`hooks/scripts/pre-bash-executor.mjs`) before it reaches a shell. Chose
invoking that hook over its documented stdin protocol rather than copying its
rules, because a command spawned from node never passes through the Bash tool
and so the hook cannot fire by itself — this keeps exactly one denylist.
- Phase 7 is now "Exit / verification check": session specs run the exit
condition, trekplans run the criteria runner. Phase 4's entry-condition skip
for trekplans stands — a plan carries no entry condition; the exit side is
not symmetrical.
- A failing criterion FELLS the run: `plan_verification.status != "passed"`
forbids `result: completed`. That is clause 2 of the stop-signal contract,
now enforced on the single-session path too.
Red first: `tests/lib/criteria-runner.test.mjs` (26 tests) against two committed
fixture plans, one of which declares a criterion that fails on purpose. The
doc pin in `tests/lib/doc-consistency.test.mjs` guards the wiring — a capability
no phase calls is the same defect wearing a lib/ file; verified red against the
pre-fix Phase 7 (skip present, runner absent, no fell-the-run clause).
Suite: 1108 (1106/0/2), up 27 from 1081.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
D-07 had been closed on CLAUDE.md alone. The same false claim lived on elsewhere:
- README.md's "Switch the planning model" said four commands default to opus.
- The same README section offered a sed recipe for `model: opus` frontmatter lines, which
0 of 7 command files have.
- commands/trekresearch.md said "the orchestrator runs on Opus".
Since v5.9.0 no command pins model:, so the orchestrator follows the session model.
- README: the section now says how it actually works. You pick the orchestrator's model with
the session model (`claude --model sonnet` or `/model`). Sub-agent models are resolved at
the spawn sites: brief phase_signals, then profile phase_models, then the agents/*.md pins
as fallback. `--profile economy` still moves the sub-agents to Sonnet.
- trekresearch Launch rules: "Use model: "opus" for all sub-agents" contradicted the same
file's own three-layer resolution, which v5.9.0 wired into the spawn sites and the file
documents in Phase 1 and Hard rules -> Cost. The rule now injects
`phase_signal_result.model` with agent frontmatter as the fallback, and says the
orchestrator follows the session model. Nothing changes under the default premium
profile: it still resolves to opus. The engine=swarm note now names the resolved-model
injection as the anchor, instead of the removed literal.
- Two doc-consistency pins were written first and failed before this change:
- no opus-orchestrator claim in README/CLAUDE.md/trekresearch, and no `^model: opus$`
recipe;
- the Launch rules name `phase_signal_result.model` and carry no blanket opus rule.
- Not touched: the follow-up agent in the default-off STORM phase still says model: "opus".
That is part of an experiment whose fate is an operator decision.
End-state gate D-07: open -> closed. Defects 3 -> 2 of 7 (D-03, D-04 remain, by design);
registry integrity intact. Suite 1081 (1079/0/2), run on a clean export of the index.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two "High-effort behavior (v5.1.1)" pins still described the rules as
"Decision B + gemini-bridge" and "contrarian-researcher + gemini-bridge
always-on". Both became wrong in 1ca48e0: trekplan's Pass 2 is now
"unavailable, skipped" and trekresearch forces only contrarian-researcher.
npm test stayed green throughout, and that is the point. The pins match on the
heading regex, not on their own message, so a stale message can never fail
them — and this file is structurally excluded from the S95 live-surface
scanner, because the S95 block is itself full of the word it bans everywhere
else. The blind spot is now written into the S95 comment rather than left for
the next reader to rediscover: when the high-effort rules change, these two
messages have to be re-read by hand.
Suite 1041 (1039/0/2), unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gemini-bridge is dropped by operator directive (three repetitions). The drop is
permanent, not a wait-for-SDK-upgrade state, so the change clears every LIVE
surface that could still steer a run toward the agent and leaves the historical
record alone.
Live surfaces cleared: agents/gemini-bridge.md deleted; trekresearch (bridge
launch block, --local help, high-effort always-on pair, stats record,
degradation list), trekplan, trekbrief, research-orchestrator (mode table,
agent table, prompting block, summary line), README (feature prose, mode table,
agent prose, mermaid EXTERNAL node, conditional legend, optional-MCP
requirement, --local section), CLAUDE.md, settings.json (the whole
trekresearch.geminiBridge block), both templates, architecture.md,
command-modes.md.
trekplan high-effort Adversarial Pass 2 now degrades EXPLICITLY: it emits its
section with status "unavailable, skipped" instead of failing or vanishing. A
high-effort plan carrying no Pass 2 marker is indistinguishable from one whose
Pass 2 crashed, which is the failure mode this wording exists to prevent.
gemini_used is deliberately KEPT as a vestigial trekresearch stats field pinned
to false. Removing it would break the observability export schema for existing
consumers, and the directive was about the agent, not the field.
Not touched: CHANGELOG history and the measurement/decision docs keep their
gemini references. They record what a past version did or what was measured
then; rewriting them is the same defect class as bumping a version string
inside a measurement doc.
Driven test-first. Five new pins in tests/lib/doc-consistency.test.mjs, verified
RED before the edits, including a KNOWN-POSITIVE CONTROL asserting the
historical records still DO carry gemini references — so the empty result on
live surfaces is a measurement and not a broken query (Verifiseringsloven
ansikt 4). Agent inventory 24 -> 23 (20 spawnable + 3 orchestrator reference
docs); the <example>-block floor moves 34 -> 32 because an agent legitimately
left the inventory, not because examples went missing from a surviving one.
Docs: docs/T1-cc26-delegated-orchestration.md §8 item 3 claimed both the §6
synthesis-agent PoC and the §5 bake-off were "designed but unbuilt". That was
written in S7 and falsified the same afternoon by S12, which ran the §6 PoC and
recorded Δ main-context (faithful flow) = 0.0%, NEGATIVE. The stale wording is
what caused the settled PoC to be re-ordered as new work on 2026-09-02, so it is
struck rather than deleted and §6 gained a RUN AND DECLINED status block. The
finding is structural, not stochastic: Phase 5 spawns the exploration swarm
foreground (trekplan.md:158,338-341), so the outputs are already resident in
main before Phase 7 — delegating only the Phase-7 digest evicts nothing.
Also measured 2026-09-03 (CC 2.1.259): claude -p --output-format stream-json
runs on subscription auth with no ANTHROPIC_API_KEY and now emits a
subagent_stats block, so S12's environment-block premise is half stale. Recorded
in §8 item 4. It lowers the cost of §5; it changes nothing about §6.
Suite 1041 (1039/0/2), up from 1036 by exactly the five tests added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up defect in the reviewer accounting added by e2aec01, found by review
and confirmed by probe before fixing.
validateFindings only WARNS on a missing `reviewer` field, so a payload can
fail schema while carrying no name. ingest then records `reviewer: null`, and
runContract turned that null into the literal reviewer name "unnamed reviewer".
MEASURED before the fix:
runContract([{findings:[{file:'x.mjs',line:1,rule_key:'NOPE',severity:'MAJOR'}]}],
{expectedReviewers:['code-correctness-reviewer']})
-> missing_reviewers = ["unnamed reviewer", "code-correctness-reviewer"]
One failure, two entries, one of them an agent nobody launched. The
`reported.delete(s.reviewer)` line was also inert for that case, since a null
name was never in the set to begin with.
Fix: skipped payloads are split by whether they carry a name. Named ones go to
missing_reviewers as before; anonymous ones increment the new
`unattributable_payloads` count, which forbids ALLOW on its own - so stripping
a reviewer name from a payload cannot restore ALLOW, and the floor does not
depend on the caller passing expectedReviewers. `allow_blocked_by` reports the
two facts separately: `missing-reviewer:<name>` and `unattributable-payload (n)`.
The old behaviour never produced a false ALLOW - it failed in the safe
direction - but it named a reviewer that did not exist, which is the kind of
output an operator would chase.
Iron Law: two failing tests first (double entry; anonymous-payload-alone must
forbid ALLOW), then the fix.
Also verified in this pass, by temporarily adding a fake reason to
UNVERIFIED_REASONS: the prose-vocabulary pin does go red when a reason is
declared in the lib but missing from agents/review-coordinator.md. A pin that
cannot fail is not a pin.
Suite 1034 (1032/0/2) -> 1036 (1034/0/2), 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
computeVerdict counted only the findings handed to it (reasoned.kept), so a
finding removed by Pass 2 or Pass 3, and a reviewer whose payload was thrown
away or never arrived, were arithmetically identical to a finding that never
existed. All three pushed the verdict toward ALLOW.
Measured before the fix (probes, 2026-09-01):
- a BLOCKER with a 101-character title -> ALLOW (Pass 2 succinctness)
- a payload with one ad-hoc rule_key is skipped WHOLE at ingest, taking a
valid BLOCKER sibling with it -> ALLOW
- a reviewer that never reported -> ALLOW
Pass 3's own no-citation / unknown-rule_key branches turned out unreachable
through runContract (validateFindings rejects those payloads first), so the
reachable exposure was Pass 2 plus the skipped/absent reviewer.
THE OPEN DESIGN DECISION, and why it went against the order's default.
The order proposed: indeterminate file-existence YES, plain succinctness NO
("a too-long finding is not an uncertain finding"). I kept the first and
overrode the second, on one principle:
A removal is `dropped` only when the test REFUTED the finding as a claim
about this codebase. Every other removal is `unverified`.
Succinctness and actionability read a `.length`. They never examine the claim,
so they cannot establish the finding is unreal - and dropping a BLOCKER for a
101-character title is precisely the fail-open shape being fixed. Three things
settled it:
1. Under the order's default the fix would have been almost inert. Pass 3's
drop branches are unreachable via runContract, so leaving Pass 2 out would
have left the only reachable finding-level exposure open.
2. Cost asymmetry, priced rather than asserted: the verdict is not a gate.
Handover 6 feeds `findings` filtered to BLOCKER+MAJOR into /trekplan
(commands/trekplan.md:218); `verdict` is optional metadata
(docs/HANDOVER-CONTRACTS.md:353). Nothing loops or re-plans on WARN. So a
false `unverified` costs WARN plus a printed reason; a false drop costs a
silent ALLOW over a live BLOCKER.
3. unknown-rule_key joins them for the same reason: an ad-hoc key is a real
defect wearing the wrong label, and v5.1.1 high-effort mode already KEEPS
those, normalised to PLAN_EXECUTE_DRIFT. Refuting them at normal effort
while keeping them at high effort would be incoherent.
no-citation stays a drop: a finding whose file is empty or whose line is
negative names no location, so it makes no checkable claim at all - the one
deterministic refutation, and what the Pass 3 prose already said it was.
Iron Law: tests/lib/coordinator-contract.test.mjs first, red (missing export +
the three measured ALLOWs), then production code. Two existing assertions were
updated AFTER implementation as contract changes, not to make the red pass.
A known-positive control pins that ALLOW is still reachable - without it,
"no ALLOW" is not a fail-closed contract, only a broken one.
lib/review/coordinator-contract.mjs
+ classifySuppression / REFUTING_REASONS / UNVERIFIED_REASONS - one
vocabulary owned by the lib, including the tokens only the LLM
coordinator emits (accuracy:refuted, file-existence:refuted/indeterminate),
so prose and lib cannot drift. Unclassified reasons default to unverified:
the default fails closed.
~ judgeFilter / reasonablenessFilter return {kept, dropped, unverified}
~ computeVerdict(findings, {unverified, missingReviewers}) -> + allow_blocked_by.
Never raises a verdict, only withholds ALLOW. Unverified findings are NOT
counted into a severity tier: their severity was never substantiated, and
counting it would be invention.
~ runContract(payloads, {expectedReviewers}) -> + unverified,
missing_reviewers, allow_blocked_by. `suppressed` stays the union of
dropped + unverified, so existing consumers (gold-eval) keep their meaning.
agents/review-coordinator.md - Pass 2/3 tables gain a fate column, new
"Suppression is two-valued" section, Pass 4 threshold table gains the two
fail-closed rows, Executive Summary must state a withheld ALLOW, Suppressed
Findings tags each line [dropped]/[unverified]. Pass 3's unknown-rule_key
bullet explicitly says high-effort does not reach that branch, so the same
input never has two documented fates.
commands/trekreview.md - Phase 5 "Reviewer accounting": the expected set is
written down before the spawn, a silent reviewer gets one re-ask and then
STOP. That extends the pattern already in the file (schema failure -> 2
bounded re-asks -> "do not feed unvalidated findings to the coordinator") to
the other two ways a reviewer goes missing, rather than softening it to WARN.
The lib's missing_reviewers stays as belt-and-braces for direct callers.
docs/agent-return-channel-defect.md - the "inferred, not observed" caveat on
the unnamed arm above 66 lines is struck: akashic-intelligence S27
(f168630) measured 2/2 unnamed agents returning against a 4370-line plan,
30449 B and 10989 B, both valid JSON. Recorded with akashic's own two
caveats intact - the measurer owns the finding, and byte-identity between
the returned string and the file on disk was not proven. The separate S25
named-arm figures are left standing; these are two measurements, not a
correction of one by the other.
No release, no version bump, no tag, no catalogue ref, no Workflow port.
Suite 1025 (1023/0/2) -> 1034 (1032/0/2), 0 failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
D-census (order 6242020304, from .claude, 2026-08-20/21) measured that
registry.npmjs.org/voyage answers 200 with someone else's package: "Advanced
HTTP Routing System for Node.js" (NEURS/voyage, v0.0.1, 2022). This manifest
declared `"name": "voyage"` with no `private` flag, so nothing stopped an
accidental `npm publish` against a name we do not own - and publishing is
one-way.
This manifest is test tooling for a Claude Code plugin distributed through the
marketplace catalogue (`ref: vX.Y.Z`), never through npm. The sibling
non-package manifests (okr, repo-mailbox, repo-standard) all carry
`private: true`; voyage was the exception.
Operator decision S93: option A (set `private: true`), not option B (scope the
name to `@<org>/voyage` for a future publish) - there is no publishing plan,
so B would only defer the same choice.
Tests first (Iron Law): tests/lib/doc-consistency.test.mjs pins
`pkg.private === true`, seen red (actual `undefined`) before the manifest
change.
New baseline: 1025 tests, 1023 pass / 0 fail / 2 skip (was 1024: 1022/0/2).
No `npm publish` in any form, no version bump, no tag, no catalogue change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude Code 2.1.218 changed /deep-research to start only when invoked
manually; from there the Skill tool refuses a model invocation outright
with `disable-model-invocation` (measured in a real `--engine deep-research`
run 2026-09-01 18:32Z - the SC3 fallback to swarm held). The engine was
built against 2.1.196 and the prose still promised a path Claude Code has
removed ("requires Claude Code 2.1.154+").
The flag is KEPT as an additive opt-in that never hard-fails. Prose only:
no lib/ change, no adapter-contract change, no default change, no version
bump.
- commands/trekresearch.md: flag bullet states the closed window
`2.1.154 <= CC < 2.1.218`; the pre-gate gained an UPPER ceiling (still a
numeric comparison, not a string one) that skips straight to the fallback
with reason `disable-model-invocation`; step 3 names the Skill-tool
refusal as a known failure manifestation; step 4 lists the reason tokens.
- docs/command-modes.md + README.md: same window, one sentence each, on the
reference row itself.
- CHANGELOG.md: Unreleased/Docs entry.
Tests first (Iron Law): tests/lib/doc-consistency.test.mjs gained a pin
requiring 2.1.218 + disable-model-invocation on all three surfaces, on both
reference rows, and in both load-bearing regions of the command (flag
bullet + pre-gate). Seen red on all three surfaces before the prose change.
tests/commands/trekresearch-engine.test.mjs unchanged and green.
New baseline: 1024 tests, 1022 pass / 0 fail / 2 skip (was 1023: 1021/0/2).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
akashic-intelligence lost /trekplan Phase 9 twice to reviewer agents that
never returned. Reproduced here and measured: the cause is the Agent tool's
`name` parameter, not the agent definitions.
Passing `name` does not label a subagent - it changes its kind. The spawn is
recorded as taskKind "in_process_teammate" (spawnDepth 0) instead of a real
subagent (spawnDepth 1). A teammate's final assistant text is not a return
value; it reaches the parent only if the teammate itself calls
SendMessage(to: "main"). plan-critic and scope-guardian declare
tools: [Read, Glob, Grep] - no SendMessage - so as teammates they are
structurally incapable of returning, whatever the prompt says.
Denominators: named 0/5 returned; named + explicit SendMessage 1/1; unnamed
3/3 (plan-critic and scope-guardian each returned full findings + the JSON
block in ~110s). Model override 2/2 non-returning, so the override is not
the variable. All 5 named agents produced correct final text in their
transcripts - only delivery failed, and that output is recoverable on disk.
The defect is a harness behaviour, so it is documented rather than silently
worked around: docs/agent-return-channel-defect.md carries the mechanism,
every denominator, the two broken queries that nearly became facts, the
unmeasured cells, and a recommended working shape for consumers (use agents,
drop `name`; do not fall back to inline review, which costs the dedup step).
Prevention is pinned, TDD red->green: the four spawning commands each state
the rule and name the mechanism, and doc-consistency derives the spawning set
from the command files so the pin cannot go vacuous.
Suite 1013 (1011/0/2), +5 from baseline 1008.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014MfB5Ecp8vvGyou8uFGxBV
The Phase 4.5 amendment to the Independence hard rule crosses that rule
deliberately, and the crossing is defensible: bounded to Phase 4.5 and the
Phase 5 loop, the initial swarm stays blind, and it resolves a tension the
brief created itself by asking for exactly this mining of Phase-4 output.
The defect was the compensating-control claim. query-privacy-gate.mjs was
named as THE control for the crossing, but it inspects outbound query
CONTENT for paths, repo identifiers and secret-shaped strings. It cannot
prevent a local finding from steering an external agent's question. It
compensates the EGRESS risk; the BIAS risk was left with no control while
the text read as though it had one.
Both risks are now stated separately with the control that actually bears on
each:
Bias - structural, not a gate. The initial external swarm stays blind, so
an independent baseline exists BEFORE anything crosses; the
crossing only ADDS to that baseline; and at effort: high - the only
effort at which any of this runs - contrarian-researcher is forced
always-on, so the brief always carries an adversarial
counter-evidence pass over the result the crossed queries fed.
Egress - query-privacy-gate.mjs, unchanged, with its non-overridable
hard-block tier.
Fixed at both sites: the Hard rules entry and the Phase 4.5 paragraph that
repeated the same attribution.
A doc-consistency pin now fails if the amendment names the privacy gate as
the compensating control again, and requires it to name both
contrarian-researcher and the egress framing - so the two cannot quietly
re-merge.
Review finding 4888ae847240142933469d70bafb6a9c23b7b2eb (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
The bounded-cost NFR asks for explicit ceilings on BOTH axes - max
conversation turns and max discovered dimensions. The turn axis got
MAX_CONV_TURNS, a ledger-backed reader and a PreToolUse enforcer. The
discovery axis got one sentence in Phase 4.5 prose ("append candidates only
while the whole list stays at or below maxDimensions: 8") with no constant
of its own, no reader, and no test that a run exceeding it is caught. That
is the brief_reviewer_iter_cap shape the operator decision warned about: a
cap nothing reads.
checkDimensionCeiling() is the reader, exposed on the CLI as
--check-dimensions N (exit 0 within, exit 1 rejected), and Phase 4.5 step 3
now calls it once the final list is settled instead of merely describing the
bound.
Three deliberate choices:
- The ceiling IS MAX_TOTAL_DIMENSIONS, the constant that sizes the turn
budget. Both axes read one settings.json:16 value, so they cannot end up
enforcing different numbers - a second constant is how that drift starts.
- An unreadable count is REJECTED ('abc', null, undefined, {}, -1, NaN,
non-integers). A cost ceiling that waves through what it cannot measure is
not a ceiling.
- --check-dimensions requires no run id, effort or VOYAGE_STORM_ENABLED.
Phase 4.5 never calls the budget gate - that is why its skip-guard reads
the flag directly - so the ceiling check must not inherit the gate's
preconditions.
The mitigation the review already verified still holds and is unchanged:
MAX_TOTAL_DIMENSIONS bounds actual retrieval cost regardless of how many
dimensions discovery appends. What was missing was anything that FAILS on a
list over the bound, and now a run over it is rejected by exit code.
Review finding 96a3ee51152dfe72aca703f771843f2f3639e7b6 (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
allowTurn() appends BEFORE the turn runs, so during granted turn N the
ledger holds N records. The hook denied at `used >= budget`, which blocked
every tool call of the FINAL granted turn: the primitive granted B turns
and the harness permitted B-1. Worse, an exhausted run therefore always
terminated through an exit-2 tool denial instead of the graceful "cap
exhausted" exit at commands/trekresearch.md - and the prose says in as
many words that exit 2 is not exit 1, so the model was pushed out through
the one exit it is told NOT to treat as a cap.
The review recommended denying at `used > budget`. Taken alone that fixes
the count and breaks the hook: once the O_EXCL claim (previous commit)
makes a breached ledger impossible, `granted > budget` can no longer fire,
and the case this hook exists for - the loop consults the gate, is denied,
and issues the tool call anyway - would be allowed. A deny branch that
cannot be reached is a dead security claim, which is the same thing S82
removed two of rather than leave standing.
So the denial itself became a record. allowTurn() appends a tombstone
{runId, exhausted: true} when it denies for budget, and the hook denies on
the tombstone. Both properties now hold at once:
granted == budget, no tombstone -> turn B is in flight -> ALLOW
tombstone present -> the gate already said no -> DENY
granted > budget -> breached, any cause -> DENY
A tombstone is not a turn: readLedger reports {granted, exhausted}
separately so it can never consume budget. allowTurn short-circuits on an
existing tombstone, so a hammered gate neither re-walks every slot nor
grows the ledger. The tombstone write is best effort on purpose - the
denial is already the correct answer, so a ledger that cannot take the
record must not turn a denial into a grant.
The parallel-boundary test now asserts GRANTED turns rather than raw
ledger lines, because the denied callers legitimately add tombstones.
Review finding 8eb53458ac3efec778094f9f03b09e1cc1077a09 (MINOR).
Operator decision: tombstone over the literal recommended_action.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
countTurns-then-appendFileSync is read-then-write. N callers that all
observe used == budget-1 all decide to grant, and the bound is exceeded
by N-1. The comment above allowTurn asserted "Append-only: never
read-modify-write" and named the concurrent case - Phase 4.5/5 may spawn
several agents in a single message - as the reason it had to be. The
decision path was exactly what the comment denied, so the concurrency
claim had nothing under it.
Each grant now creates <data root>/trekresearch-loop-claims/<runId>-<slot>.claim
with flag 'wx' (O_CREAT|O_EXCL) before appending. The kernel picks the
winner per slot, slot numbers are bounded by the budget, and each can be
created exactly once - so total grants for a run cannot exceed the budget
however many callers arrive together. The ledger count now only says
where to start looking for a free slot.
Two of the three tests are deterministic and do not race anything: they
assert the invariant directly by pre-creating claims, including the state
a mid-append competitor leaves behind (ledger 7, slots 1-8 claimed, budget
8 -> deny). That matters because the third test - six real concurrent shim
processes at the boundary - passed even BEFORE the fix, since process
startup jitter serialised them. A race test that passes by luck is not
evidence, so it ships as a real-world regression guard next to the two
that are.
Stated rather than left to be discovered: claim files are empty, at most
budget per run, and never cleaned - the same standing as the ledger, which
also grows for the life of the data root. Reusing a runId across runs, or
two runIds colliding after filename sanitisation, both deny a turn, which
is the safe direction for a budget control.
Review finding 3994491ef1fdba6e0e3645b5b713cbdbdeb2b328 (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
An unreadable ledger returned 0 from countTurns in BOTH the primitive and
the hook, so a run whose ledger existed but could not be read (EISDIR,
EACCES, EIO) was handed the full budget again on every call - unbounded.
research-loop-cap.mjs argues against exactly that three lines above the
code that did it, and its missing-DIRECTORY case already failed closed.
The unreadable-FILE case now agrees with it.
Only ENOENT still counts as zero turns spent: that is the legitimate
first-turn state, and the reason this cannot just throw on any read
failure.
The hook no longer carries its own countTurns. It imports the primitive's
exported readLedger(), the same way it already resolves the data root
through resolveDataRoot() - a reader and a writer with private copies of
the counting rule is how a hook ends up enforcing a different bound than
the gate it backs. In scope + cannot count now exits 2 with a message
that says counting failed, not that the budget is spent.
Fail-closed stays scoped to the loop: a test pins that an unreadable
ledger in an OUT-of-scope session still exits 0, because a PreToolUse
hook that over-blocks bricks every session on the box.
Also dropped the existsSync pre-check before the read - readFileSync's
own ENOENT carries the same information without a second syscall that
can disagree with the read that follows it.
Review finding 5e1c6230f48ead38fa77cd8f4b06bfdc2b5b7bbf (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
A fractional TREKRESEARCH_MAX_CONV_TURNS below 1 cleared the `n <= 0`
guard on its raw value and only then floored, so '0.5' and '0.9' became
0 and the budget became 0 x MAX_TOTAL_DIMENSIONS = 0: every turn denied,
the loop silently dead rather than bounded.
README.md:229 and docs/architecture.md:15 both promise that invalid
values fall back to 3. docs/command-modes.md:42 enumerated "empty,
non-numeric, zero, or negative" and happened to sidestep the case; the
enumeration is now exhaustive about it.
Measured before: '0.5' -> 0, '0.9' -> 0, '2.7' -> 2, '' / 'abc' / '-2'
/ '0' -> 3. Measured after: '0.5' -> 3, '0.9' -> 3, '2.7' -> 2, and
'Infinity' -> 3 (it is not a cap either).
A cap of 0 is not a narrower cap, it is an off switch. The tests pin
both directions: the fraction falls back, and allowTurn cannot report a
budget of 0 under it.
Review finding fc516799e6042e246a4b62d81903ac27c2efab84 (MINOR).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
CLAUDE_PLUGIN_DATA is empty in the Bash tool's process env, and the Phase 5
bash snippet is the cap's only caller. resolveLedgerPath() returned null there
and allowTurn() failed closed, so the budget gate denied turn 1 of every real
run: the loop this delivery exists to bound could never spend a turn, and the
pre-registered measurement could not be run at all.
resolveDataRoot() is now the single root for everything the loop writes --
CLAUDE_PLUGIN_DATA when the harness sets it, ~/.claude/voyage when it does
not. Three consumers resolve through it, which is the point: the cap ledger,
the PreToolUse hook's scope-marker lookup, and the command's bash snippets.
A writer and a reader that resolved the root separately are what made the
enforcement hook allow unconditionally in every real run while CLAUDE.md and
docs/architecture.md called it enforcing.
Same root cause, same commit:
- Marker write and remove now share ONE absolute-path guard and one root; the
write requires a non-empty CLAUDE_CODE_SESSION_ID before composing the path
(unset, the marker was named `.json`, which no lookup matches and no TTL
sweep cleans up).
- The per-turn gates resolve VOYAGE_ROOT with a plugin-cache fallback and
reserve exit 2 for "gate could not run". Interpolating an empty
${CLAUDE_PLUGIN_ROOT} ran `node /lib/...` -> exit 1, which the contract read
as "privacy gate says no" -- an unsatisfiable rewrite loop no query could
clear.
Two now-unreachable deny branches are removed rather than left as dead safety
claims (allowTurn's no_plugin_data_dir; the hook's uncountable-ledger deny).
The fail-closed stance stays where it is still real: a ledger that cannot be
WRITTEN denies the turn.
Verified end-to-end through the real bash snippets and the real hook with both
variables stripped and HOME sandboxed: marker written under the fallback root,
8 turns spent, 9th denied, hook exits 2, and exits 0 again after removal.
Note: the fallback exit-2 branch fires against the installed v5.9.1 cache,
which predates lib/util/research-loop-cap.mjs -- correct behaviour, and it
clears when the plugin is reinstalled.
Review findings 2670c10a, fbd6d534, 93550dfb.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vPSXe88qp5aqWUqbDNWoF
CLAUDE.md claimed both STORM phases go inert when the flag is unset. Only
Phase 5 did: the flag check lives in research-loop-cap.mjs, and Phase 4.5
never invokes the cap — it was gated on effort: high alone. At high effort
with the flag unset, discovery still mined Phase-4 output and mutated the
dimension list, so `dimensions` diverged from `dimensions_baseline` and the
decline branch was unreachable for half the mechanism.
The code was the deviator, so the guard is fixed rather than the claim: the
Phase 4.5 skip-guard now names both conditions, with the reason inline.
Three surfaces scoped the flag to "the loop" and are corrected with it
(README, docs/command-modes, docs/architecture), plus the orchestrator phase
map. CLAUDE.md's claim is now true, but its stated MECHANISM was not — Phase
4.5's inertness comes from its own guard, not from the cap module — so that
sentence is corrected too.
New doc-consistency pin: the flag must be documented as gating both phases on
all four reference surfaces. Review finding 00a3af1a.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vPSXe88qp5aqWUqbDNWoF
pre-agent-cap.mjs (S78) enforces the Phase 5 loop bound only while a scope
marker exists for the calling session. Nothing wrote that marker, so the hook
shipped correct but latent. Phase 5 now writes it at loop start and removes it
on all three exits.
- Write: ${CLAUDE_PLUGIN_DATA}/trekresearch-loop-scope/<session_id>.json with
{runId, startedAt}, keyed by CLAUDE_CODE_SESSION_ID. Verified 2026-08-12 that
this equals the session_id on the hook's PreToolUse payload.
- runId must be the same --run-id the ledger is counted under; a mismatched id
counts zero turns and enforces nothing.
- Fail-soft on write: the hook is defence in depth, research-loop-cap.mjs stays
the gate. Report and continue. The reverse (skipping the budget gate because
a marker exists) stays forbidden.
- Removal on every exit, load-bearing on the exhausted one: the hook keeps
denying WebSearch/WebFetch/Task while the marker is there, and Phase 6 spawns
agents. Crash is covered by the hook TTL, not by cleanup - stated as such
rather than claiming cleanup covers it.
- Marker written in Phase 5, not Phase 4.5: 4.5 mines already-retrieved Phase-4
results and spends no loop turns, so scoping there widens the window for
nothing. Pinned by a test.
Six pins in tests/lib/doc-consistency.test.mjs derive the directory name from
SCOPE_DIRNAME in the hook and the payload fields from marker.runId/startedAt,
so drift in either direction fails. hooks/scripts/pre-agent-cap.mjs untouched.
Verified end-to-end with the snippets as shipped: marker written -> hook allows
under budget, denies 8/8 at budget, allows again after removal; removal is
idempotent; unset CLAUDE_PLUGIN_DATA takes the fail-soft branch.
Suite 937 (935/0/2, baseline 931 + 6).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R77nGJjZ1hqjAQQHefFdnc
Scores committed agent-run fixtures against the golden corpus at
(file, rule_key) granularity, building on the deterministic coordinator
contract (4a). Offline: committed reviewer payloads, no live agent spawn,
no LLM, no network (the LLM-in-the-loop grading is the separate 4c tier).
- lib/review/gold-scorer.mjs: scoreFindings (precision/recall/f1 at
(file,rule_key) granularity, line+severity ignored) + scoreVerdict; pure,
with documented vacuous-set conventions.
- tests/fixtures/bakeoff-rich/runs/run-perfect.json: committed run that
reproduces all 5 seeded gold findings through runContract.
- tests/lib/gold-eval.test.mjs: the scoring RUN (precision/recall/f1 = 1.0,
verdict == expected_verdict BLOCK, nothing suppressed/skipped).
- lib/util/test-census.mjs: third census category (goldEval) — a scoring run
is neither behavior coverage nor a doc-pin; honest-count invariant now 3-way.
- docs/eval-corpus/README.md: 4b moved from Future hardening to implemented.
Suite 809 -> 822 (820/0/2). gold-scorer covers TP+FP+FN+degenerate paths.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJQYC5vpkJWxndS55vQQZ6
F1 cache-analyzer regression guard (SC5): pin percentile (wall_time_ms_p50/p90)
and time-range (oldest/newest_event_iso) — 2 of 3 'unchanged' categories were
previously un-asserted on the mixed-input fixture.
F2 lastMainChainModel: add direct coverage — last-wins across 2 distinct
main-chain models, sidechain exclusion (even when the sidechain is the last
record), and model-absent → null propagating to deriveCost refuse-to-estimate.
804 -> 807 tests (805 pass / 0 fail / 2 skipped). No test pins the test count.
The economy profile's cross-tier Jaccard floor (0.55) rests on parked-synthetic
fixtures; empirical Step-17 calibration is v4.2-budget-gated ($60–120,
unauthorized). Fork resolved as label-not-calibrate: the experimental status,
previously prose-only in docs/profiles.md, now lives in the profile data and is
machine-checked. No new user-facing capability — honest labeling + a guard.
- lib/profiles/economy.yaml: add `experimental: true` (with rationale comment).
- lib/validators/profile-validator.mjs: recognize `experimental` as an OPTIONAL
boolean; non-boolean → PROFILE_INVALID_ENUM. Absent ⇒ tier is stable
(premium/balanced unaffected, profile_version stays 1.0 — additive).
- README.md + docs/operations.md + docs/profiles.md: flag the `economy` table
row "⚠ Experimental (uncalibrated Jaccard floor)".
- tests/synthetic/profile-jaccard-calibration.md + analysis §6/§10 + backlog
plan §S34: cross-reference the marker; mark V30/S34 done.
+5 tests (739 → 744, 742/2/0): economy declares experimental:true; premium and
balanced do not; validator rejects non-boolean experimental; every profile-doc
economy row is flagged; the flag tracks the calibration's parked-synthetic
status (must drop in the same change that lands real calibration).
Closes the balance backlog (4/4, S31–S34). claude plugin validate green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Balance-backlog S33 (DOC, non-breaking). The D1–D3 forks resolved conservatively
(balance-backlog-plan.md), collapsing the model + observability work into a
documentation record. Three deliverables, all doc-only, no code/frontmatter change:
- V35 (doc half): the "24 agents" headline is reconciled to its honest split —
21 spawnable (one dormant: synthesis-agent, Δ≈0) + 3 orchestrator reference
docs (planning-/research-/review-orchestrator document the inline /trek*
workflow; not spawned). Each orchestrator header now declares itself a
"reference document, not a spawnable capability". README + CLAUDE.md state the
split; counts in the new pins are DERIVED from agents/ so they survive reword.
- D2 (V32 rationale): docs/observability.md gains a "Why direct export rather
than a native collector" section — direct export keeps the path / SSRF /
field-allowlist guards in audited in-process code (the S21 hardening) instead
of re-hosting a collector; textfile mode remains the collector escape hatch.
- D3 (kept-opus): docs/voyage-vs-cc-balance-analysis.md §10 decision record —
opus on V09 (glue), V35 (dormant), V11 (retrieval), V16 (mechanical), V08
(researchers) was reconsidered for sonnet and KEPT (pin 40d8742 firm).
No agent frontmatter changed — tests/lib/agent-frontmatter.test.mjs is the
structural model source-of-truth and is untouched (diff is description-only:
model: opus + tools lists unchanged, no Agent tool granted). No Handover-1
change; no exporter/gemini-bridge removal. Non-breaking, no version bump.
tests/lib/doc-consistency.test.mjs: +5 S33 pins (inventory split derived from
agents/; synthesis-agent dormant; orchestrator relabel; observability D2
rationale; analysis-doc D1–D3 record). Tests 739 (737 pass / 2 skip / 0 fail),
bar `node --test`; `claude plugin validate` green (1 accepted warning).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
Balance-backlog S32 (CODE+AUDIT). Audited whether the brief interview and the
research / exploration / reviewer swarms ride NATIVE Claude Code primitives or
hand-roll their own engine. Finding: all five ALREADY delegate natively — no
re-implementation, so no command-file change. Documented the principle + a
standing regression guard instead.
- V01 (trekbrief Phase 3): Q&A turn-taking is `AskUserQuestion` (line 144 / step
4); the "selection rule" is section-selection POLICY, not a hand-rolled menu.
- V07 (research interview): `AskUserQuestion`, one-at-a-time.
- V08/V11/V24 (research / exploration / reviewer swarms): parallel `Agent`
fan-out in a single message ("in parallel … single message" / "via the Agent
tool — one message, multiple tool calls"). Policy layers (dimensions/schemas/
triangulation, typed roles/effort/scaling, 12-key rule catalogue/no-cross-feed/
dedup) cleanly separated from the engine.
- docs/architecture.md: new cross-cutting principle note "delegate the engine,
keep the policy" recording the native primitives, the per-command policy, and
the audit verdict.
- tests/lib/doc-consistency.test.mjs: +3 S32 pins (architecture note present;
each swarm command lists Agent + mandates single-message parallel spawn;
trekbrief Phase 3 delegates to AskUserQuestion). Guards engine creep-back.
No command-file edits (all native). No model/frontmatter change (D3 firm). No
Handover-1 change. Non-breaking. Tests 734 (732 pass / 2 skip / 0 fail), bar
`node --test`; `claude plugin validate` green (1 accepted warning).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
