fix(docs): the put-back, the research triggers and Pass 3's high-effort skip say what the code does

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-24 21:19:58 +02:00
commit f6bdc2de59
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
6 changed files with 143 additions and 15 deletions

View file

@ -95,6 +95,9 @@ Fix round 4.5a (decisions 57, 67 and 68): the round's 30 findings, where a Voyag
**Stats**
- **No command promises a stats key its record does not write** (finding #46). The "Stats records emit `profile` …" sentences left `/trekbrief`, `/trekplan`, `/trekexecute`, `/trekreview` and `/trekcontinue`; `/trekresearch` keeps only `engine`, which its record writes; `docs/profiles.md`, `docs/operations.md` and README no longer say a profile reaches the stats records.
**Across the round**
- **The texts next to the fixes say the same** (siblings found during the round): `/trekreview`, the review reference and the Arm B coordinator prompt no longer drop a finding whose rule key is unknown; `/trekbrief` writes its stats record on every exit of its manual and auto paths, says that `--gates` adds one pause before its auto chain, and that the chain ends at the plan; `/trekexecute` says `--gates` changes nothing there, checks the wave marker before a low-effort single session, and names no validator check that does not exist; plan mode goes through the plan approval before it prints the `/trekexecute` line; the gates shim reads `--gates=true` as on.
### Known issues

View file

@ -804,7 +804,7 @@ flowchart TB
end
subgraph RES["/trekresearch · Phase 4 — parallel"]
RL["LOCAL: architecture-mapper · dependency-tracer<br/>task-finder · git-historian · convention-scanner*"]
RE["EXTERNAL: docs-researcher · community-researcher<br/>security-researcher* · contrarian-researcher*"]
RE["EXTERNAL: docs-researcher · community-researcher<br/>security-researcher · contrarian-researcher*"]
end
subgraph PL["/trekplan"]
PLG["Phase 4b · brief-reviewer (gate)"]
@ -823,7 +823,7 @@ flowchart TB
BR --> RES --> PL --> EX --> RV
```
`* = conditional`: convention-scanner / test-strategist on medium+ codebases (50+ files); research-scout for unknown external tech; security-researcher when a leading recommendation forms (or always at `effort=high`); contrarian-researcher, when selected, runs in wave 2 after the other external agents return, with their leading answer to attack (always at `effort=high`); brief-conformance-reviewer skipped under `--quick`.
`* = conditional`: convention-scanner / test-strategist on medium+ codebases (50+ files); research-scout for unknown external tech; contrarian-researcher, when selected, runs in wave 2 after wave 1 returns, with the leading answer of wave 1's external findings to attack (with `--engine deep-research`, after the engine returns, from the leading answer of its report), always at `effort=high`; brief-conformance-reviewer skipped under `--quick`.
> **Which Claude Code primitive each phase uses — and the alternatives considered (Workflow substrate, delegated orchestrator, dormant synthesis-agent)** → see [docs/architecture.md §Primitives per step](docs/architecture.md#primitives-per-step-decision-matrix).

View file

@ -191,6 +191,10 @@ The coordinator runs a 4-pass process:
`RULE_CATALOGUE` (`unknown-rule_key`) or whose file cannot be checked
(`file-existence:indeterminate`) to `unverified`, which forbids `ALLOW`;
reset a severity that does not match its catalogue tier.
Skipped at high effort, as in `quick` mode (below); at high effort an
unknown `rule_key` is normalized to `PLAN_EXECUTE_DRIFT` instead
(`agents/review-coordinator.md`, "High-effort normalization"); the
brief-conflict rule still applies in every mode.
4. **Compute verdict** — `BLOCK` if `BLOCKER ≥ 1`, `WARN` if `MAJOR ≥ 1`,
else `ALLOW`.
@ -199,8 +203,8 @@ The coordinator's output is the full review.md content — frontmatter
and `## Remediation Steps` with one numbered step per BLOCKER) + trailing
JSON block — ready to write.
In `quick` mode, skip pass 3 (reasonableness filter). Passes 1, 2, 4
still run.
In `quick` mode, skip pass 3 (reasonableness filter), except the
brief-conflict rule, which applies in every mode. Passes 1, 2, 4 still run.
### Phase 7 — Write review.md

View file

@ -516,6 +516,9 @@ The coordinator runs the 4-pass process documented in
`rule_key` is not in `RULE_CATALOGUE` (`unknown-rule_key`) or whose file
cannot be checked (`file-existence:indeterminate`) to `unverified`, which
forbids `ALLOW`; reset a severity that does not match its catalogue tier.
Skipped at high effort too, where an unknown `rule_key` is normalized to
`PLAN_EXECUTE_DRIFT` instead (`### High-effort behavior (v5.1.1)` below);
the brief-conflict rule still applies in every mode.
4. **Verdict** — BLOCK / WARN / ALLOW per the threshold table.
**Fail-closed.** Every removal in Pass 2 and Pass 3 is either
@ -563,11 +566,8 @@ Workflow({
A reviewer that returns nothing, an engine listed in `engineNotRun`, or a coordinator that returns no verdict and findings forbids `ALLOW` on this path too (`missing_reviewers` in the result), and the result's `allow_blocked_by` names each one (`missing-reviewer:<name>`, the coordinator as `missing-reviewer:review-coordinator`);
Phase 7 writes that list as the trailing JSON block's `allow_blocked_by`, so the verdict gate reads such a WARN review as not performed in full (exit 2; a BLOCK stays exit 1).
On this path dedup ranks as Pass 1 does (both paths), every engine finding the
coordinator did not return at its exact `(file, line, rule_key)` is put back at its own
severity (operator decision 23), and a returned finding is never below the lower of its
reviewer counterpart's severity and its catalogue tier (floor (b)), so the workflow path
is stricter than the prose path.
On this path dedup ranks as Pass 1 does (both paths), and after the coordinator each finding is matched to its own row, its `(file, line, rule_key)` triplet plus its title, trimmed and in lower case: of the engine findings in a row no returned finding is on, the one with the highest severity (on a tie, the first seen) is put back at its own severity (operator decision 23), and a returned finding is never below the lower of its reviewer counterpart's severity and its catalogue tier (floor (b)); its counterpart is its own row, else the strictest row on its triplet.
So the workflow path is stricter than the prose path.
Contract (verified in S10 part B — follow exactly):
- **Pass `args` as a JSON object.** The script defensively re-parses a JSON *string*, but
@ -903,8 +903,8 @@ readable.
### High-effort behavior (v5.1.1)
When `phase_signal_result.effort == 'high'` for the `review` phase,
skip Pass 3 (Cloudflare reasonableness filter) in
`agents/review-coordinator.md`. Passes 1, 2, and 4 still run.
skip Pass 3 (Cloudflare reasonableness filter) in `agents/review-coordinator.md`, except the brief-conflict rule, which applies in every mode.
Passes 1, 2, and 4 still run.
Rationale: high-effort review trusts the operator to weigh borderline
findings rather than have the coordinator drop them. To prevent
unknown `rule_key` values from polluting downstream remediation plans

View file

@ -484,7 +484,7 @@ The gate cannot check an `engine` label on the prose path: there the coordinator
the workflow path (`--workflow`) takes `engine` from the payload source and writes it into the returned finding.
`runContract` (`lib/review/coordinator-contract.mjs`) likewise takes `engine` from the payload a finding arrived in, never from the finding's own field,
and in dedup the survivor is ranked by catalogue tier (for a catalogue key it always ties inside one triplet), so a reviewer-agent finding survives over an engine finding and carries the highest severity the group asserted; an engine's mark never lowers what a reviewer agent asserted.
On the workflow path dedup ranks as Pass 1 does (both paths), every engine finding the coordinator did not return at its exact `(file, line, rule_key)` is put back at its own severity (operator decision 23), and a returned finding is never below the lower of its reviewer counterpart's severity and its catalogue tier (floor (b)), so the workflow path is stricter than the prose path; a coordinator that returns no verdict and findings counts as a missing reviewer there (`missing-reviewer:review-coordinator` in `allow_blocked_by`), and its findings are read at their catalogue tier or higher, so the review is at least WARN and the verdict gate exits 2 on such a WARN (a BLOCK stays exit 1).
On the workflow path dedup ranks as Pass 1 does (both paths), and after the coordinator each finding is matched to its own row, its `(file, line, rule_key)` triplet plus its title, trimmed and in lower case: of the engine findings in a row no returned finding is on, the one with the highest severity (on a tie, the first seen) is put back at its own severity (operator decision 23), and a returned finding is never below the lower of its reviewer counterpart's severity and its catalogue tier (floor (b)); its counterpart is its own row, else the strictest row on its triplet, so the workflow path is stricter than the prose path; a coordinator that returns no verdict and findings counts as a missing reviewer there (`missing-reviewer:review-coordinator` in `allow_blocked_by`), and its findings are read at their catalogue tier or higher, so the review is at least WARN and the verdict gate exits 2 on such a WARN (a BLOCK stays exit 1).
A plan made from the review skips a lowered finding: it is MINOR, and `/trekplan` takes BLOCKER and MAJOR findings only.
**Validation strategy:**

View file

@ -3,8 +3,12 @@
// .claude/projects/2026-10-20-fix-round-serious-findings): the fixes that are
// text a session or a helper reads. The name is the brief's;
// lib/util/test-census.mjs counts it as a behaviour test (it does not match
// PIN_FILE_RE). Every test carries its id tag in brackets, and every tagged
// test fails on an assertion at ab6b32c (the round's start point).
// PIN_FILE_RE). A test that carries a round id in brackets fails on an
// assertion at ab6b32c (the round's start point), except DJ's pins: they pass
// there by design, and tests/fix-round-4-5a-red.sh proves them red by mutation.
// Untagged tests (prefixes `helpers:`, `step 25:`, `keep:`, `control:`,
// `arch:`, `e2e script:`, `red script:`, `CHANGELOG:`, `found in execution:`)
// are outside the round's id list.
// #23 code-correctness-reviewer cites a "MANDATORY OVERRIDE rule above"
// that is no longer there; the sentence says the rule itself.
// #24 research-scout says "Try Tavily first" but its tools held no Tavily
@ -1387,7 +1391,7 @@ test('[#113] no conditional trigger is left, and high effort keeps contrarian-re
test('[#113] sibling: README.md', () => {
const text = flat(read('README.md'));
assert.doesNotMatch(text, /contrarian-researcher when a leading recommendation forms/);
assert.ok(text.includes('contrarian-researcher, when selected, runs in wave 2 after the other external agents return, with their leading answer to attack (always at `effort=high`)'),
assert.ok(text.includes('contrarian-researcher, when selected, runs in wave 2 after wave 1 returns, with the leading answer of wave 1\'s external findings to attack (with `--engine deep-research`, after the engine returns, from the leading answer of its report), always at `effort=high`;'),
'README.md does not describe wave 2');
});
@ -1727,3 +1731,120 @@ test('CHANGELOG: the released v6.5.0 section is byte-equal to its text at ab6b32
assert.equal(createHash('sha256').update(part, 'utf8').digest('hex'), V650_SHA256,
'the released v6.5.0 section changed; edit only the round\'s section');
});
// Found in execution (untagged, outside the round's id list): texts next to
// the round's fixes that still said what the code no longer does. Each test
// fails on an assertion at 8a2a989, before the text changed; its presence
// assertion holds there.
// D1: the workflow matches each finding to its own row (`rowOf` in
// scripts/trekreview-armB.workflow.mjs), not to its exact triplet.
const PUT_BACK_TEXTS = [
['commands/trekreview.md', () => partAt('commands/trekreview.md', 'On this path dedup ranks as Pass 1 does', toBlank)],
['docs/HANDOVER-CONTRACTS.md', () => partAt('docs/HANDOVER-CONTRACTS.md', 'On the workflow path dedup ranks as Pass 1 does', () => true)],
];
const ROW_IS = 'its `(file, line, rule_key)` triplet plus its title, trimmed and in lower case';
const COUNTERPART_IS = 'its counterpart is its own row, else the strictest row on its triplet';
const PUT_BACK_IS = 'of the engine findings in a row no returned finding is on, the one with the highest severity (on a tie, the first seen) is put back at its own severity (operator decision 23)';
for (const [rel, part] of PUT_BACK_TEXTS) {
test(`found in execution: ${rel} no longer puts back what the coordinator did not return "at its exact (file, line, rule_key)"`, () => {
const text = part();
assert.ok(text.includes('put back at its own severity'), `${rel}: the put-back sentence is missing: ${text}`);
assert.doesNotMatch(text, /at its exact `\(file, line, rule_key\)`/, `${rel}: the put-back still matches on the exact triplet`);
});
test(`found in execution: ${rel} says a finding is matched to its row, the triplet plus the normalized title`, () => {
const text = part();
assert.ok(text.includes('put back at its own severity'), `${rel}: the put-back sentence is missing: ${text}`);
assert.ok(text.includes(`each finding is matched to its own row, ${ROW_IS}`), `${rel}: the put-back does not say the row: ${text}`);
});
test(`found in execution: ${rel} puts back one copy per row, the row's engine finding with the highest severity`, () => {
const text = part();
assert.ok(text.includes('put back at its own severity'), `${rel}: the put-back sentence is missing: ${text}`);
assert.ok(text.includes(PUT_BACK_IS), `${rel}: the put-back does not say which one copy of a row is put back: ${text}`);
});
test(`found in execution: ${rel} says floor (b)'s counterpart is the finding's own row, else the strictest row on its triplet`, () => {
const text = part();
assert.ok(text.includes('(floor (b))'), `${rel}: floor (b) is missing: ${text}`);
assert.ok(text.includes(COUNTERPART_IS), `${rel}: floor (b) does not say which row is the counterpart: ${text}`);
});
}
// D2: the README's conditional line says what /trekresearch says.
const conditionalLine = () => partAt('README.md', '`* = conditional`:', () => true);
test('found in execution: README.md names no "leading recommendation" trigger for security-researcher', () => {
const text = conditionalLine();
assert.ok(text.includes('contrarian-researcher'), `the conditional line does not name contrarian-researcher: ${text}`);
assert.doesNotMatch(text, /security-researcher when a leading recommendation forms/, 'README.md still names a trigger /trekresearch does not define');
});
test('found in execution: README.md stars security-researcher nowhere, as docs-researcher and community-researcher (selected by scope alike)', () => {
const diagram = partAt('README.md', 'RE["EXTERNAL: docs-researcher', () => true);
assert.ok(diagram.includes('security-researcher'), `the diagram's external line does not name security-researcher: ${diagram}`);
assert.ok(diagram.includes('security-researcher · contrarian-researcher*'), `the diagram still stars security-researcher: ${diagram}`);
});
test('found in execution: README.md lists security-researcher as no conditional agent', () => {
const text = conditionalLine();
assert.ok(text.includes('contrarian-researcher'), `the conditional line does not name contrarian-researcher: ${text}`);
assert.ok(!text.includes('security-researcher'), `the conditional line still lists security-researcher: ${text}`);
});
test('found in execution: README.md names the deep-research case of wave 2', () => {
const text = conditionalLine();
assert.ok(text.includes('contrarian-researcher, when selected, runs in wave 2'), `the conditional line does not name wave 2: ${text}`);
assert.ok(text.includes('contrarian-researcher, when selected, runs in wave 2 after wave 1 returns, with the leading answer of wave 1\'s external findings to attack (with `--engine deep-research`, after the engine returns, from the leading answer of its report), always at `effort=high`;'), `README.md does not name the deep-research case of wave 2: ${text}`);
});
test('found in execution: README.md runs contrarian-researcher after wave 1 returns', () => {
const text = conditionalLine();
assert.ok(text.includes('contrarian-researcher, when selected, runs in wave 2'), `the conditional line does not name wave 2: ${text}`);
assert.ok(text.includes('runs in wave 2 after wave 1 returns'), `README.md does not say "after wave 1 returns": ${text}`);
});
// D3: item 3 of both 4-pass lists names the high-effort skip and points at the normalization.
const HIGH_EFFORT_SKIP = {
'commands/trekreview.md': /Skipped at high effort too, where an unknown `rule_key` is normalized to `PLAN_EXECUTE_DRIFT` instead \(.*High-effort.*\); the brief-conflict rule still applies in every mode\./,
// This item 3 names no quick mode, so no "too": it points at the quick-mode line below the list.
'agents/review-orchestrator.md': /Skipped at high effort, as in `quick` mode \(below\); at high effort an unknown `rule_key` is normalized to `PLAN_EXECUTE_DRIFT` instead \(.*High-effort.*\); the brief-conflict rule still applies in every mode\./,
};
for (const rel of Object.keys(HIGH_EFFORT_SKIP)) {
test(`found in execution: ${rel} item 3 names Pass 3's high-effort skip and the PLAN_EXECUTE_DRIFT normalization`, () => {
const item = pass3Item(rel);
assert.ok(item.includes(PASS3_ITEM.slice(PASS3_ITEM.indexOf(' — ') + 3)), `${rel}: item 3 is not the Pass 3 sentence: ${item}`);
assert.match(item, HIGH_EFFORT_SKIP[rel], `${rel}: item 3 does not name the high-effort skip: ${item}`);
});
}
// D4: the siblings entry has its own group label.
test('found in execution: CHANGELOG ### Fixed has the **Across the round** label directly above the siblings entry', () => {
const lines = section(roundPart(), '### Fixed').split('\n');
const at = lines.map((l, i) => (l.startsWith('- **The texts next to the fixes say the same**') ? i : -1)).filter((i) => i !== -1);
assert.equal(at.length, 1, `### Fixed must hold the siblings entry once; found ${at.length}`);
assert.deepEqual(lines.slice(at[0] - 2, at[0]), ['**Across the round**', ''],
`the line pair above the siblings entry is not the **Across the round** label: ${JSON.stringify(lines.slice(at[0] - 2, at[0]))}`);
});
// The skip sentences outside the 4-pass lists keep the brief-conflict rule,
// as agents/review-coordinator.md does ("still applies under high-effort mode",
// "still applies in quick mode").
const BRIEF_CONFLICT_KEPT = ', except the brief-conflict rule, which applies in every mode.';
test('found in execution: commands/trekreview.md high-effort section skips Pass 3 except the brief-conflict rule', () => {
const text = flat(section(read('commands/trekreview.md'), '### High-effort behavior (v5.1.1)'));
assert.ok(text.includes('skip Pass 3 (Cloudflare reasonableness filter)'), `the high-effort section does not skip Pass 3: ${text}`);
assert.ok(text.includes(`skip Pass 3 (Cloudflare reasonableness filter) in \`agents/review-coordinator.md\`${BRIEF_CONFLICT_KEPT}`),
`the high-effort skip keeps no brief-conflict rule: ${text}`);
});
test('found in execution: agents/review-orchestrator.md quick mode skips pass 3 except the brief-conflict rule', () => {
const text = partAt('agents/review-orchestrator.md', 'In `quick` mode, skip pass 3', toBlank);
assert.ok(text.includes('skip pass 3 (reasonableness filter)'), `the quick-mode line does not skip pass 3: ${text}`);
assert.ok(text.includes(`In \`quick\` mode, skip pass 3 (reasonableness filter)${BRIEF_CONFLICT_KEPT}`),
`the quick-mode skip keeps no brief-conflict rule: ${text}`);
});