feat(trekresearch): replace single follow-up pass with bounded conversation loop
This commit is contained in:
parent
0a569eec55
commit
3e8af75015
2 changed files with 210 additions and 5 deletions
|
|
@ -393,13 +393,98 @@ other agents — the value of Gemini is independence.
|
|||
|
||||
## Phase 5 — Targeted follow-ups
|
||||
|
||||
Review all agent results. Identify knowledge gaps — areas where findings are
|
||||
thin, contradictory, or missing.
|
||||
Review all agent results. Identify knowledge gaps — dimensions where findings
|
||||
are thin, contradictory, or missing (**under-illuminated dimensions**).
|
||||
|
||||
For each significant gap, launch a targeted follow-up agent (model: "opus")
|
||||
with a narrow, specific brief. Maximum 2 follow-ups.
|
||||
**Standard and low effort — unchanged single pass.** For each significant gap,
|
||||
launch a targeted follow-up agent (model: "opus") with a narrow, specific
|
||||
brief. Maximum 2 follow-ups. If no gaps exist, skip: "Initial research
|
||||
sufficient — no follow-ups needed." Then go to Phase 6.
|
||||
|
||||
If no gaps exist, skip: "Initial research sufficient — no follow-ups needed."
|
||||
**The bounded loop below runs ONLY when `phase_signal_result.effort == 'high'`**
|
||||
(resolved in Phase 1; see `### High-effort behavior (v5.1.1)`). At any other
|
||||
effort this whole sub-section is inert — no loop, no cap ledger, no new
|
||||
counters beyond zero.
|
||||
|
||||
### Loop bound
|
||||
|
||||
**Maximum 3 turns per under-illuminated dimension.** The bound is per
|
||||
dimension, not per run: the worst case is 3 turns × the whole dimension list
|
||||
under the `maxDimensions: 8` ceiling (`settings.json:16`), which is what
|
||||
`research-loop-cap.mjs` sizes itself against. The cap counts itself from its
|
||||
own append-only ledger — it never asks this prose how many turns it has used.
|
||||
|
||||
The loop is **default-off**: `research-loop-cap.mjs` grants a budget of 0
|
||||
unless `VOYAGE_STORM_ENABLED=1`. Doing nothing leaves the mechanism off.
|
||||
|
||||
### Per-turn protocol
|
||||
|
||||
Each turn targets exactly one under-illuminated dimension, and runs two gates
|
||||
before it spends anything:
|
||||
|
||||
```bash
|
||||
# 1. Budget gate — per turn, per dimension. Exit 0 = granted, exit 1 = denied.
|
||||
# JSON on stdout: {ok, used, budget, reason?}
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/util/research-loop-cap.mjs \
|
||||
--run-id {run_id} --dimension {dimension} --effort {phase_signal_result.effort}
|
||||
|
||||
# 2. Privacy gate — EVERY outbound query, before it leaves the machine.
|
||||
# Exit 0 = send as-is; exit 1 = rewrite the query and re-gate. Never bypass.
|
||||
node ${CLAUDE_PLUGIN_ROOT}/lib/validators/query-privacy-gate.mjs "{query text}"
|
||||
```
|
||||
|
||||
A denied budget gate is an exit condition, not a retry. A failed privacy gate
|
||||
is a rewrite: the hard-block tier (secret-shaped strings) is never
|
||||
operator-overridable, so a query that trips it must be reformulated, not
|
||||
forced through.
|
||||
|
||||
**Empty turns.** A turn that returns no findings, or findings without
|
||||
citations, is marked `empty`. An empty turn is counted in `empty_turns` and
|
||||
does NOT re-target the same dimension — re-asking the same question of the
|
||||
same silence is how a bounded loop turns into an unbounded one. Move to the
|
||||
next under-illuminated dimension, or exit.
|
||||
|
||||
### Exits (all three, always one of them)
|
||||
|
||||
1. **Converged** — the dimension carries findings with citations and no
|
||||
remaining contradiction. Stop turning on it. This is the normal exit.
|
||||
2. **Cap exhausted** — `research-loop-cap.mjs` denies the turn. Print the
|
||||
exhaustion **visibly** to the operator, never silently:
|
||||
`Loop bound reached for dimension {dimension} after {N} turns — remaining
|
||||
gaps are carried into the brief as open questions.` A silent cap is
|
||||
indistinguishable from convergence, and that confusion is exactly what this
|
||||
phase exists to prevent.
|
||||
3. **Operator stop** — the operator interrupts. Carry whatever has been
|
||||
gathered into Phase 6 and record the remaining gaps as open questions. Do
|
||||
not re-enter the loop after a stop.
|
||||
|
||||
### When the loop does not apply
|
||||
|
||||
**No-brief default.** Without `--project` (or with a project whose `brief.md`
|
||||
is absent), there are no `phase_signals` to resolve, so `effort = 'standard'`,
|
||||
the loop does not run, and all new counters (`conv_turns`, `empty_turns`) are
|
||||
emitted as `0`.
|
||||
|
||||
**Precedence matrix — each entry independently makes the loop moot**, the same
|
||||
way `--engine` is moot when the external phase does not run (see the moot gate
|
||||
in Phase 4):
|
||||
|
||||
| Condition | Effect on the loop |
|
||||
|-----------|--------------------|
|
||||
| `--quick` | Moot — Phase 3.5 skips to Phase 8; the swarm never runs |
|
||||
| `--local` | Moot — no outbound queries to bound |
|
||||
| `external_research_enabled: false` (profile) | Moot — the profile's on/off switch wins |
|
||||
|
||||
**Interaction rule.** A brief that carries `effort: high` **without** a
|
||||
`model`, under a cheap profile (`economy`/`balanced`): the effort signal
|
||||
governs orchestration shape, so the loop is armed, but the profile still
|
||||
supplies the model — and if that profile disables external research, the
|
||||
matrix above wins and the loop is moot regardless of effort.
|
||||
|
||||
**Honesty (hard rule, restated for this loop).** More turns do not make a
|
||||
finding more credible. Turn count is a cost, not evidence: report what the
|
||||
citations support, and let an exhausted cap show up as open questions rather
|
||||
than as confidence.
|
||||
|
||||
## Phase 6 — Triangulation
|
||||
|
||||
|
|
@ -623,6 +708,14 @@ significant architectural questions or when triangulation value is
|
|||
high; in high-effort mode it runs unconditionally to provide an
|
||||
independent second opinion.
|
||||
|
||||
High effort additionally arms the Phase 5 bounded follow-up loop (max 3
|
||||
turns per under-illuminated dimension, budgeted by
|
||||
`research-loop-cap.mjs`, every outbound query gated by
|
||||
`query-privacy-gate.mjs`). The loop stays default-off until
|
||||
`VOYAGE_STORM_ENABLED=1`, and the moot matrix in Phase 5 (`--quick`,
|
||||
`--local`, `external_research_enabled: false`) overrides the effort
|
||||
signal whenever the external phase does not run at all.
|
||||
|
||||
Standard effort (or absent): use the existing conditional triggers.
|
||||
Low effort: inline research only, no agent swarm (existing
|
||||
`--quick`-equivalent code-path).
|
||||
|
|
|
|||
|
|
@ -54,6 +54,118 @@ test('trekresearch — low-effort path references --quick equivalent', () => {
|
|||
'heading-bounded slice must still fail on a genuine removal');
|
||||
});
|
||||
|
||||
// --- Step 7: Phase 5 bounded conversation loop (heading-bounded slices) ---
|
||||
|
||||
// Same bounding discipline as the Composition-rule pin above: slice from the
|
||||
// phase heading to the NEXT phase heading, never a fixed character window.
|
||||
function phaseSlice(doc, startHeading, endHeading) {
|
||||
const start = doc.indexOf(startHeading);
|
||||
assert.ok(start >= 0, `${startHeading} missing`);
|
||||
const end = doc.indexOf(endHeading, start);
|
||||
assert.ok(end > start, `${endHeading} missing — could not bound ${startHeading}`);
|
||||
return doc.slice(start, end);
|
||||
}
|
||||
|
||||
function phase5(doc) {
|
||||
return phaseSlice(doc, '## Phase 5 —', '## Phase 6 —');
|
||||
}
|
||||
|
||||
test('trekresearch — Phase 5 loop is gated on effort == high and names both primitives', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /effort == 'high'/, 'Phase 5 loop must be gated on effort == \'high\'');
|
||||
assert.match(p5, /research-loop-cap\.mjs/, 'Phase 5 must call the loop-cap shim per turn');
|
||||
assert.match(p5, /query-privacy-gate\.mjs/, 'Phase 5 must route outbound queries through the privacy gate');
|
||||
assert.match(p5, /\$\{CLAUDE_PLUGIN_ROOT\}/, 'shim invocations must use the ${CLAUDE_PLUGIN_ROOT} path form');
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 declares the loop bound and all three exits', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /### Loop bound/, 'Phase 5 must carry a `### Loop bound` sub-heading');
|
||||
assert.match(
|
||||
p5,
|
||||
/\*\*Maximum 3 turns per under-illuminated dimension\.\*\*/,
|
||||
'the bound must be stated verbatim',
|
||||
);
|
||||
// Three exits — converged / cap exhausted / operator stop.
|
||||
assert.match(p5, /converged/i, 'exit 1 (converged) must be documented');
|
||||
assert.match(p5, /exhaust/i, 'exit 2 (cap exhausted) must be documented');
|
||||
assert.match(p5, /operator stop/i, 'exit 3 (operator stop) must be documented');
|
||||
// Exhaustion must reach the operator — a silent cap is indistinguishable
|
||||
// from convergence, which is the failure this loop exists to avoid.
|
||||
assert.match(
|
||||
p5,
|
||||
/visibl|visible|print/i,
|
||||
'cap exhaustion must be written visibly to the operator',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 marks empty turns without re-targeting the same dimension', () => {
|
||||
const p5 = phase5(read());
|
||||
assert.match(p5, /`empty`/, 'a finding-less or citation-less turn must be marked `empty`');
|
||||
assert.match(p5, /empty_turns/, 'empty turns must be counted (empty_turns)');
|
||||
assert.match(
|
||||
p5,
|
||||
/does NOT re-target|not re-target/i,
|
||||
'an empty turn must not re-target the same dimension',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 states the no-brief default and the moot precedence matrix', () => {
|
||||
const p5 = phase5(read());
|
||||
// (a) no-brief default
|
||||
assert.match(p5, /effort = 'standard'/, 'no-brief default effort must be stated');
|
||||
assert.match(
|
||||
p5,
|
||||
/--project/,
|
||||
'the no-brief default must be anchored to the absence of --project/brief.md',
|
||||
);
|
||||
// (b) precedence matrix — each entry independently makes the loop moot,
|
||||
// mirroring the --engine moot gate in Phase 4.
|
||||
for (const token of ['--quick', '--local', 'external_research_enabled']) {
|
||||
assert.ok(p5.includes(token), `moot matrix must name ${token}`);
|
||||
}
|
||||
assert.match(p5, /moot/i, 'the matrix must use the same moot vocabulary as the engine gate');
|
||||
// (c) interaction rule — effort: high without model under a cheap profile.
|
||||
assert.match(
|
||||
p5,
|
||||
/effort: high/,
|
||||
'the interaction rule for a brief carrying effort: high without model must be stated',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 restates the honesty rule for loop output', () => {
|
||||
const p5 = phase5(read());
|
||||
// Whitespace-tolerant: the pin is on the sentence, not on where the
|
||||
// paragraph happens to wrap.
|
||||
assert.match(
|
||||
p5,
|
||||
/more\s+turns\s+do\s+not\s+make\s+a\s+finding\s+more\s+credible/i,
|
||||
'the honesty hard rule must be restated for the loop output',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — Phase 5 pins survive only while the prose does (mutation control)', () => {
|
||||
const text = read();
|
||||
const mutated = text.replace(/research-loop-cap\.mjs/g, 'removed-cap.mjs');
|
||||
assert.doesNotMatch(
|
||||
phase5(mutated),
|
||||
/research-loop-cap\.mjs/,
|
||||
'heading-bounded Phase 5 slice must still fail on a genuine removal',
|
||||
);
|
||||
});
|
||||
|
||||
test('trekresearch — High-effort behavior keeps the standard/low effort sentences verbatim', () => {
|
||||
const text = read();
|
||||
assert.ok(
|
||||
text.includes('Standard effort (or absent): use the existing conditional triggers.'),
|
||||
'the standard-effort sentence must survive the Phase 5 rewrite verbatim',
|
||||
);
|
||||
assert.ok(
|
||||
text.includes('Low effort: inline research only, no agent swarm'),
|
||||
'the low-effort sentence must survive the Phase 5 rewrite verbatim',
|
||||
);
|
||||
});
|
||||
|
||||
// --- v5.1.1 runtime SC4 + SC7 ---
|
||||
|
||||
test('trekresearch — SC4: low-effort fixture → resolver returns {effort: low, model: sonnet}', () => {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue