open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

chore(cleanup): R-b R2 — STORM loop, cap hook and the --engine research choice removed

Browse source 
/trekresearch loses Phase 4.5, the bounded Phase 5 loop (Phase 5 keeps its
single pass of at most 2 follow-ups), the --engine flag, the deep-research
adapter and its output self-check. hooks.json loses the pre-agent-cap
PreToolUse entry (README: 9 hooks = 9 scripts). The STORM env vars leave the
docs; query-privacy-gate.mjs stays, called by no command, and says so.

Files removed: 13 (4 docs, 1 hook, 1 lib module, 1 script, 5 tests, 1 fixture).
Files edited: 17. Tests: 2600 (2598 pass / 0 fail / 2 skipped) before,
2467 (2466 pass / 0 fail / 1 skipped, R3 pending) after.
Durable check: group R2 green and no longer pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-11-07 20:32:38 +01:00
commit 32621dcb63
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
30 changed files with 34 additions and 4070 deletions
Download patch file Download diff file Expand all files Collapse all files

351
lib/util/research-loop-cap.mjs
View file

@ -1,351 +0,0 @@
// lib/util/research-loop-cap.mjs
// Stateful, default-off cost cap for the /trekresearch bounded conversation
// loop (Phase 4.5 dimension discovery + Phase 5 loop turns).
//
// Three properties the plan review required:
// (a) Default-off — VOYAGE_STORM_ENABLED must be '1'; otherwise the budget
// is 0 regardless of effort. This IS the decline branch: doing nothing
// leaves the mechanism off, and adopt is flipping this one constant.
// (b) The cap counts itself — allowTurn() derives used-turn count from an
// append-only JSONL ledger, never from a caller-supplied number. A cap
// that asks the caller how many turns it has used is not a cap. Each
// grant additionally claims a turn SLOT with O_EXCL, so the bound holds
// when several callers decide at once instead of only when they queue.
// (c) Correct size bound — worst case is max_conv_turns × max_total_dimensions,
// where max_total_dimensions is the WHOLE list (interview + discovered)
// under the maxDimensions cap of 8 — not × discovered-only.
//
// CLAUDE_PLUGIN_DATA absent => fall back to ~/.claude/voyage. The variable is
// EMPTY in the Bash tool's process env (measured in a live plugin-enabled
// session), and the Phase 5 bash snippet is this module's only caller — so
// denying on its absence denied turn 1 of every real run. The root is resolved
// in code rather than demanded of the environment, and hooks/scripts/
// pre-agent-cap.mjs resolves it through the SAME function, so the writer and
// the reader can never disagree about where the ledger lives.
//
// The fail-closed stance covers both directions of ledger IO: a ledger that
// cannot be WRITTEN denies the turn, and a ledger that exists but cannot be
// READ denies it too. Only ENOENT counts as zero turns spent, because that is
// the legitimate first-turn state. This module is a budget control, not
// telemetry.
//
// CLI shim, two modes:
// node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E
// → JSON: { ok, used, budget, reason? } (exit 0 = granted, exit 1 = denied)
//
// node lib/util/research-loop-cap.mjs --check-dimensions N
// → JSON: { ok, count, ceiling, reason? } (exit 0 = within, exit 1 = rejected)
// The second axis of the bounded-cost NFR. Phase 4.5 never calls the budget
// gate, so this mode requires no run id, effort or STORM flag — but it reads
// the SAME MAX_TOTAL_DIMENSIONS the budget is sized against.
import { existsSync, mkdirSync, appendFileSync, readFileSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { homedir } from 'node:os';
import { isMain } from './is-main.mjs';
export const MAX_CONV_TURNS = 3;
export const MAX_TOTAL_DIMENSIONS = 8; // maxDimensions — whole list, not discovered-only
const LEDGER_FILENAME = 'trekresearch-loop-ledger.jsonl';
const CLAIM_DIRNAME = 'trekresearch-loop-claims';
export function isStormEnabled(env = process.env) {
return env.VOYAGE_STORM_ENABLED === '1';
}
/**
* The OTHER cost ceiling: how large the whole dimension list may get after
* Phase 4.5 discovery has appended to it.
*
* The bounded-cost NFR asks for explicit ceilings on both axes. The turn axis
* had a constant, a ledger-backed reader and a PreToolUse enforcer; the
* discovery axis had only a sentence in Phase 4.5 prose — a cap nothing reads,
* which is the failure mode the operator decision on brief_reviewer_iter_cap
* warned about. This is the reader.
*
* The ceiling is MAX_TOTAL_DIMENSIONS on purpose: the value that sizes the turn
* budget IS maxDimensions, and a second constant for the same
* number is how two readers end up enforcing different bounds.
*
* Accepts the dimension list or its count, because Phase 4.5 has the list and
* the CLI has a number. A count that cannot be read is REJECTED — a cost ceiling
* that waves through what it cannot measure is not a ceiling.
*
* @param {string[]|number|string} dimensions
* @param {{ceiling?: number}} [opts]
* @returns {{ok: boolean, count: number|null, ceiling: number, reason?: string}}
*/
export function checkDimensionCeiling(dimensions, opts = {}) {
const ceiling = Number.isFinite(opts.ceiling) ? opts.ceiling : MAX_TOTAL_DIMENSIONS;
const count = Array.isArray(dimensions) ? dimensions.length : Number(dimensions);
if (dimensions === null || dimensions === undefined || !Number.isInteger(count) || count < 0) {
return { ok: false, count: null, ceiling, reason: 'unreadable_dimension_count' };
}
if (count > ceiling) {
return { ok: false, count, ceiling, reason: 'ceiling_exceeded' };
}
return { ok: true, count, ceiling };
}
/**
* Coerce TREKRESEARCH_MAX_CONV_TURNS. NaN, empty, negative, zero, Infinity, or
* any fraction that floors below 1 all fall back to MAX_CONV_TURNS — never to
* unbounded, and never to 0.
*
* The floor is applied BEFORE the `<= 0` guard, not after. Flooring afterwards
* let '0.5' and '0.9' clear a guard written against the raw value and then
* become 0, making the budget 0 × MAX_TOTAL_DIMENSIONS = 0: every turn denied
* and the loop silently dead rather than bounded. A cap of 0 is not a narrower
* cap, it is an off switch that the documented fallback promises not to be.
*/
export function resolveMaxConvTurns(env = process.env) {
const raw = env.TREKRESEARCH_MAX_CONV_TURNS;
if (raw === undefined || raw === null || raw === '') return MAX_CONV_TURNS;
const n = Math.floor(Number(raw));
if (!Number.isFinite(n) || n <= 0) return MAX_CONV_TURNS;
return n;
}
/**
* The one data root for everything this loop writes: the turn ledger and the
* PreToolUse scope marker. CLAUDE_PLUGIN_DATA when the harness provides it,
* ~/.claude/voyage when it does not — which is the case in every Bash tool
* invocation today.
*/
export function resolveDataRoot(env = process.env) {
const dir = env.CLAUDE_PLUGIN_DATA;
if (dir && typeof dir === 'string' && dir.length > 0) return dir;
const home = env.HOME && env.HOME.length > 0 ? env.HOME : homedir();
return join(home, '.claude', 'voyage');
}
export function resolveLedgerPath(env = process.env) {
return join(resolveDataRoot(env), LEDGER_FILENAME);
}
/**
* Where the per-turn claim files live. A claim is the ATOMIC record that a turn
* slot is taken; the ledger is the readable record of what that turn was for.
*
* The claim exists because the ledger alone cannot bound the loop. Counting the
* ledger and then appending is read-then-write: N callers that all observe
* `used == budget - 1` all decide to grant, and the bound is exceeded by N-1 —
* precisely the concurrent case (several agents spawned in one message) that
* allowTurn's own comment named as the reason it had to be append-only.
*
* Claim files are empty, at most `budget` per run, and never cleaned up — the
* same standing as the ledger itself, which also grows for the life of the data
* root. Two consequences worth stating rather than discovering: reusing a
* runId across runs finds its slots already taken and denies, and two runIds
* that collide after filename sanitisation block each other. Both err toward
* denying a turn, which is the safe direction for a budget control.
*/
export function resolveClaimDir(env = process.env) {
return join(resolveDataRoot(env), CLAIM_DIRNAME);
}
function claimFileName(runId, slot) {
return `${String(runId).replace(/[^A-Za-z0-9._-]/g, '_')}-${slot}.claim`;
}
/**
* Try to take turn slot `slot` for `runId`. `wx` is O_CREAT|O_EXCL: the kernel
* decides the winner, so exactly one caller can ever create a given slot file.
*
* @returns {boolean} true when this caller took the slot, false when it was already taken
* @throws on any IO error other than EEXIST — the caller turns that into a denial
*/
function claimSlot(claimDir, runId, slot) {
try {
writeFileSync(join(claimDir, claimFileName(runId, slot)), '', { flag: 'wx' });
return true;
} catch (e) {
if (e && e.code === 'EEXIST') return false;
throw e;
}
}
/**
* Read one run's turn count off the append-only ledger.
*
* ENOENT is 0 turns spent — the legitimate first-turn state, and the reason
* this cannot simply throw on every read failure. Every OTHER read error
* (EISDIR, EACCES, EIO) THROWS, because returning 0 from an unreadable ledger
* re-granted the full budget on every call: unbounded, and the exact
* silently-grant-unlimited failure this module's header argues against three
* lines above the code that did it. The missing-directory case already failed
* closed; this makes the unreadable-file case agree with it.
*
* The `existsSync` pre-check is deliberately gone: readFileSync's own ENOENT
* carries the same information without a second syscall that can disagree with
* the read that follows it.
*
* Exported so hooks/scripts/pre-agent-cap.mjs counts through this exact
* function. A reader and a writer with private copies of the counting rule are
* how a hook ends up enforcing a different bound than the gate it backs.
*
* Two counts, deliberately separate. `granted` is turns handed out. `exhausted`
* is tombstones — records this gate wrote when it DENIED a turn. A tombstone is
* not a turn and must never consume budget; it exists so the PreToolUse hook can
* tell "turn B is in flight" (granted == budget, no tombstone) apart from "the
* gate already said no and something kept going" (tombstone present).
*
* @param {string} ledgerPath
* @param {string} runId
* @returns {{granted: number, exhausted: number}}
* @throws when the ledger exists but cannot be read
*/
export function readLedger(ledgerPath, runId) {
let text;
try {
text = readFileSync(ledgerPath, 'utf-8');
} catch (e) {
if (e && e.code === 'ENOENT') return { granted: 0, exhausted: 0 };
const err = new Error(`ledger unreadable at ${ledgerPath}: ${e.message}`);
err.code = 'VOYAGE_LEDGER_UNREADABLE';
throw err;
}
let granted = 0;
let exhausted = 0;
for (const line of text.split('\n')) {
if (!line) continue;
try {
const rec = JSON.parse(line);
if (rec.runId !== runId) continue;
if (rec.exhausted === true) exhausted++;
else granted++;
} catch { /* skip malformed lines */ }
}
return { granted, exhausted };
}
/**
* Record that this run has been denied a turn for budget.
*
* Best effort on purpose: the denial itself is already the correct answer, so a
* ledger that cannot take the tombstone must not turn a denial into a grant. The
* tombstone only strengthens the harness-level backstop.
*/
function markExhausted(ledgerPath, runId, now) {
try {
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, exhausted: true }) + '\n');
} catch { /* best effort — see above */ }
}
/**
* Decide whether one more research-loop turn may run.
*
* Phase 4.5/5 may spawn several agents in a single message, so the decision has
* to survive concurrent callers. It does that by CLAIMING a turn slot with
* O_EXCL (see claimSlot) and only then appending to the ledger. The comment
* that used to sit here asserted "append-only: never read-modify-write" as if
* appending were itself the concurrency guarantee — but the decision path was
* count-then-append, which is read-then-write, so the claim was unsupported by
* the code beneath it. The kernel now picks the winner for each slot.
*
* @param {{runId: string, dimension: string, effort: string}} args
* @param {{env?: object, now?: Date}} [opts]
* @returns {{ok: boolean, used: number, budget: number, reason?: string}}
*/
export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
const env = opts.env || process.env;
const now = opts.now || new Date();
if (!isStormEnabled(env)) {
return { ok: false, used: 0, budget: 0, reason: 'storm_disabled' };
}
if (effort !== 'high') {
return { ok: false, used: 0, budget: 0, reason: 'effort_not_high' };
}
if (!runId || !dimension) {
return { ok: false, used: 0, budget: 0, reason: 'missing_args' };
}
const maxConvTurns = resolveMaxConvTurns(env);
const budget = maxConvTurns * MAX_TOTAL_DIMENSIONS;
const ledgerPath = resolveLedgerPath(env);
let ledger;
try {
ledger = readLedger(ledgerPath, runId);
} catch (e) {
return { ok: false, used: 0, budget, reason: `ledger-read-failed: ${e.message}` };
}
const used = ledger.granted;
// Already tombstoned: this run is over. Short-circuit so a hammered gate
// neither walks every slot again nor appends a second tombstone.
if (ledger.exhausted > 0) {
return { ok: false, used, budget, reason: 'budget_exhausted' };
}
// Claim a turn SLOT before spending anything. The ledger count only says
// where to start looking; the claim is what makes the grant exclusive. Slot
// numbers are bounded by `budget`, and each can be created exactly once, so
// the total number of grants for a run can never exceed the budget however
// many callers arrive at once.
const claimDir = resolveClaimDir(env);
let slot = used + 1;
let claimed = false;
try {
mkdirSync(claimDir, { recursive: true });
while (slot <= budget) {
if (claimSlot(claimDir, runId, slot)) { claimed = true; break; }
slot++;
}
} catch (e) {
return { ok: false, used, budget, reason: `claim-failed: ${e.message}` };
}
if (!claimed) {
markExhausted(ledgerPath, runId, now);
return { ok: false, used, budget, reason: 'budget_exhausted' };
}
try {
const dir = dirname(ledgerPath);
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, dimension, effort, slot }) + '\n');
} catch (e) {
return { ok: false, used, budget, reason: `ledger-write-failed: ${e.message}` };
}
return { ok: true, used: slot, budget };
}
// ---- CLI shim ----------------------------------------------------------------
function parseArgs(argv) {
const out = {};
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--run-id') out.runId = argv[++i];
else if (a === '--dimension') out.dimension = argv[++i];
else if (a === '--effort') out.effort = argv[++i];
else if (a === '--check-dimensions') out.checkDimensions = argv[++i];
}
return out;
}
if (isMain(import.meta.url)) {
const args = parseArgs(process.argv.slice(2));
// The dimension ceiling is a Phase 4.5 concern, and Phase 4.5 never calls the
// budget gate — so this branch must not inherit the gate's preconditions
// (run id, effort, STORM flag). It is a pure bound on list size.
if (args.checkDimensions !== undefined) {
const result = checkDimensionCeiling(args.checkDimensions);
process.stdout.write(JSON.stringify(result) + '\n');
process.exit(result.ok ? 0 : 1);
}
if (!args.runId || !args.dimension || !args.effort) {
process.stdout.write(JSON.stringify({
ok: false,
reason: 'usage: research-loop-cap.mjs --run-id ID --dimension D --effort standard|high|low',
}) + '\n');
process.exit(1);
}
const result = allowTurn(args);
process.stdout.write(JSON.stringify(result) + '\n');
process.exit(result.ok ? 0 : 1);
}

6
lib/validators/query-privacy-gate.mjs
View file

@ -1,8 +1,6 @@
// lib/validators/query-privacy-gate.mjs
// Inspect an outbound research query before it leaves the machine. Called
// only from the new high-effort steps (Phase 4.5 dimension discovery + the
// bounded Phase 5 loop turns) — the existing single-pass Phase 5 path is
// unchanged (Step 6, plan-v2).
// Inspect an outbound research query before it leaves the machine. No
// command calls it now: its callers were the removed STORM steps.
//
// Two-tier:
// - WARN tier — absolute filesystem paths, repo-internal identifiers.