fix(cap): move the enforcement boundary to a denial tombstone, not the turn count
allowTurn() appends BEFORE the turn runs, so during granted turn N the
ledger holds N records. The hook denied at `used >= budget`, which blocked
every tool call of the FINAL granted turn: the primitive granted B turns
and the harness permitted B-1. Worse, an exhausted run therefore always
terminated through an exit-2 tool denial instead of the graceful "cap
exhausted" exit at commands/trekresearch.md - and the prose says in as
many words that exit 2 is not exit 1, so the model was pushed out through
the one exit it is told NOT to treat as a cap.
The review recommended denying at `used > budget`. Taken alone that fixes
the count and breaks the hook: once the O_EXCL claim (previous commit)
makes a breached ledger impossible, `granted > budget` can no longer fire,
and the case this hook exists for - the loop consults the gate, is denied,
and issues the tool call anyway - would be allowed. A deny branch that
cannot be reached is a dead security claim, which is the same thing S82
removed two of rather than leave standing.
So the denial itself became a record. allowTurn() appends a tombstone
{runId, exhausted: true} when it denies for budget, and the hook denies on
the tombstone. Both properties now hold at once:
granted == budget, no tombstone -> turn B is in flight -> ALLOW
tombstone present -> the gate already said no -> DENY
granted > budget -> breached, any cause -> DENY
A tombstone is not a turn: readLedger reports {granted, exhausted}
separately so it can never consume budget. allowTurn short-circuits on an
existing tombstone, so a hammered gate neither re-walks every slot nor
grows the ledger. The tombstone write is best effort on purpose - the
denial is already the correct answer, so a ledger that cannot take the
record must not turn a denial into a grant.
The parallel-boundary test now asserts GRANTED turns rather than raw
ledger lines, because the denied callers legitimately add tombstones.
Review finding 8eb53458ac3efec778094f9f03b09e1cc1077a09 (MINOR).
Operator decision: tombstone over the literal recommended_action.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv
This commit is contained in:
parent
156539204a
commit
32e20fc0dc
6 changed files with 180 additions and 33 deletions
|
|
@ -495,9 +495,11 @@ the failure to the operator and run the loop. The reverse — skipping the budge
|
||||||
gate because a marker exists — is never allowed.
|
gate because a marker exists — is never allowed.
|
||||||
|
|
||||||
**Removal belongs to every exit below, especially the exhausted one.** The hook
|
**Removal belongs to every exit below, especially the exhausted one.** The hook
|
||||||
denies `WebSearch`/`WebFetch`/`Task` once the budget is spent, and it keeps
|
denies `WebSearch`/`WebFetch`/`Task` once `research-loop-cap.mjs` has denied a
|
||||||
denying for as long as the marker is there — including Phase 6, which spawns
|
turn — the gate records its own denials, so the LAST granted turn still runs its
|
||||||
agents. A marker that outlives the loop turns a bound on this loop into a brick
|
queries and exhaustion reaches you through exit 2 of the budget gate below, not
|
||||||
|
through a blocked tool call. Once denied, the hook keeps denying for as long as
|
||||||
|
the marker is there — including Phase 6, which spawns agents. A marker that outlives the loop turns a bound on this loop into a brick
|
||||||
on the rest of the session. Cleanup covers the three exits and nothing else: a
|
on the rest of the session. Cleanup covers the three exits and nothing else: a
|
||||||
crashed session runs no cleanup at all, and is covered instead by the hook's
|
crashed session runs no cleanup at all, and is covered instead by the hook's
|
||||||
TTL (default 6h, `VOYAGE_CAP_SCOPE_TTL_MS`), which auto-resets a stale marker.
|
TTL (default 6h, `VOYAGE_CAP_SCOPE_TTL_MS`), which auto-resets a stale marker.
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ Imported from `CLAUDE.md` via pointer.
|
||||||
- `lib/stats/event-emit.mjs` — single-source stats event emitter for autonomy-gate transitions and main-merge-gate (v3.4.0)
|
- `lib/stats/event-emit.mjs` — single-source stats event emitter for autonomy-gate transitions and main-merge-gate (v3.4.0)
|
||||||
- `lib/validators/{brief,research,plan,progress,session-state}-validator.mjs` — schema validators with CLI shims (`node lib/validators/X.mjs --json <path>`)
|
- `lib/validators/{brief,research,plan,progress,session-state}-validator.mjs` — schema validators with CLI shims (`node lib/validators/X.mjs --json <path>`)
|
||||||
- `lib/validators/architecture-discovery.mjs` — drift-WARN external-contract discovery for `architecture/overview.md`
|
- `lib/validators/architecture-discovery.mjs` — drift-WARN external-contract discovery for `architecture/overview.md`
|
||||||
- `lib/util/research-loop-cap.mjs` — stateful, **default-off** turn budget for the `/trekresearch` bounded conversation loop. `allowTurn()` derives the used-turn count from its own append-only JSONL ledger; it never asks the caller how many turns it has spent, because a cap that does is not a cap. Each grant first claims a turn **slot** with `O_EXCL` under `trekresearch-loop-claims/`, so the bound survives several callers deciding at once — counting the ledger and then appending is read-then-write, and Phase 4.5/5 can spawn several agents in one message. Budget = `TREKRESEARCH_MAX_CONV_TURNS` (default `3`, invalid values fall back to `3`) × `maxDimensions` (8, `settings.json:16`). Grants 0 unless `VOYAGE_STORM_ENABLED=1`, which is also the second condition on Phase 4.5's skip-guard (that phase does not call this module): unset, **both** STORM phases are inert. `resolveDataRoot()` is the single root for everything the loop writes — `CLAUDE_PLUGIN_DATA` when the harness sets it, `~/.claude/voyage` when it does not (it is empty in the Bash tool's process env, which is where the loop actually runs); the cap hook resolves through the same function, so writer and reader cannot disagree. A ledger that cannot be **written** denies the turn, and one that exists but cannot be **read** denies it too — only `ENOENT` counts as zero turns spent, that being the legitimate first-turn state (fail-closed — the opposite of `event-emit.mjs`, which is telemetry and must never block). The exported `readLedger()` is the single counting rule; the cap hook calls it rather than keeping a private copy. CLI shim: `node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E`
|
- `lib/util/research-loop-cap.mjs` — stateful, **default-off** turn budget for the `/trekresearch` bounded conversation loop. `allowTurn()` derives the used-turn count from its own append-only JSONL ledger; it never asks the caller how many turns it has spent, because a cap that does is not a cap. Each grant first claims a turn **slot** with `O_EXCL` under `trekresearch-loop-claims/`, so the bound survives several callers deciding at once — counting the ledger and then appending is read-then-write, and Phase 4.5/5 can spawn several agents in one message. Budget = `TREKRESEARCH_MAX_CONV_TURNS` (default `3`, invalid values fall back to `3`) × `maxDimensions` (8, `settings.json:16`). **Default-off:** grants 0 unless `VOYAGE_STORM_ENABLED=1`, which is also the second condition on Phase 4.5's skip-guard (that phase does not call this module): unset, **both** STORM phases are inert. `resolveDataRoot()` is the single root for everything the loop writes — `CLAUDE_PLUGIN_DATA` when the harness sets it, `~/.claude/voyage` when it does not (it is empty in the Bash tool's process env, which is where the loop actually runs); the cap hook resolves through the same function, so writer and reader cannot disagree. A ledger that cannot be **written** denies the turn, and one that exists but cannot be **read** denies it too — only `ENOENT` counts as zero turns spent, that being the legitimate first-turn state (fail-closed — the opposite of `event-emit.mjs`, which is telemetry and must never block). The exported `readLedger()` is the single counting rule; the cap hook calls it rather than keeping a private copy. CLI shim: `node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E`
|
||||||
- `lib/validators/query-privacy-gate.mjs` — gates **every** outbound research query before it leaves the machine; the hard-block tier (secret-shaped strings) is not operator-overridable, so a query that trips it must be reformulated rather than forced through. CLI shim: `node lib/validators/query-privacy-gate.mjs "<query>"`
|
- `lib/validators/query-privacy-gate.mjs` — gates **every** outbound research query before it leaves the machine; the hard-block tier (secret-shaped strings) is not operator-overridable, so a query that trips it must be reformulated rather than forced through. CLI shim: `node lib/validators/query-privacy-gate.mjs "<query>"`
|
||||||
|
|
||||||
Wiring points (replaces previous prose-grep instructions):
|
Wiring points (replaces previous prose-grep instructions):
|
||||||
|
|
@ -33,7 +33,7 @@ Doc-consistency test at `tests/lib/doc-consistency.test.mjs` pins agent-table co
|
||||||
|
|
||||||
`hooks/scripts/post-bash-stats.mjs` (PostToolUse, CC v2.1.97+) appends `duration_ms` for each Bash call into `${CLAUDE_PLUGIN_DATA}/trekexecute-stats.jsonl`. Useful for finding long-running verify or checkpoint commands.
|
`hooks/scripts/post-bash-stats.mjs` (PostToolUse, CC v2.1.97+) appends `duration_ms` for each Bash call into `${CLAUDE_PLUGIN_DATA}/trekexecute-stats.jsonl`. Useful for finding long-running verify or checkpoint commands.
|
||||||
|
|
||||||
`hooks/scripts/pre-agent-cap.mjs` (PreToolUse on `WebSearch|WebFetch|Task`) enforces the `/trekresearch` Phase 5 loop bound in the harness, so the cap is not merely prose the model is asked to obey. It counts spent turns read-only from the append-only ledger `research-loop-cap.mjs` writes, and denies (exit 2) once the budget is gone. Scope key = `session_id` + a marker file only the loop writes (`<resolveDataRoot()>/trekresearch-loop-scope/<session_id>.json`, the same root the ledger uses); without a marker the hook allows unconditionally, which is what keeps a globally-wired `PreToolUse` hook from over-blocking unrelated sessions. Stale markers auto-reset on a TTL and `VOYAGE_DISABLE_CAP_HOOK=1` is the kill switch. Defence in depth only — `lib/util/research-loop-cap.mjs` must stay correct if the hook stops firing (see `docs/spike-pretooluse-subagent-reach.md`).
|
`hooks/scripts/pre-agent-cap.mjs` (PreToolUse on `WebSearch|WebFetch|Task`) enforces the `/trekresearch` Phase 5 loop bound in the harness, so the cap is not merely prose the model is asked to obey. It counts spent turns read-only from the append-only ledger `research-loop-cap.mjs` writes, through that module's own exported `readLedger()`. It denies (exit 2) once the budget **gate has denied a turn** — the primitive records its denials as tombstones, and the tombstone is the boundary rather than the count, because `allowTurn()` appends before the turn runs and so the final granted turn already shows `budget` records. A ledger showing more granted turns than the budget denies too, as a backstop. Being in scope but unable to count the ledger also denies: a budget control that cannot count must not grant. Scope key = `session_id` + a marker file only the loop writes (`<resolveDataRoot()>/trekresearch-loop-scope/<session_id>.json`, the same root the ledger uses); without a marker the hook allows unconditionally, which is what keeps a globally-wired `PreToolUse` hook from over-blocking unrelated sessions. Stale markers auto-reset on a TTL and `VOYAGE_DISABLE_CAP_HOOK=1` is the kill switch. Defence in depth only — `lib/util/research-loop-cap.mjs` must stay correct if the hook stops firing (see `docs/spike-pretooluse-subagent-reach.md`).
|
||||||
|
|
||||||
`hooks/scripts/post-compact-flush.mjs` (PostCompact event, v3.4.0) re-injects `.session-state.local.json` after context compaction so multi-session work survives a compaction boundary. Companion to `pre-compact-flush.mjs` (which writes the state file before compaction); together they form the rehydrate cycle that keeps `/trekcontinue` reliable across long-running multi-session work.
|
`hooks/scripts/post-compact-flush.mjs` (PostCompact event, v3.4.0) re-injects `.session-state.local.json` after context compaction so multi-session work survives a compaction boundary. Companion to `pre-compact-flush.mjs` (which writes the state file before compaction); together they form the rehydrate cycle that keeps `/trekcontinue` reliable across long-running multi-session work.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -128,9 +128,9 @@ const ledgerPath = resolveLedgerPath(env);
|
||||||
// list under settings.json:16's maxDimensions ceiling.
|
// list under settings.json:16's maxDimensions ceiling.
|
||||||
const budget = resolveMaxConvTurns(env) * MAX_TOTAL_DIMENSIONS;
|
const budget = resolveMaxConvTurns(env) * MAX_TOTAL_DIMENSIONS;
|
||||||
|
|
||||||
let used;
|
let ledger;
|
||||||
try {
|
try {
|
||||||
used = readLedger(ledgerPath, marker.runId).granted;
|
ledger = readLedger(ledgerPath, marker.runId);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
deny(
|
deny(
|
||||||
` Run ${marker.runId} is in scope, but its turn ledger could not be read:\n` +
|
` Run ${marker.runId} is in scope, but its turn ledger could not be read:\n` +
|
||||||
|
|
@ -140,13 +140,42 @@ try {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (used >= budget) {
|
const toolLine =
|
||||||
|
` Tool: ${input?.tool_name ?? 'unknown'}${input?.agent_type ? ` (agent: ${input.agent_type})` : ''}\n`;
|
||||||
|
|
||||||
|
// 8. The boundary is the TOMBSTONE, not the count.
|
||||||
|
//
|
||||||
|
// allowTurn() appends before the turn runs, so during the final granted turn the
|
||||||
|
// ledger already holds `budget` records. Denying at `granted >= budget` blocked
|
||||||
|
// that turn's own tool calls — the primitive granted B turns and this hook
|
||||||
|
// permitted B-1 — and it forced every exhausted run out through an exit-2 tool
|
||||||
|
// denial rather than the graceful "cap exhausted" exit, the only exit the prose
|
||||||
|
// at commands/trekresearch.md teaches the model to handle.
|
||||||
|
//
|
||||||
|
// Moving the boundary to `granted > budget` alone would have made this hook
|
||||||
|
// unable to fire at all once the claim mechanism made a breached ledger
|
||||||
|
// impossible — a deny branch that cannot be reached is a dead security claim,
|
||||||
|
// not a backstop. So the primitive records its own denials, and the case this
|
||||||
|
// hook exists for is the one it now catches: the gate said no and a tool call
|
||||||
|
// arrived anyway.
|
||||||
|
if (ledger.exhausted > 0) {
|
||||||
deny(
|
deny(
|
||||||
` Run ${marker.runId} has spent ${used}/${budget} loop turns.\n` +
|
` Run ${marker.runId} was already denied a turn by the budget gate\n` +
|
||||||
` Tool: ${input?.tool_name ?? 'unknown'}${input?.agent_type ? ` (agent: ${input.agent_type})` : ''}\n` +
|
` (${ledger.granted}/${budget} loop turns spent), and this call came after it.\n` +
|
||||||
|
toolLine +
|
||||||
` Remaining gaps belong in the brief as open questions, not in another turn.\n` +
|
` Remaining gaps belong in the brief as open questions, not in another turn.\n` +
|
||||||
` Raise TREKRESEARCH_MAX_CONV_TURNS deliberately, or set VOYAGE_DISABLE_CAP_HOOK=1.`,
|
` Raise TREKRESEARCH_MAX_CONV_TURNS deliberately, or set VOYAGE_DISABLE_CAP_HOOK=1.`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 9. Backstop for a ledger that exceeded the bound however it managed to.
|
||||||
|
if (ledger.granted > budget) {
|
||||||
|
deny(
|
||||||
|
` Run ${marker.runId} shows ${ledger.granted} granted turns against a budget of ${budget}.\n` +
|
||||||
|
toolLine +
|
||||||
|
` The ledger has been breached; the loop is over regardless of cause.\n` +
|
||||||
|
` Raise TREKRESEARCH_MAX_CONV_TURNS deliberately, or set VOYAGE_DISABLE_CAP_HOOK=1.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
allow();
|
allow();
|
||||||
|
|
|
||||||
|
|
@ -144,9 +144,15 @@ function claimSlot(claimDir, runId, slot) {
|
||||||
* function. A reader and a writer with private copies of the counting rule are
|
* function. A reader and a writer with private copies of the counting rule are
|
||||||
* how a hook ends up enforcing a different bound than the gate it backs.
|
* how a hook ends up enforcing a different bound than the gate it backs.
|
||||||
*
|
*
|
||||||
|
* Two counts, deliberately separate. `granted` is turns handed out. `exhausted`
|
||||||
|
* is tombstones — records this gate wrote when it DENIED a turn. A tombstone is
|
||||||
|
* not a turn and must never consume budget; it exists so the PreToolUse hook can
|
||||||
|
* tell "turn B is in flight" (granted == budget, no tombstone) apart from "the
|
||||||
|
* gate already said no and something kept going" (tombstone present).
|
||||||
|
*
|
||||||
* @param {string} ledgerPath
|
* @param {string} ledgerPath
|
||||||
* @param {string} runId
|
* @param {string} runId
|
||||||
* @returns {{granted: number}}
|
* @returns {{granted: number, exhausted: number}}
|
||||||
* @throws when the ledger exists but cannot be read
|
* @throws when the ledger exists but cannot be read
|
||||||
*/
|
*/
|
||||||
export function readLedger(ledgerPath, runId) {
|
export function readLedger(ledgerPath, runId) {
|
||||||
|
|
@ -154,19 +160,36 @@ export function readLedger(ledgerPath, runId) {
|
||||||
try {
|
try {
|
||||||
text = readFileSync(ledgerPath, 'utf-8');
|
text = readFileSync(ledgerPath, 'utf-8');
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
if (e && e.code === 'ENOENT') return { granted: 0 };
|
if (e && e.code === 'ENOENT') return { granted: 0, exhausted: 0 };
|
||||||
const err = new Error(`ledger unreadable at ${ledgerPath}: ${e.message}`);
|
const err = new Error(`ledger unreadable at ${ledgerPath}: ${e.message}`);
|
||||||
err.code = 'VOYAGE_LEDGER_UNREADABLE';
|
err.code = 'VOYAGE_LEDGER_UNREADABLE';
|
||||||
throw err;
|
throw err;
|
||||||
}
|
}
|
||||||
let granted = 0;
|
let granted = 0;
|
||||||
|
let exhausted = 0;
|
||||||
for (const line of text.split('\n')) {
|
for (const line of text.split('\n')) {
|
||||||
if (!line) continue;
|
if (!line) continue;
|
||||||
try {
|
try {
|
||||||
if (JSON.parse(line).runId === runId) granted++;
|
const rec = JSON.parse(line);
|
||||||
|
if (rec.runId !== runId) continue;
|
||||||
|
if (rec.exhausted === true) exhausted++;
|
||||||
|
else granted++;
|
||||||
} catch { /* skip malformed lines */ }
|
} catch { /* skip malformed lines */ }
|
||||||
}
|
}
|
||||||
return { granted };
|
return { granted, exhausted };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Record that this run has been denied a turn for budget.
|
||||||
|
*
|
||||||
|
* Best effort on purpose: the denial itself is already the correct answer, so a
|
||||||
|
* ledger that cannot take the tombstone must not turn a denial into a grant. The
|
||||||
|
* tombstone only strengthens the harness-level backstop.
|
||||||
|
*/
|
||||||
|
function markExhausted(ledgerPath, runId, now) {
|
||||||
|
try {
|
||||||
|
appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, exhausted: true }) + '\n');
|
||||||
|
} catch { /* best effort — see above */ }
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
@ -202,12 +225,19 @@ export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
|
||||||
const budget = maxConvTurns * MAX_TOTAL_DIMENSIONS;
|
const budget = maxConvTurns * MAX_TOTAL_DIMENSIONS;
|
||||||
|
|
||||||
const ledgerPath = resolveLedgerPath(env);
|
const ledgerPath = resolveLedgerPath(env);
|
||||||
let used;
|
let ledger;
|
||||||
try {
|
try {
|
||||||
used = readLedger(ledgerPath, runId).granted;
|
ledger = readLedger(ledgerPath, runId);
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
return { ok: false, used: 0, budget, reason: `ledger-read-failed: ${e.message}` };
|
return { ok: false, used: 0, budget, reason: `ledger-read-failed: ${e.message}` };
|
||||||
}
|
}
|
||||||
|
const used = ledger.granted;
|
||||||
|
|
||||||
|
// Already tombstoned: this run is over. Short-circuit so a hammered gate
|
||||||
|
// neither walks every slot again nor appends a second tombstone.
|
||||||
|
if (ledger.exhausted > 0) {
|
||||||
|
return { ok: false, used, budget, reason: 'budget_exhausted' };
|
||||||
|
}
|
||||||
// Claim a turn SLOT before spending anything. The ledger count only says
|
// Claim a turn SLOT before spending anything. The ledger count only says
|
||||||
// where to start looking; the claim is what makes the grant exclusive. Slot
|
// where to start looking; the claim is what makes the grant exclusive. Slot
|
||||||
// numbers are bounded by `budget`, and each can be created exactly once, so
|
// numbers are bounded by `budget`, and each can be created exactly once, so
|
||||||
|
|
@ -226,6 +256,7 @@ export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
|
||||||
return { ok: false, used, budget, reason: `claim-failed: ${e.message}` };
|
return { ok: false, used, budget, reason: `claim-failed: ${e.message}` };
|
||||||
}
|
}
|
||||||
if (!claimed) {
|
if (!claimed) {
|
||||||
|
markExhausted(ledgerPath, runId, now);
|
||||||
return { ok: false, used, budget, reason: 'budget_exhausted' };
|
return { ok: false, used, budget, reason: 'budget_exhausted' };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -33,17 +33,20 @@ const RUN_ID = 'run-xyz789';
|
||||||
* Build a throwaway CLAUDE_PLUGIN_DATA dir holding a scope marker for
|
* Build a throwaway CLAUDE_PLUGIN_DATA dir holding a scope marker for
|
||||||
* `sessionId` and `turns` ledger entries for RUN_ID.
|
* `sessionId` and `turns` ledger entries for RUN_ID.
|
||||||
*/
|
*/
|
||||||
function fixture({ turns = 0, sessionId = SESSION, startedAt = new Date() } = {}) {
|
function fixture({ turns = 0, sessionId = SESSION, startedAt = new Date(), exhausted = false } = {}) {
|
||||||
const dir = mkdtempSync(join(tmpdir(), 'voyage-cap-'));
|
const dir = mkdtempSync(join(tmpdir(), 'voyage-cap-'));
|
||||||
mkdirSync(join(dir, 'trekresearch-loop-scope'), { recursive: true });
|
mkdirSync(join(dir, 'trekresearch-loop-scope'), { recursive: true });
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(dir, 'trekresearch-loop-scope', `${sessionId}.json`),
|
join(dir, 'trekresearch-loop-scope', `${sessionId}.json`),
|
||||||
JSON.stringify({ runId: RUN_ID, startedAt: startedAt.toISOString() }),
|
JSON.stringify({ runId: RUN_ID, startedAt: startedAt.toISOString() }),
|
||||||
);
|
);
|
||||||
const ledger = Array.from({ length: turns }, (_, i) =>
|
const lines = Array.from({ length: turns }, (_, i) =>
|
||||||
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, dimension: `d${i}`, effort: 'high' }),
|
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, dimension: `d${i}`, effort: 'high', slot: i + 1 }),
|
||||||
).join('\n');
|
);
|
||||||
writeFileSync(join(dir, 'trekresearch-loop-ledger.jsonl'), ledger ? ledger + '\n' : '');
|
// The tombstone research-loop-cap.mjs appends when it denies a turn for
|
||||||
|
// budget. Its presence is what tells this hook "the gate already said no".
|
||||||
|
if (exhausted) lines.push(JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, exhausted: true }));
|
||||||
|
writeFileSync(join(dir, 'trekresearch-loop-ledger.jsonl'), lines.length ? lines.join('\n') + '\n' : '');
|
||||||
return dir;
|
return dir;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -65,8 +68,8 @@ function searchInput(sessionId = SESSION) {
|
||||||
// -----------------------------------------------------------------------
|
// -----------------------------------------------------------------------
|
||||||
// DENY — budget spent
|
// DENY — budget spent
|
||||||
// -----------------------------------------------------------------------
|
// -----------------------------------------------------------------------
|
||||||
test('pre-agent-cap DENIES a loop turn once the ledger shows the budget spent', async () => {
|
test('pre-agent-cap DENIES once the budget gate has denied a turn (tombstone present)', async () => {
|
||||||
const dir = fixture({ turns: BUDGET });
|
const dir = fixture({ turns: BUDGET, exhausted: true });
|
||||||
const { code, stderr } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
const { code, stderr } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
||||||
...CAPPED_ENV,
|
...CAPPED_ENV,
|
||||||
CLAUDE_PLUGIN_DATA: dir,
|
CLAUDE_PLUGIN_DATA: dir,
|
||||||
|
|
@ -76,7 +79,7 @@ test('pre-agent-cap DENIES a loop turn once the ledger shows the budget spent',
|
||||||
assert.match(stderr, new RegExp(`${BUDGET}`), 'stderr must state the budget');
|
assert.match(stderr, new RegExp(`${BUDGET}`), 'stderr must state the budget');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('pre-agent-cap DENIES above the budget too (used > budget)', async () => {
|
test('pre-agent-cap DENIES above the budget too (a breached ledger, whatever caused it)', async () => {
|
||||||
const dir = fixture({ turns: BUDGET + 5 });
|
const dir = fixture({ turns: BUDGET + 5 });
|
||||||
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
||||||
...CAPPED_ENV,
|
...CAPPED_ENV,
|
||||||
|
|
@ -86,7 +89,15 @@ test('pre-agent-cap DENIES above the budget too (used > budget)', async () => {
|
||||||
});
|
});
|
||||||
|
|
||||||
// -----------------------------------------------------------------------
|
// -----------------------------------------------------------------------
|
||||||
// ALLOW — under the cap
|
// ALLOW — under the cap, and ON the cap.
|
||||||
|
//
|
||||||
|
// allowTurn appends BEFORE the turn runs, so during the FINAL granted turn the
|
||||||
|
// ledger already holds `budget` records. Denying at `used >= budget` therefore
|
||||||
|
// blocked that turn's own tool calls: the primitive granted B turns, the
|
||||||
|
// harness permitted B-1, and an exhausted run always ended through an exit-2
|
||||||
|
// denial rather than the graceful "cap exhausted" exit the prose defines. The
|
||||||
|
// boundary belongs one turn later, and the tombstone above — not the count —
|
||||||
|
// is what marks a run actually finished.
|
||||||
// -----------------------------------------------------------------------
|
// -----------------------------------------------------------------------
|
||||||
test('pre-agent-cap ALLOWS a loop turn under the budget', async () => {
|
test('pre-agent-cap ALLOWS a loop turn under the budget', async () => {
|
||||||
const dir = fixture({ turns: BUDGET - 1 });
|
const dir = fixture({ turns: BUDGET - 1 });
|
||||||
|
|
@ -97,6 +108,18 @@ test('pre-agent-cap ALLOWS a loop turn under the budget', async () => {
|
||||||
assert.strictEqual(code, 0);
|
assert.strictEqual(code, 0);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('pre-agent-cap ALLOWS the FINAL granted turn — its own record is already on the ledger', async () => {
|
||||||
|
const dir = fixture({ turns: BUDGET });
|
||||||
|
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
||||||
|
...CAPPED_ENV,
|
||||||
|
CLAUDE_PLUGIN_DATA: dir,
|
||||||
|
});
|
||||||
|
assert.strictEqual(
|
||||||
|
code, 0,
|
||||||
|
'turn B is granted and in flight; denying it makes the harness permit B-1 turns and forces the wrong exit',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
// -----------------------------------------------------------------------
|
// -----------------------------------------------------------------------
|
||||||
// DOES NOT OVER-BLOCK — the property that keeps this hook safe to wire
|
// DOES NOT OVER-BLOCK — the property that keeps this hook safe to wire
|
||||||
// globally. A broken PreToolUse hook would brick every session on the box.
|
// globally. A broken PreToolUse hook would brick every session on the box.
|
||||||
|
|
@ -223,9 +246,11 @@ test('pre-agent-cap enforces via the fallback root when CLAUDE_PLUGIN_DATA is ab
|
||||||
);
|
);
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
join(root, 'trekresearch-loop-ledger.jsonl'),
|
join(root, 'trekresearch-loop-ledger.jsonl'),
|
||||||
Array.from({ length: BUDGET }, (_, i) =>
|
[
|
||||||
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, dimension: `d${i}`, effort: 'high' }),
|
...Array.from({ length: BUDGET }, (_, i) =>
|
||||||
).join('\n') + '\n',
|
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, dimension: `d${i}`, effort: 'high', slot: i + 1 })),
|
||||||
|
JSON.stringify({ ts: new Date().toISOString(), runId: RUN_ID, exhausted: true }),
|
||||||
|
].join('\n') + '\n',
|
||||||
);
|
);
|
||||||
|
|
||||||
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
const { code } = await runHookWithEnv(CAP_HOOK, searchInput(), {
|
||||||
|
|
|
||||||
|
|
@ -415,17 +415,77 @@ test('allowTurn — parallel processes at the boundary cannot exceed the budget'
|
||||||
runShimAsync(['--run-id', 'r-par', '--dimension', `p${i}`, '--effort', 'high'], env),
|
runShimAsync(['--run-id', 'r-par', '--dimension', `p${i}`, '--effort', 'high'], env),
|
||||||
),
|
),
|
||||||
);
|
);
|
||||||
const granted = results.filter((r) => r.code === 0).length;
|
const winners = results.filter((r) => r.code === 0).length;
|
||||||
assert.equal(granted, 1, `exactly one of six concurrent callers may take the last slot, got ${granted}`);
|
assert.equal(winners, 1, `exactly one of six concurrent callers may take the last slot, got ${winners}`);
|
||||||
|
|
||||||
const lines = readFileSync(join(dir, 'trekresearch-loop-ledger.jsonl'), 'utf-8')
|
// Granted turns, not raw lines: the five denied callers also record
|
||||||
.split('\n').filter(Boolean);
|
// exhaustion tombstones, and a tombstone is not a turn.
|
||||||
assert.equal(lines.length, 8, `the ledger must never exceed the budget, got ${lines.length}`);
|
const { granted } = readLedger(join(dir, 'trekresearch-loop-ledger.jsonl'), 'r-par');
|
||||||
|
assert.equal(granted, 8, `granted turns must never exceed the budget, got ${granted}`);
|
||||||
} finally {
|
} finally {
|
||||||
rmSync(dir, { recursive: true, force: true });
|
rmSync(dir, { recursive: true, force: true });
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ---- the exhaustion tombstone -----------------------------------------------
|
||||||
|
//
|
||||||
|
// allowTurn appends BEFORE the turn runs, so during granted turn N the ledger
|
||||||
|
// holds N records. The hook denied at `used >= budget`, which blocked every
|
||||||
|
// tool call of the FINAL granted turn: the primitive granted B turns and the
|
||||||
|
// harness permitted B-1. Worse, an exhausted run then always terminated through
|
||||||
|
// an exit-2 tool denial instead of the graceful "cap exhausted" exit at
|
||||||
|
// commands/trekresearch.md, which is the only exit the prose teaches.
|
||||||
|
//
|
||||||
|
// Letting the hook allow at `used == budget` fixes the count but would leave it
|
||||||
|
// unable to catch the one case it exists for — the loop consults the gate, is
|
||||||
|
// denied, and issues the tool call anyway. So the denial itself becomes a
|
||||||
|
// record: a tombstone the hook can see.
|
||||||
|
|
||||||
|
test('allowTurn — denying for budget writes an exhaustion tombstone', () => {
|
||||||
|
withTmpDataDir((dir) => {
|
||||||
|
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
|
||||||
|
for (let i = 0; i < 8; i++) {
|
||||||
|
assert.equal(allowTurn({ runId: 'r-tomb', dimension: `d${i}`, effort: 'high' }, { env }).ok, true);
|
||||||
|
}
|
||||||
|
const ledgerPath = join(dir, 'trekresearch-loop-ledger.jsonl');
|
||||||
|
assert.equal(readLedger(ledgerPath, 'r-tomb').exhausted, 0, 'no tombstone before the gate has denied anything');
|
||||||
|
|
||||||
|
const denied = allowTurn({ runId: 'r-tomb', dimension: 'd0', effort: 'high' }, { env });
|
||||||
|
assert.equal(denied.ok, false);
|
||||||
|
assert.equal(denied.reason, 'budget_exhausted');
|
||||||
|
|
||||||
|
const after = readLedger(ledgerPath, 'r-tomb');
|
||||||
|
assert.equal(after.exhausted, 1, 'the denial must leave a record the harness gate can read');
|
||||||
|
assert.equal(after.granted, 8, 'a tombstone is not a granted turn and must not count as one');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test('allowTurn — the tombstone is written once, not once per repeated denial', () => {
|
||||||
|
withTmpDataDir((dir) => {
|
||||||
|
const env = { CLAUDE_PLUGIN_DATA: dir, VOYAGE_STORM_ENABLED: '1', TREKRESEARCH_MAX_CONV_TURNS: '1' };
|
||||||
|
for (let i = 0; i < 8; i++) allowTurn({ runId: 'r-once', dimension: `d${i}`, effort: 'high' }, { env });
|
||||||
|
for (let i = 0; i < 5; i++) allowTurn({ runId: 'r-once', dimension: 'd0', effort: 'high' }, { env });
|
||||||
|
const after = readLedger(join(dir, 'trekresearch-loop-ledger.jsonl'), 'r-once');
|
||||||
|
assert.equal(after.exhausted, 1, 'a hammered gate must not grow the ledger without bound');
|
||||||
|
assert.equal(after.granted, 8);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test('readLedger — a tombstone is reported separately and never as a granted turn', () => {
|
||||||
|
withTmpDataDir((dir) => {
|
||||||
|
const p = join(dir, 'trekresearch-loop-ledger.jsonl');
|
||||||
|
writeFileSync(p, [
|
||||||
|
JSON.stringify({ runId: 'r1', dimension: 'd1', slot: 1 }),
|
||||||
|
JSON.stringify({ runId: 'r1', exhausted: true }),
|
||||||
|
JSON.stringify({ runId: 'other', exhausted: true }),
|
||||||
|
].join('\n') + '\n');
|
||||||
|
const l = readLedger(p, 'r1');
|
||||||
|
assert.equal(l.granted, 1);
|
||||||
|
assert.equal(l.exhausted, 1);
|
||||||
|
assert.equal(readLedger(p, 'other').granted, 0, 'another run’s tombstone is not a granted turn either');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
// ---- (g) shim contract --------------------------------------------------------
|
// ---- (g) shim contract --------------------------------------------------------
|
||||||
|
|
||||||
test('CLI shim — grants and exits 0 when enabled + high effort + budget available', () => {
|
test('CLI shim — grants and exits 0 when enabled + high effort + budget available', () => {
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue