fix(research-loop-cap): claim each turn slot with O_EXCL so the bound survives concurrency

countTurns-then-appendFileSync is read-then-write. N callers that all
observe used == budget-1 all decide to grant, and the bound is exceeded
by N-1. The comment above allowTurn asserted "Append-only: never
read-modify-write" and named the concurrent case - Phase 4.5/5 may spawn
several agents in a single message - as the reason it had to be. The
decision path was exactly what the comment denied, so the concurrency
claim had nothing under it.

Each grant now creates <data root>/trekresearch-loop-claims/<runId>-<slot>.claim
with flag 'wx' (O_CREAT|O_EXCL) before appending. The kernel picks the
winner per slot, slot numbers are bounded by the budget, and each can be
created exactly once - so total grants for a run cannot exceed the budget
however many callers arrive together. The ledger count now only says
where to start looking for a free slot.

Two of the three tests are deterministic and do not race anything: they
assert the invariant directly by pre-creating claims, including the state
a mid-append competitor leaves behind (ledger 7, slots 1-8 claimed, budget
8 -> deny). That matters because the third test - six real concurrent shim
processes at the boundary - passed even BEFORE the fix, since process
startup jitter serialised them. A race test that passes by luck is not
evidence, so it ships as a real-world regression guard next to the two
that are.

Stated rather than left to be discovered: claim files are empty, at most
budget per run, and never cleaned - the same standing as the ledger, which
also grows for the life of the data root. Reusing a runId across runs, or
two runIds colliding after filename sanitisation, both deny a turn, which
is the safe direction for a budget control.

Review finding 3994491ef1fdba6e0e3645b5b713cbdbdeb2b328 (MINOR).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuGhWAbWyRFBFeemfhxoVv

lib/util/research-loop-cap.mjs

@@ -8,7 +8,9 @@
 //       leaves the mechanism off, and adopt is flipping this one constant.
 //   (b) The cap counts itself — allowTurn() derives used-turn count from an
 //       append-only JSONL ledger, never from a caller-supplied number. A cap
-//       that asks the caller how many turns it has used is not a cap.
+//       that asks the caller how many turns it has used is not a cap. Each
+//       grant additionally claims a turn SLOT with O_EXCL, so the bound holds
+//       when several callers decide at once instead of only when they queue.
 //   (c) Correct size bound — worst case is max_conv_turns × max_total_dimensions,
 //       where max_total_dimensions is the WHOLE list (interview + discovered)
 //       under settings.json:16's cap of 8 — not × discovered-only.
@@ -31,7 +33,7 @@
 //   node lib/util/research-loop-cap.mjs --run-id ID --dimension D --effort E
 //   → JSON: { ok, used, budget, reason? }   (exit 0 = granted, exit 1 = denied)
 
-import { existsSync, mkdirSync, appendFileSync, readFileSync } from 'node:fs';
+import { existsSync, mkdirSync, appendFileSync, readFileSync, writeFileSync } from 'node:fs';
 import { dirname, join } from 'node:path';
 import { homedir } from 'node:os';
 
@@ -39,6 +41,7 @@ export const MAX_CONV_TURNS = 3;
 export const MAX_TOTAL_DIMENSIONS = 8; // settings.json:16 maxDimensions — whole list, not discovered-only
 
 const LEDGER_FILENAME = 'trekresearch-loop-ledger.jsonl';
+const CLAIM_DIRNAME = 'trekresearch-loop-claims';
 
 export function isStormEnabled(env = process.env) {
   return env.VOYAGE_STORM_ENABLED === '1';
@@ -80,6 +83,48 @@ export function resolveLedgerPath(env = process.env) {
   return join(resolveDataRoot(env), LEDGER_FILENAME);
 }
 
+/**
+ * Where the per-turn claim files live. A claim is the ATOMIC record that a turn
+ * slot is taken; the ledger is the readable record of what that turn was for.
+ *
+ * The claim exists because the ledger alone cannot bound the loop. Counting the
+ * ledger and then appending is read-then-write: N callers that all observe
+ * `used == budget - 1` all decide to grant, and the bound is exceeded by N-1 —
+ * precisely the concurrent case (several agents spawned in one message) that
+ * allowTurn's own comment named as the reason it had to be append-only.
+ *
+ * Claim files are empty, at most `budget` per run, and never cleaned up — the
+ * same standing as the ledger itself, which also grows for the life of the data
+ * root. Two consequences worth stating rather than discovering: reusing a
+ * runId across runs finds its slots already taken and denies, and two runIds
+ * that collide after filename sanitisation block each other. Both err toward
+ * denying a turn, which is the safe direction for a budget control.
+ */
+export function resolveClaimDir(env = process.env) {
+  return join(resolveDataRoot(env), CLAIM_DIRNAME);
+}
+
+function claimFileName(runId, slot) {
+  return `${String(runId).replace(/[^A-Za-z0-9._-]/g, '_')}-${slot}.claim`;
+}
+
+/**
+ * Try to take turn slot `slot` for `runId`. `wx` is O_CREAT|O_EXCL: the kernel
+ * decides the winner, so exactly one caller can ever create a given slot file.
+ *
+ * @returns {boolean} true when this caller took the slot, false when it was already taken
+ * @throws on any IO error other than EEXIST — the caller turns that into a denial
+ */
+function claimSlot(claimDir, runId, slot) {
+  try {
+    writeFileSync(join(claimDir, claimFileName(runId, slot)), '', { flag: 'wx' });
+    return true;
+  } catch (e) {
+    if (e && e.code === 'EEXIST') return false;
+    throw e;
+  }
+}
+
 /**
  * Read one run's turn count off the append-only ledger.
  *
@@ -125,10 +170,15 @@ export function readLedger(ledgerPath, runId) {
 }
 
 /**
- * Decide whether one more research-loop turn may run. Append-only: never
- * read-modify-write, because Phase 4.5/5 may spawn multiple agents in a
- * single message and a read-modify-write counter would lose concurrent
- * grants.
+ * Decide whether one more research-loop turn may run.
+ *
+ * Phase 4.5/5 may spawn several agents in a single message, so the decision has
+ * to survive concurrent callers. It does that by CLAIMING a turn slot with
+ * O_EXCL (see claimSlot) and only then appending to the ledger. The comment
+ * that used to sit here asserted "append-only: never read-modify-write" as if
+ * appending were itself the concurrency guarantee — but the decision path was
+ * count-then-append, which is read-then-write, so the claim was unsupported by
+ * the code beneath it. The kernel now picks the winner for each slot.
  *
  * @param {{runId: string, dimension: string, effort: string}} args
  * @param {{env?: object, now?: Date}} [opts]
@@ -158,19 +208,36 @@ export function allowTurn({ runId, dimension, effort } = {}, opts = {}) {
   } catch (e) {
     return { ok: false, used: 0, budget, reason: `ledger-read-failed: ${e.message}` };
   }
-  if (used >= budget) {
+  // Claim a turn SLOT before spending anything. The ledger count only says
+  // where to start looking; the claim is what makes the grant exclusive. Slot
+  // numbers are bounded by `budget`, and each can be created exactly once, so
+  // the total number of grants for a run can never exceed the budget however
+  // many callers arrive at once.
+  const claimDir = resolveClaimDir(env);
+  let slot = used + 1;
+  let claimed = false;
+  try {
+    mkdirSync(claimDir, { recursive: true });
+    while (slot <= budget) {
+      if (claimSlot(claimDir, runId, slot)) { claimed = true; break; }
+      slot++;
+    }
+  } catch (e) {
+    return { ok: false, used, budget, reason: `claim-failed: ${e.message}` };
+  }
+  if (!claimed) {
     return { ok: false, used, budget, reason: 'budget_exhausted' };
   }
 
   try {
     const dir = dirname(ledgerPath);
     if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
-    appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, dimension, effort }) + '\n');
+    appendFileSync(ledgerPath, JSON.stringify({ ts: now.toISOString(), runId, dimension, effort, slot }) + '\n');
   } catch (e) {
     return { ok: false, used, budget, reason: `ledger-write-failed: ${e.message}` };
   }
 
-  return { ok: true, used: used + 1, budget };
+  return { ok: true, used: slot, budget };
 }
 
 // ---- CLI shim ----------------------------------------------------------------