fix: the trial's mechanical faults -- approval label, import path, stats, annotate path, red kind

voyage-lekerepo end-to-end trial (2026-10-06), report points 8, 10, 12, 15, 16: - approval-log: «Approve (Recommended)» / «Godkjenn (anbefalt)» is the approval it labels; the suffix never turns a hollow answer into a yes. - trekexecute Phase 8: the state-file block imports atomic-write through ${CLAUDE_PLUGIN_ROOT}; a relative import only ran with cwd = plugin root (the test runs the block from another directory). - trekreview, trekresearch, trekcontinue: stats go through stats-append like the other commands, and a failed write is reported, never skipped silently (closes the open STATE finding for all three). The schema fixture's stale line refs for these and trekplan are re-measured. - annotate prints an absolute path, so file://{$ANNOT_HTML} is a live link. - red-proof: `red_kind` = load-error | test-failure on a proven red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:53:15 +02:00 · 2026-10-08 06:53:15 +02:00 · 81dc5af680
commit 81dc5af680
parent 609eb35015
9 changed files with 191 additions and 24 deletions
--- a/commands/trekcontinue.md
+++ b/commands/trekcontinue.md
@ -285,13 +285,21 @@ disk. Cannot continue automatically.` and exit. Do not guess.

 ## Phase 5 — Stats tracking

-Append a one-line JSON record to `${CLAUDE_PLUGIN_DATA}/trekcontinue-stats.jsonl`
-if the env var is set; silently skip otherwise.
+Append one record to `trekcontinue-stats.jsonl` in the plugin data directory.
+Run it exactly like this. The record goes on stdin, and the writer resolves
+the directory: `CLAUDE_PLUGIN_DATA`, which the Bash tool env does not carry,
+else the directory `scripts/yardstick.mjs` reads.

-```json
+```bash
+node ${CLAUDE_PLUGIN_ROOT}/lib/stats/stats-append.mjs trekcontinue <<'JSON'
 {"ts":"<iso-8601>","project":"<project>","next_session_label":"<label>","status":"<status>"}
+JSON
 ```

+The writer prints `{"written": true, ...}`. On `"written": false` (exit 1),
+print its `reason` in one line and continue. A tracking failure never blocks
+the main workflow, but it is reported, not skipped silently.
+
 ## Profile (v4.1) — inheritance from plan-frontmatter

 Accepts `--profile <name>` where `<name>` is `economy`, `balanced`, `premium`,
--- a/commands/trekexecute.md
+++ b/commands/trekexecute.md
@ -2164,7 +2164,7 @@ ESM inline block so state-file + prompt-file writes succeed or fail together:
 node --input-type=module -e "
 import path from 'node:path';
 import { writeFileSync } from 'node:fs';
-import { atomicWriteJson } from './lib/util/atomic-write.mjs';
+import { atomicWriteJson } from '${CLAUDE_PLUGIN_ROOT}/lib/util/atomic-write.mjs';
 const [, dir, briefPath, label, status] = process.argv;
 const now = new Date().toISOString();
 const stateObj = { schema_version: 1, project: dir, next_session_brief_path: briefPath, next_session_label: label, status, updated_at: now };
--- a/commands/trekresearch.md
+++ b/commands/trekresearch.md
@ -774,11 +774,13 @@ You can:

 ### Stats tracking

-Write a session record to `${CLAUDE_PLUGIN_DATA}/trekresearch-stats.jsonl`
-(create the file if it does not exist).
+Append one record to `trekresearch-stats.jsonl` in the plugin data directory.
+Run it exactly like this. The record goes on stdin, and the writer resolves
+the directory: `CLAUDE_PLUGIN_DATA`, which the Bash tool env does not carry,
+else the directory `scripts/yardstick.mjs` reads.

-Record format (one JSON line):
-```json
+```bash
+node ${CLAUDE_PLUGIN_ROOT}/lib/stats/stats-append.mjs trekresearch <<'JSON'
 {
  "ts": "{ISO-8601 timestamp}",
  "question": "{research question (first 100 chars)}",
@ -802,6 +804,7 @@ Record format (one JSON line):
  "contradictions": {N},
  "open_questions": {N}
 }
+JSON
 ```

 **The six measurement fields (v5.10).** `effort` is the grouping key — the
@ -819,7 +822,9 @@ that could are prose the exporter allowlist denies. On a standard run the loop
 never arms, so `dimensions_baseline == dimensions`, both turn counters are `0`,
 and `dimensions_baseline_preserved` is `true` (nothing touched the list).

-If `${CLAUDE_PLUGIN_DATA}` is not set or not writable, skip tracking silently.
+The writer prints `{"written": true, ...}`. On `"written": false` (exit 1),
+print its `reason` in one line and continue. A tracking failure never blocks
+the main workflow, but it is reported, not skipped silently.

 ## Profile (v4.1)

--- a/commands/trekreview.md
+++ b/commands/trekreview.md
@ -620,15 +620,20 @@ with WARN, `--json` also returns `warn_reasons` (the JSON block's
 as returned. This is the command that later stops a merge or a release on
 BLOCK; `/trekreview` reports it and never overrides it.

-Append a stats line to `${CLAUDE_PLUGIN_DATA}/trekreview-stats.jsonl`
-(create the file if it does not exist):
+Then append one record to `trekreview-stats.jsonl` in the plugin data directory.
+Run it exactly like this. The record goes on stdin, and the writer resolves
+the directory: `CLAUDE_PLUGIN_DATA`, which the Bash tool env does not carry,
+else the directory `scripts/yardstick.mjs` reads.

-```json
+```bash
+node ${CLAUDE_PLUGIN_ROOT}/lib/stats/stats-append.mjs trekreview <<'JSON'
 {"ts":"{ISO-8601}","slug":"{slug}","verdict":"BLOCK|WARN|ALLOW","counts":{"BLOCKER":N,"MAJOR":N,"MINOR":N,"SUGGESTION":N},"reviewed_files_count":N,"mode":"default|quick|validate|dry-run","duration_ms":N}
+JSON
 ```

-If `${CLAUDE_PLUGIN_DATA}` is unset or not writable, skip stats silently.
-Never let stats failures block the main workflow.
+The writer prints `{"written": true, ...}`. On `"written": false` (exit 1),
+print its `reason` in one line and continue. A tracking failure never blocks
+the main workflow, but it is reported, not skipped silently.

 **Build the operator-annotation HTML.** After stats land, run:

--- a/lib/util/approval-log.mjs
+++ b/lib/util/approval-log.mjs
@ -242,8 +242,15 @@ const HOLLOW = [
 ];

 /** @returns {{ approved: boolean, reason: 'explicit'|'empty'|'hollow'|'not-explicit' }} */
+// AskUserQuestion's own guidance puts «(Recommended)» after the label it
+// recommends, so the operator's real «Approve (Recommended)» was refused in the
+// voyage-lekerepo trial (2026-10-06). The suffix is stripped; what is left must
+// still be an explicit approval on its own.
+const RECOMMENDED_SUFFIX = /\s*\((?:recommended|anbefalt)\)\s*$/u;
+
 export function isExplicitApproval(answer) {
-  const t = String(answer ?? '').trim().toLowerCase().replace(/[\s.!]+$/u, '').replace(/\s+/g, ' ');
+  const t = String(answer ?? '').trim().toLowerCase().replace(/[\s.!]+$/u, '')
+    .replace(RECOMMENDED_SUFFIX, '').replace(/[\s.!]+$/u, '').replace(/\s+/g, ' ');
  if (!t) return { approved: false, reason: 'empty' };
  if (EXPLICIT.has(t)) return { approved: true, reason: 'explicit' };
  if (HOLLOW.some((h) => t === h || (h.includes(' ') && t.includes(h)))) return { approved: false, reason: 'hollow' };
--- a/lib/verification/red-proof.mjs
+++ b/lib/verification/red-proof.mjs
@ -79,12 +79,22 @@ export function repoRoot(dir) {
  return top.status === 0 ? top.stdout.trim() : null;
 }

+// A red that is the test file failing to LOAD (an import the start point
+// lacks) rather than a named test failing. Still a red — the change is what is
+// missing — but the result says which kind, in a field (`red_kind`), instead
+// of leaving it to a reader of `tap` (null without a TAP reporter: the
+// voyage-lekerepo trial's step 1, 2026-10-06).
+const LOAD_ERROR = /SyntaxError|ERR_MODULE_NOT_FOUND|does not provide an export named|Cannot find module|ERR_REQUIRE_ESM|ModuleNotFoundError|ImportError/;
+
 function run(cmd, cwd, timeoutMs) {
  const r = spawnSync('/bin/sh', ['-c', cmd], { cwd, env: commandEnv(), encoding: 'utf8', timeout: timeoutMs, maxBuffer: 64 * 1024 * 1024 });
  const output = `${r.stdout ?? ''}${r.stderr ?? ''}`;
-  return { exit: r.status, signal: r.signal ?? null, tap: tap(output), tail: output.slice(-TAIL), spawnError: r.error ? r.error.message : null };
+  return { exit: r.status, signal: r.signal ?? null, tap: tap(output), tail: output.slice(-TAIL), spawnError: r.error ? r.error.message : null, loadError: LOAD_ERROR.test(output) };
 }

+/** What the red at the start point was: the test file failing to load, or a test failing. */
+const redKind = (base) => (base.loadError ? 'load-error' : 'test-failure');
+
 const notRun = (reason, extra = {}) => ({ status: 'not-run', reason, ...extra });

 // A shell that could not find or execute the command (126/127), a signal (the
@ -188,7 +198,7 @@ export function proveRed(o) {
    if (baseBroken) return notRun(`at the start point: ${baseBroken}`, { base, result, linked });
    const out = { base_sha: baseSha, linked, base, result };
    if (base.exit === 0) return { status: 'rejected', reason: 'not-red-at-base', ...out };
-    if (o.redOnly) return { status: 'proven', reason: 'red at the start point (--red-only: the result was not run)', base_sha: baseSha, linked, base };
+    if (o.redOnly) return { status: 'proven', reason: 'red at the start point (--red-only: the result was not run)', red_kind: redKind(base), base_sha: baseSha, linked, base };
    if (result.exit !== 0) return { status: 'rejected', reason: 'not-green-at-result', ...out };

    // Control: the same copy with the whole change laid on. Red here means the
@ -198,7 +208,7 @@ export function proveRed(o) {
    if (control.exit !== 0) {
      return notRun('the control run (start-point copy + the change) stayed red: the start point failed for an environment reason, not for the missing change', { ...out, control });
    }
-    return { status: 'proven', reason: 'red at the start point, green at the result, green in the control', ...out, control };
+    return { status: 'proven', reason: 'red at the start point, green at the result, green in the control', red_kind: redKind(base), ...out, control };
  } finally {
    rmSync(tmp, { recursive: true, force: true });
  }
@ -234,7 +244,7 @@ export function main(argv) {
  if (json) process.stdout.write(JSON.stringify(r, null, 2) + '\n');
  else {
    const counts = (x) => (x?.tap ? ` (tap: ${x.tap.fail} of ${x.tap.tests} failed)` : '');
-    process.stdout.write(`red proof: ${r.status.toUpperCase()} - ${r.reason}\n`);
+    process.stdout.write(`red proof: ${r.status.toUpperCase()} - ${r.reason}${r.red_kind ? ` (red: ${r.red_kind})` : ''}\n`);
    if (r.base) process.stdout.write(`  start point ${r.base_sha ?? o.base}: exit ${r.base.exit}${counts(r.base)}\n`);
    if (r.result) process.stdout.write(`  result: exit ${r.result.exit}${counts(r.result)}\n`);
    if (r.control) process.stdout.write(`  control: exit ${r.control.exit}${counts(r.control)}\n`);
--- a/scripts/annotate.mjs
+++ b/scripts/annotate.mjs
@ -949,7 +949,9 @@ if (import.meta.url === `file://${process.argv[1]}`) {
    );
    process.exit(args.help ? 0 : 2);
  }
-  const out = render(args.input, args.out);
+  // Absolute, so the reports' `file://{$ANNOT_HTML}` link works from a
+  // relative input (it printed the path as given: voyage-lekerepo trial).
+  const out = resolve(render(args.input, args.out));
  process.stdout.write(out + '\n');
 }

--- a/tests/fixtures/jsonl-schemas.md
+++ b/tests/fixtures/jsonl-schemas.md
@ -20,13 +20,13 @@
 | schema_id | fields | writer_path | line_ref | v4.1 additive | PII |
 |-----------|--------|-------------|----------|---------------|-----|
 | trekbrief-stats | ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason | commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) | trekbrief.md:1124-1139 | profile, phase_models, profile_source | none |
-| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | commands/trekresearch.md (orchestrator-emit Stats tracking) | trekresearch.md:634-676 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
-| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | commands/trekplan.md (orchestrator-emit Phase 12) | trekplan.md:805-826 | profile, phase_models, parallel_agents, profile_source | none |
+| trekresearch-stats | ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions | lib/stats/stats-append.mjs (trekresearch.md Stats tracking) | trekresearch.md:782-808 | profile, phase_models, parallel_agents, external_research_enabled, profile_source | none |
+| trekplan-stats | ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome | lib/stats/stats-append.mjs (trekplan.md Phase 12) | trekplan.md:1115-1135 | profile, phase_models, parallel_agents, profile_source | none |
 | trekexecute-stats (Phase 9 record) | ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step | lib/stats/stats-append.mjs (trekexecute.md Phase 9) | trekexecute.md:2330-2352 | profile, phase_models, profile_source | none |
 | trekexecute-stats (autonomy events) | ts, event, known_event, payload | lib/stats/event-emit.mjs `emit()` | event-emit.mjs:78-99 | payload.profile, payload.phase_models, payload.profile_source | none |
 | trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
-| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | commands/trekreview.md (orchestrator-emit Phase 8) | trekreview.md:255 | profile, phase_models, profile_source | none |
-| trekcontinue-stats | ts, project, next_session_label, status | commands/trekcontinue.md (orchestrator-emit Phase 5) | trekcontinue.md:289 | profile, profile_source | none |
+| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | lib/stats/stats-append.mjs (trekreview.md Phase 8) | trekreview.md:628-632 | profile, phase_models, profile_source | none |
+| trekcontinue-stats | ts, project, next_session_label, status | lib/stats/stats-append.mjs (trekcontinue.md Phase 5) | trekcontinue.md:293-297 | profile, profile_source | none |
 | token-usage | ts, session_id, scope, model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read, cost_usd, is_estimate, price_table_version | lib/stats/token-usage.mjs `captureTokenUsage` (via hooks/scripts/otel-export.mjs Stop hook, opt-in `VOYAGE_TOKEN_METER`) | token-usage.mjs:buildRecord | n/a (SKAL-2 schema) | session_id (stripped at export) |

 ## Field-allowlist input for Step 11
--- a/tests/lib/trial-mechanical-fixes.test.mjs
+++ b/tests/lib/trial-mechanical-fixes.test.mjs
@ -0,0 +1,130 @@
+// tests/lib/trial-mechanical-fixes.test.mjs
+// Mechanical faults from the voyage-lekerepo end-to-end trial (2026-10-06),
+// report points 8, 10, 12, 15 and 16. Each test fails on the code the trial
+// ran.
+
+import { test } from 'node:test';
+import { strict as assert } from 'node:assert';
+import { spawnSync } from 'node:child_process';
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync, existsSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { dirname, join, isAbsolute } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { isExplicitApproval } from '../../lib/util/approval-log.mjs';
+import { proveRed } from '../../lib/verification/red-proof.mjs';
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+const ROOT = join(HERE, '..', '..');
+const read = (p) => readFileSync(join(ROOT, p), 'utf8');
+
+// --- 8. the label AskUserQuestion recommends is still an approval -----------
+
+test('approval: «Approve (Recommended)» is the explicit approval it labels', () => {
+  for (const a of ['Approve (Recommended)', 'Approve (recommended)', 'Godkjenn (anbefalt)', 'Approve  (Recommended).']) {
+    assert.equal(isExplicitApproval(a).approved, true, a);
+  }
+});
+
+test('approval: the suffix does not turn a hollow answer into a yes', () => {
+  for (const a of ['Sounds good (Recommended)', 'Høres bra ut (anbefalt)', '(Recommended)', 'Revise (Recommended)']) {
+    assert.equal(isExplicitApproval(a).approved, false, a);
+  }
+});
+
+// --- 10. the state-file block runs from any working directory ---------------
+
+test('trekexecute Phase 8: the state-file block imports through the plugin root and runs outside it', () => {
+  const md = read('commands/trekexecute.md');
+  assert.doesNotMatch(md, /from '\.\/lib\//, 'a relative import only works with cwd = plugin root');
+  const start = md.indexOf('node --input-type=module -e "\nimport path from \'node:path\';');
+  assert.ok(start > 0, 'the state-file block');
+  const end = md.indexOf('\n```', start);
+  const block = md.slice(start, end)
+    .replaceAll('${CLAUDE_PLUGIN_ROOT}', ROOT)
+    .replace("'{project_dir}' '{next_session_brief_path}' '{next_session_label}' '{status}'", '"$1" brief.md "Session 2" ready');
+  const elsewhere = mkdtempSync(join(tmpdir(), 'voyage-state-'));
+  try {
+    const r = spawnSync('/bin/bash', ['-c', block, 'x', elsewhere], { cwd: elsewhere, encoding: 'utf8' });
+    assert.equal(r.status, 0, r.stderr);
+    assert.ok(existsSync(join(elsewhere, '.session-state.local.json')));
+    assert.match(readFileSync(join(elsewhere, 'NEXT-SESSION-PROMPT.local.md'), 'utf8'), /produced_by: trekexecute/);
+  } finally {
+    rmSync(elsewhere, { recursive: true, force: true });
+  }
+});
+
+// --- 12. every command's stats go through the one writer --------------------
+
+test('stats: trekreview, trekresearch and trekcontinue write through stats-append, never silently skipped', () => {
+  for (const [file, kind] of [['commands/trekreview.md', 'trekreview'], ['commands/trekresearch.md', 'trekresearch'], ['commands/trekcontinue.md', 'trekcontinue']]) {
+    const md = read(file);
+    assert.match(md, new RegExp(`node \\$\\{CLAUDE_PLUGIN_ROOT\\}/lib/stats/stats-append\\.mjs ${kind} <<'JSON'`), file);
+    assert.doesNotMatch(md, /skip (stats )?silently|silently skip/i, file);
+    assert.doesNotMatch(md, new RegExp(`\\$\\{CLAUDE_PLUGIN_DATA\\}/${kind}-stats\\.jsonl`), file);
+  }
+});
+
+// --- 15. annotate prints a path a file:// link can use ----------------------
+
+test('annotate: a relative input still prints an absolute .html path', () => {
+  const dir = mkdtempSync(join(tmpdir(), 'voyage-annot-'));
+  try {
+    writeFileSync(join(dir, 'brief.md'), '# Brief\n\nText.\n');
+    const r = spawnSync(process.execPath, [join(ROOT, 'scripts', 'annotate.mjs'), 'brief.md'], { cwd: dir, encoding: 'utf8' });
+    assert.equal(r.status, 0, r.stderr);
+    const out = r.stdout.trim();
+    assert.ok(isAbsolute(out), out);
+    assert.ok(existsSync(out), out);
+  } finally {
+    rmSync(dir, { recursive: true, force: true });
+  }
+});
+
+// --- 16. the proof says what kind of red it was ------------------------------
+
+function sh(cwd, cmd, args) {
+  const r = spawnSync(cmd, args, { cwd, encoding: 'utf8' });
+  assert.equal(r.status, 0, `${cmd} ${args.join(' ')}: ${r.stderr}`);
+  return r.stdout.trim();
+}
+
+function repo(moduleText) {
+  const dir = mkdtempSync(join(tmpdir(), 'red-kind-'));
+  sh(dir, 'git', ['init', '-q']);
+  sh(dir, 'git', ['config', 'user.email', 't@example.invalid']);
+  sh(dir, 'git', ['config', 'user.name', 'test']);
+  mkdirSync(join(dir, 'lib'));
+  writeFileSync(join(dir, 'lib', 'sum.mjs'), moduleText);
+  sh(dir, 'git', ['add', 'lib/sum.mjs']);
+  sh(dir, 'git', ['commit', '-q', '-m', 'base']);
+  mkdirSync(join(dir, 'tests'));
+  writeFileSync(join(dir, 'tests', 'sum.test.mjs'), [
+    "import { test } from 'node:test';",
+    "import { strict as assert } from 'node:assert';",
+    "import { sum } from '../lib/sum.mjs';",
+    "test('sum adds', () => { assert.equal(sum(1, 2), 3); });",
+    '',
+  ].join('\n'));
+  writeFileSync(join(dir, 'lib', 'sum.mjs'), 'export function sum(a, b) { return a + b; }\n');
+  return { dir, base: sh(dir, 'git', ['rev-parse', 'HEAD']) };
+}
+
+const CMD = 'node --test tests/sum.test.mjs';
+
+test('red proof: a start point that cannot even load the test is proven with red_kind "load-error"', () => {
+  const { dir, base } = repo('export function other() { return 0; }\n');
+  try {
+    const r = proveRed({ cwd: dir, base, cmd: CMD, tests: ['tests/sum.test.mjs'] });
+    assert.equal(r.status, 'proven', JSON.stringify(r));
+    assert.equal(r.red_kind, 'load-error');
+  } finally { rmSync(dir, { recursive: true, force: true }); }
+});
+
+test('red proof: a named test that fails at the start point is red_kind "test-failure"', () => {
+  const { dir, base } = repo('export function sum(a, b) { return a - b; }\n');
+  try {
+    const r = proveRed({ cwd: dir, base, cmd: CMD, tests: ['tests/sum.test.mjs'] });
+    assert.equal(r.status, 'proven', JSON.stringify(r));
+    assert.equal(r.red_kind, 'test-failure');
+  } finally { rmSync(dir, { recursive: true, force: true }); }
+});