open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity

fix(execute): the progress prose validates clean

Browse source 
The progress file /trekexecute's prose described did not pass the command's own validator: run status in-progress and no partial, no plan_version (required), step status running and passed. Measured on the 4 real progress files: 3 used passed, 2 failed on plan_version. The prose now speaks the validator's vocabulary, site by site: schema (plan_version = the value Phase 2 parsed; run status in_progress|completed|failed|stopped|partial), fresh-mode warning, both resume rules, step start in_progress, step finish completed, the escalation staging rule, the Phase 7.5 commit count (status=completed - left alone, every successful run would read as drift and be overridden to partial) and the final write (adds partial; a sandbox-blocked run writes stopped, report result blocked). Untouched on purpose: plan_verification.status, session-state / NEXT-SESSION-PROMPT status, stats and report result, the tolerant session-orders reader list, Phase 2.6's running state.

Valgt: tolerate a legacy passed on resume (one sentence after each resume rule) because Voyage is a public plugin and an in-flight run written under the old prose would otherwise re-execute finished steps. The validator gets no alias. Valgt: fresh mode also warns for stopped and partial - both are resumable, so they now get the --resume hint instead of a silent overwrite (intended behaviour change).

Red: 3 of 5 new tests fail on the pre-fix text - SC4 (documented progress file validates 0/0 per run status), SC5 (offenders at trekexecute.md 1011, 1055, 1062, 1164, 1169, 1326, 1352, 1489), the legacy-sentence count. Regression guard and known-positive control green on arrival.
Verify 125/125 (trekexecute-progress-schema, trekexecute, session-orders, criteria-runner, progress-validator). Suite 1388 (1386/0/2).

Voyage order 20261005T104513Z-1250514346, plan step 4/5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-10-05 17:39:22 +02:00
commit 48abdb47d6
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 184 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

29
commands/trekexecute.md
View file

@ -1015,12 +1015,13 @@ progress file.
"schema_version": "1",
"plan": "{path}",
"plan_type": "{plan | session-spec}",
"plan_version": "{1.7 | 1.6 | legacy}",
"started_at": "{ISO-8601}",
"updated_at": "{ISO-8601}",
"mode": "{execute | resume | step}",
"total_steps": 0,
"current_step": 0,
"status": "{in-progress | completed | failed | stopped}",
"status": "{in_progress | completed | failed | stopped | partial}",
"steps": {
"1": { "status": "pending", "attempts": 0, "error": null, "completed_at": null, "commit": null }
},
@ -1032,6 +1033,10 @@ progress file.
}
```
`plan_version` is the value Phase 2 parsed from the plan header (required by
`progress-validator.mjs`). The status words here are the validator's:
`PROGRESS_VALID_STATUSES` for the run, `PROGRESS_STEP_STATUSES` for a step.
**`plan_verification` (additive-optional):** written by Phase 7 when the input
is a trekplan — `{status, summary, failed_criteria}`, where `status` is
`"passed" | "failed" | "not-run"`. `null` for session specs (they have an exit
@ -1052,14 +1057,15 @@ present in the spec at minimum — there is **no `(if wired)` conditional**.
### Mode-specific behavior
**mode = execute (fresh):**
- If a progress file exists with status `in-progress` or `failed`: warn that
- If a progress file exists with status `in_progress`, `failed`, `stopped` or `partial`: warn that
`--resume` is available, then wait 3 seconds (`sleep 3`) and start fresh.
This allows headless runs to proceed without blocking.
- Otherwise: create the progress file with all steps in `pending` status.
**mode = resume:**
- If no progress file exists: start from step 1 (same as fresh execute).
- If progress file exists: find the first step with status != `passed`.
- If progress file exists: find the first step with status != `completed`.
A legacy progress file may say `passed` for a finished step; treat it as `completed`.
```
Resuming from step {N}. {M}/{total} steps already completed.
```
@ -1161,12 +1167,13 @@ The core execution phase. Runs for modes: `execute`, `resume`, `step`.
### Determine starting step
- **execute**: step 1
- **resume**: first step where status != `passed`
- **resume**: first step where status != `completed`.
A legacy progress file may say `passed` for a finished step; treat it as `completed`.
- **step N**: step N only
### For each step
Update progress: `steps.{N}.status = "running"`, `current_step = N`, `updated_at = now`.
Update progress: `steps.{N}.status = "in_progress"`, `current_step = N`, `updated_at = now`.
```
--- Step {N}/{total}: {description} ---
@ -1324,10 +1331,10 @@ If **FAIL**, read the `On failure:` clause. Apply the retry cap: **maximum 2 ret
Step {N}: ESCALATED — requires human judgment
```
Commit all completed work before stopping. Stage ONLY files from steps with
`status: "passed"` in the progress file — collect their `Files:` fields. Never
`status: "completed"` in the progress file — collect their `Files:` fields. Never
use `git add -A` (risks staging secrets, binaries, or unrelated work).
```bash
git add {files from passed steps' Files: fields} && git commit -m "wip: trekexecute stopped at step {N} — escalation needed"
git add {files from completed steps' Files: fields} && git commit -m "wip: trekexecute stopped at step {N} — escalation needed"
```
Jump to Phase 7.
@ -1349,7 +1356,7 @@ already passed) but is recorded as `checkpoint_drift: {expected_pattern,
actual_message}` in the progress file. Phase 7.5 audit reports drift as
an advisory.
Update progress: `steps.{N}.status = "passed"`, `steps.{N}.commit = {hash}`,
Update progress: `steps.{N}.status = "completed"`, `steps.{N}.commit = {hash}`,
`steps.{N}.completed_at = now`, `steps.{N}.manifest_audit = "pass"`.
### Step mode exit
@ -1486,7 +1493,7 @@ produces an independent truth based on objective state, not self-narrative.
3. **Commit-count check:** count commits since session-start
(`git rev-list --count {start_sha}..HEAD`). Expected count =
number of steps with `status=passed` (excludes Step 0 if sandbox_preflight).
number of steps with `status=completed` (excludes Step 0 if sandbox_preflight).
4. **Commit-message pattern sweep:** walk `git log {start_sha}..HEAD` and
confirm each commit matches one of the declared
@ -1714,7 +1721,9 @@ integrated result with `git log` on the current branch.
### Progress + state-file writes
Update progress file: `status` to `completed`/`failed`/`stopped`, `updated_at`, `summary`.
Update progress file: `status` to `completed`/`failed`/`stopped`/`partial`, `updated_at`, `summary`.
A sandbox-blocked run (Step 0 sentinel) writes run status `stopped`; its report
result is `blocked`.
**Also atomically write `.session-state.local.json`** (Handover 7) at this
convergence point — every successful, failed, stopped, or partial run that

165
tests/commands/trekexecute-progress-schema.test.mjs Normal file
View file

@ -0,0 +1,165 @@
// tests/commands/trekexecute-progress-schema.test.mjs
// The progress file /trekexecute's prose tells the executor to write must pass the
// command's own validator. Measured 2026-10-05: 3 of 4 real progress files used
// step status `passed` (warnings) and 2 of 4 failed outright on the missing
// `plan_version`, because the prose and lib/validators/progress-validator.mjs
// spoke different vocabularies.
//
// - SC4: the `### Progress file schema` fence, filled in, validates with 0 errors
// and 0 warnings for every documented run status.
// - SC5: every status value the prose instructs the executor to WRITE is in the
// validator's lists. Excluded on purpose (their own vocabularies):
// plan_verification.status, the session-state / NEXT-SESSION-PROMPT status,
// the stats and report `result`, and the tolerant session-orders reader list.
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import * as progressValidator from '../../lib/validators/progress-validator.mjs';
const { validateProgress, checkResumeReadiness, PROGRESS_VALID_STATUSES, PROGRESS_STEP_STATUSES } = progressValidator;
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const COMMAND_FILE = join(ROOT, 'commands', 'trekexecute.md');
const read = () => readFileSync(COMMAND_FILE, 'utf8');
// The one sentence that names the legacy word on purpose (resume tolerance for
// progress files written under the old prose). Matched exactly; nothing else may.
const LEGACY_SENTENCE = 'A legacy progress file may say `passed` for a finished step; treat it as `completed`.';
/** The JSON fence right after `### Progress file schema`, parsed. */
function schemaObject(text) {
const at = text.indexOf('### Progress file schema');
assert.ok(at >= 0, 'trekexecute.md must have "### Progress file schema"');
const m = /```json\n([\s\S]*?)```/.exec(text.slice(at));
assert.ok(m, 'the progress schema section must carry a json fence');
return JSON.parse(m[1]);
}
const alternatives = (v) => v.replace(/^\{|\}$/g, '').split('|').map((x) => x.trim());
/** Fill `{a | b}` with its first alternative and free placeholders with a sample. */
function fill(value) {
if (typeof value !== 'string' || !/^\{.*\}$/.test(value)) return value;
if (value.includes('|')) return alternatives(value)[0];
return value === '{ISO-8601}' ? '2026-10-05T12:00:00Z' : '.claude/projects/x/plan.md';
}
/** The step-finish status: the "Update progress" line that also records the commit. */
function finishedStepStatus(text) {
const m = /Update progress: `steps\.\{N\}\.status = "([^"]+)"`, `steps\.\{N\}\.commit/.exec(text);
assert.ok(m, 'the step-finish "Update progress" line must set steps.{N}.status and .commit');
return m[1];
}
/** Every status value the prose tells the executor to write, with its line. */
function writtenStatuses(text) {
const lines = text.split('\n');
const out = [];
const add = (line, value, kind, site) => out.push({ line, value, kind, site });
const lineOf = (index) => text.slice(0, index).split('\n').length;
const schema = schemaObject(text);
const schemaLine = lineOf(text.indexOf('### Progress file schema'));
for (const v of alternatives(schema.status)) add(schemaLine, v, 'run', 'schema');
for (const s of Object.values(schema.steps)) add(schemaLine, s.status, 'step', 'schema');
lines.forEach((l, i) => {
const n = i + 1;
const fresh = /progress file exists with status (.+?):/.exec(l);
if (fresh) for (const m of fresh[1].matchAll(/`([^`]+)`/g)) add(n, m[1], 'run', 'fresh-mode');
for (const m of l.matchAll(/first step (?:with|where) status != `([^`]+)`/g)) add(n, m[1], 'step', 'resume');
for (const m of l.matchAll(/steps with `status=([a-z_-]+)`/g)) add(n, m[1], 'step', 'phase-7.5');
for (const m of l.matchAll(/Set step status = [`"]([^`"]+)[`"]/g)) add(n, m[1], 'step', 'set-step');
for (const m of l.matchAll(/steps\.\{N\}\.status = "([^"]+)"/g)) add(n, m[1], 'step', 'steps.{N}.status');
if (/update progress/i.test(l)) {
for (const m of l.matchAll(/`status: "([^"]+)"`/g)) add(n, m[1], 'run', 'update-progress');
const to = /`status` to ((?:`[a-z_-]+`\/?)+)/.exec(l);
if (to) for (const m of to[1].matchAll(/`([^`]+)`/g)) add(n, m[1], 'run', 'update-progress');
}
});
// The escalation rule spans a line break: "steps with\n`status: "X"`".
for (const m of text.matchAll(/steps with\s+`status: "([^"]+)"`/g)) add(lineOf(m.index), m[1], 'step', 'escalation');
return out;
}
function offenders(entries) {
return entries.filter(({ value, kind }) =>
!(kind === 'run' ? PROGRESS_VALID_STATUSES : PROGRESS_STEP_STATUSES).includes(value))
.map(({ line, value, kind, site }) => `trekexecute.md:${line} ${site} ${kind} status "${value}"`);
}
test('SC4: each documented run status validates with 0 errors and 0 warnings', () => {
const text = read();
const schema = schemaObject(text);
const runStatuses = alternatives(schema.status);
for (const s of ['stopped', 'partial']) assert.ok(runStatuses.includes(s), `the schema must document run status ${s}`);
const finished = finishedStepStatus(text);
const dir = mkdtempSync(join(tmpdir(), 'trekexecute-progress-'));
try {
for (const status of runStatuses) {
const p = Object.fromEntries(Object.entries(schema).map(([k, v]) => [k, fill(v)]));
const step = Object.fromEntries(Object.entries(schema.steps['1']).map(([k, v]) => [k, fill(v)]));
p.status = status;
p.total_steps = 3;
p.current_step = 3;
p.steps = { 1: { ...step, status: finished }, 2: { ...step, status: finished }, 3: { ...step, status: finished } };
const file = join(dir, `progress-${status}.json`);
writeFileSync(file, JSON.stringify(p));
const r = validateProgress(file);
assert.deepEqual(r.errors.map((e) => e.code), [], `run status ${status}: ${JSON.stringify(r.errors)}`);
assert.deepEqual(r.warnings.map((w) => w.code), [], `run status ${status}: ${JSON.stringify(r.warnings)}`);
}
} finally { rmSync(dir, { recursive: true, force: true }); }
});
// Regression guard (a validator contract; resume itself is prose-driven).
test('SC4: every documented run status but completed stays resumable', () => {
for (const status of alternatives(schemaObject(read()).status)) {
const r = checkResumeReadiness({ status, current_step: 1 });
assert.equal(r.valid, status !== 'completed', `run status ${status}`);
}
});
test('SC5: every status the prose tells the executor to write is in the validator lists', () => {
const text = read();
const entries = writtenStatuses(text);
const sites = new Set(entries.map((e) => e.site));
for (const site of ['schema', 'fresh-mode', 'resume', 'phase-7.5', 'set-step', 'steps.{N}.status', 'update-progress', 'escalation']) {
assert.ok(sites.has(site), `the ${site} extractor found nothing - teach it the new wording`);
}
assert.equal(entries.filter((e) => e.site === 'resume').length, 2, 'both resume rules');
const updateLines = text.split('\n').filter((l) => /update progress/i.test(l));
assert.ok(updateLines.length >= 5, `expected >= 5 "Update progress" sites, found ${updateLines.length}`);
const withValue = new Set(entries.filter((e) => e.site === 'update-progress' || e.site === 'steps.{N}.status').map((e) => e.line));
const lines = text.split('\n');
const silent = lines.map((l, i) => [i + 1, l]).filter(([n, l]) => /update progress/i.test(l) && !withValue.has(n));
assert.deepEqual(silent, [], 'an "Update progress" site whose status the extractor cannot read');
assert.deepEqual(offenders(entries), []);
});
test('SC5: the legacy word appears only in the resume-tolerance sentence', () => {
const text = read();
assert.equal(text.split(LEGACY_SENTENCE).length - 1, 2, 'one tolerance sentence after each resume rule');
});
// Known-positive control: the extractors DO see today's (pre-fix) wording.
test('SC5 control: the extractors report the old vocabulary', () => {
const old = [
'### Progress file schema',
'```json',
'{ "status": "{in-progress | completed | failed | stopped}", "steps": { "1": { "status": "pending" } } }',
'```',
'- If a progress file exists with status `in-progress` or `failed`: warn that',
'- If progress file exists: find the first step with status != `passed`.',
'- **resume**: first step where status != `passed`',
'Update progress: `steps.{N}.status = "running"`, `current_step = N`, `updated_at = now`.',
' number of steps with `status=passed` (excludes Step 0 if sandbox_preflight).',
].join('\n');
const bad = offenders(writtenStatuses(old)).join('\n');
for (const word of ['"in-progress"', '"passed"', '"running"']) assert.ok(bad.includes(word), `control must flag ${word}:\n${bad}`);
});