Kjell Tore Guttormsen
33b58f460a
1. A step of only tests can never be proven red. One reading, stepTestsOnly (lib/parsers/plan-schema.mjs, test paths from the new lib/util/test-path.mjs): the plan validator warns PLAN_STEP_TESTS_ONLY, red-proof --plan/--step adds tests_only to a not-red-at-base, and /trekexecute and the step helper send such a step straight up to the plan instead of into fix rounds that cannot succeed. /trekplan, the plan template and plan-critic (dimension 8, still 12) say the test belongs in the step that builds the behaviour. 2. Timing tests under load: the hook helper's kill limit is 30 s and per call overridable; SC #14 passes on the fastest of up to three runs against the same 800 ms limit (tests/helpers/timing.mjs). 3. The example plan's On failure lines open with a K11 word. 4. /trekreview saves the reviewer's whole answer for the findings check and names FINDING_PROSE_LINE_OUT_OF_RANGE (error: re-ask; warning: report). 5. The criteria runner's commands run without NODE_TEST_CONTEXT (commandEnv, moved to lib/util/command-env.mjs). 6. The resolver returns effort only when the brief's phase signal sets one; the four commands said it always did. Also: the K2 hook tests used fixed session ids in a per-user temp dir, so two suites running at once took each other's approval records; the ids are now unique per run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
55 lines
2.3 KiB
JavaScript
55 lines
2.3 KiB
JavaScript
// hook-helper.mjs — Shared test helper for hook scripts.
|
|
// Spawns a hook as a child process and feeds it JSON via stdin.
|
|
//
|
|
// Source: ../../../llm-security/tests/hooks/hook-helper.mjs (copied; since order 10
|
|
// the kill limit is HOOK_TIMEOUT_MS and a call can set its own)
|
|
// Provenance: borrowed within the same marketplace (same author, MIT).
|
|
|
|
import { execFile } from 'node:child_process';
|
|
|
|
// How long a hook may run before the helper kills it. The tests check a
|
|
// hook's answer, not its speed (SC #14 times itself, tests/helpers/timing.mjs),
|
|
// so the limit only has to catch a hook that hangs. 5 s killed P7
|
|
// (post-edit-check) at load 95-177 in four wave sessions while it was 8/8
|
|
// green alone (order 10, 2026-10-09). hooks/hooks.json sets no timeout for
|
|
// these hooks, so 5 s was stricter than anything the hook meets in use.
|
|
export const HOOK_TIMEOUT_MS = 30_000;
|
|
|
|
/**
|
|
* Run a hook script by spawning `node <scriptPath>` and piping `input` to stdin.
|
|
*
|
|
* @param {string} scriptPath - Absolute path to the hook .mjs file
|
|
* @param {object|string} input - JSON payload (object will be stringified)
|
|
* @returns {Promise<{ code: number, stdout: string, stderr: string }>}
|
|
*/
|
|
export function runHook(scriptPath, input) {
|
|
return runHookWithEnv(scriptPath, input, {});
|
|
}
|
|
|
|
/**
|
|
* Run a hook script with custom environment variables.
|
|
*
|
|
* @param {string} scriptPath - Absolute path to the hook .mjs file
|
|
* @param {object|string} input - JSON payload (object will be stringified)
|
|
* @param {Record<string, string>} envOverrides - Extra env vars to set
|
|
* @param {{ timeoutMs?: number }} [opts] - kill limit for this call (default HOOK_TIMEOUT_MS)
|
|
* @returns {Promise<{ code: number, stdout: string, stderr: string }>}
|
|
*/
|
|
export function runHookWithEnv(scriptPath, input, envOverrides, opts = {}) {
|
|
return new Promise((resolve) => {
|
|
const env = { ...process.env, ...envOverrides };
|
|
const child = execFile(
|
|
'node',
|
|
[scriptPath],
|
|
{ timeout: opts.timeoutMs ?? HOOK_TIMEOUT_MS, env },
|
|
(err, stdout, stderr) => {
|
|
resolve({
|
|
code: child.exitCode ?? (err && err.code === 'ERR_CHILD_PROCESS_STDIO_FINAL' ? 0 : 1),
|
|
stdout: stdout || '',
|
|
stderr: stderr || '',
|
|
});
|
|
}
|
|
);
|
|
child.stdin.end(typeof input === 'string' ? input : JSON.stringify(input));
|
|
});
|
|
}
|