Kjell Tore Guttormsen
33b58f460a
1. A step of only tests can never be proven red. One reading, stepTestsOnly (lib/parsers/plan-schema.mjs, test paths from the new lib/util/test-path.mjs): the plan validator warns PLAN_STEP_TESTS_ONLY, red-proof --plan/--step adds tests_only to a not-red-at-base, and /trekexecute and the step helper send such a step straight up to the plan instead of into fix rounds that cannot succeed. /trekplan, the plan template and plan-critic (dimension 8, still 12) say the test belongs in the step that builds the behaviour. 2. Timing tests under load: the hook helper's kill limit is 30 s and per call overridable; SC #14 passes on the fastest of up to three runs against the same 800 ms limit (tests/helpers/timing.mjs). 3. The example plan's On failure lines open with a K11 word. 4. /trekreview saves the reviewer's whole answer for the findings check and names FINDING_PROSE_LINE_OUT_OF_RANGE (error: re-ask; warning: report). 5. The criteria runner's commands run without NODE_TEST_CONTEXT (commandEnv, moved to lib/util/command-env.mjs). 6. The resolver returns effort only when the brief's phase signal sets one; the four commands said it always did. Also: the K2 hook tests used fixed session ids in a per-user temp dir, so two suites running at once took each other's approval records; the ids are now unique per run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
25 lines
1.1 KiB
JavaScript
25 lines
1.1 KiB
JavaScript
// tests/helpers/timing.mjs
|
|
// A timing assertion that tolerates a busy machine without loosening its limit.
|
|
//
|
|
// Load only ever ADDS time to a run, so the fastest of a few runs is the best
|
|
// reading of what the code itself costs. `fastestWithin` measures up to
|
|
// `tries` runs and passes as soon as one is under the limit. A real slowness
|
|
// is slow every time and still fails; one run slowed by four other test suites
|
|
// does not (measured S123 and in wave 2, 2026-10-08: SC #14 red under load
|
|
// 95-177, green alone).
|
|
|
|
/**
|
|
* @param {number} limitMs - the limit; a run must be strictly under it
|
|
* @param {() => Promise<number>} measure - runs once, resolves to its elapsed ms
|
|
* @param {number} [tries=3]
|
|
* @returns {Promise<{ok: boolean, fastest: number, runs: number[]}>}
|
|
*/
|
|
export async function fastestWithin(limitMs, measure, tries = 3) {
|
|
const runs = [];
|
|
for (let i = 0; i < tries; i++) {
|
|
runs.push(await measure());
|
|
if (runs[runs.length - 1] < limitMs) break;
|
|
}
|
|
const fastest = Math.min(...runs);
|
|
return { ok: fastest < limitMs, fastest, runs };
|
|
}
|