voyage/scripts/bakeoff-armA-merge.mjs
Kjell Tore Guttormsen 29b4ac91b3
fix: round 7 of the release review of 6.3.0, for 6.3.1
Round 7 (516dffa..45aa9e1, WARN: 12 MAJOR, 0 BLOCKER) and the fence defect
found while writing its review.md. Each fix began with a test that failed on
0f400d3 (v6.3.0); suite 2208 (2206/0/2), 14 tests more.

- Tests-only (operator 2026-10-11): the plan's Files: alone decides, and the
  text says so (red-proof, trekexecute Sub-step B, step-implementer,
  HANDOVER-CONTRACTS). red-proof no longer reads the step's diff, so a change
  already in the tree when the step began no longer counts against it, and a
  plan placed outside .claude/ reads the same. The round-6 test that had the
  diff overrule the plan is rewritten to the new rule.
- The Files: reader keeps a plain file beside a backticked path
  (", Dockerfile", "the CLI entry src/cli.mjs", "and Makefile"). Measured on
  the 520 Files: fields in 182 plans: no field reads differently.
- Red proof reports a command its timeout killed as killed after the timeout,
  not "could not start"; the release check tells a timeout from output past
  its buffer and from another signal. One reading of a stop
  (criteria-runner stoppedBy) for red proof, the criteria runner and the
  release check.
- A trailing `# comment` is dropped in a backticked check too, and in a
  command passed as written, so the words screened are the words run.
- The release check reads a relative --review in --repo.
- The 15-minute timeout is defined once (criteria-runner.mjs).
- The findings check reads each cited file once per validation.
- One isMain helper (lib/util/is-main.mjs) for every CLI: the 31 copies of
  the realpath guard and the 20 modules still on the file:// guard, which
  exited 0 having done nothing under a symlink or a path with a space.
- The Bash guard writes its no-one-to-ask refusal once, on a direct condition.
- The findings check ends a ```json block only at a fence on its own line, so
  three backticks inside a JSON string no longer end it (also in 6.2.0).
- Brief-conformance notes: a test for test-lock green on exit 127 (it
  passed at once; it fails when 127 is read as could-not-run); the prose now
  names the git revision `@~1` as the one leading-@ exception.

Not changed: the example plan's steps 1-2 keep their tests-only warning
(operator decision 2026-10-09; the r5 test records it). Brief note 4 (a
Regression: whose runner is not installed goes to the fix rounds) is a
routing question for the operator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 09:11:57 +02:00

102 lines
4.1 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

// scripts/bakeoff-armA-merge.mjs
// NW2 bake-off — Arm A (prose path) merge step.
//
// Arm A is the CURRENT /trekreview Phase 5–6 substrate: main spawns the two
// reviewers FOREGROUND (Agent tool), each emits a trailing fenced ```json block
// of findings (the prose contract), and main hand-validates + dedups before
// spawning the coordinator. This script is that hand-validate + dedup, made
// reproducible:
//
// for each reviewer raw-output file:
// validateReviewerOutput() (NW1 — extract last json fence, parse, schema-check)
// collect findings from VALID reviewers
// triplet-dedup by (file,line,rule_key) [SAME logic as Arm B's dedupByTriplet,
// so the two arms dedup identically —
// triplet-only, no jaccard pass-2]
// → emit { merged, validation, raw_finding_count, deduped_count }
//
// The merged findings are then handed to the review-coordinator (spawned by main),
// mirroring Arm B's coordinatorPrompt input. JSON-robustness metric = the
// `validation` report (parse/schema failures + which codes).
//
// Usage:
// node scripts/bakeoff-armA-merge.mjs [--json] <reviewerA.txt> <reviewerB.txt> ...
// → stdout: merged findings JSON (or full report with --json). Exit 0 always
// (validation failures are reported in-band, not as a nonzero exit).
import { readFileSync, existsSync } from 'node:fs';
import { validateReviewerOutput } from '../lib/review/findings-schema.mjs';
import { isMain } from '../lib/util/is-main.mjs';
// Triplet dedup — byte-for-byte the same key + first-wins policy as
// scripts/trekreview-armB.workflow.mjs `dedupByTriplet`, so neither arm gets a
// dedup advantage. Triplet-only (file,line,rule_key); NO jaccard pass-2.
export function dedupByTriplet(findings) {
const seen = new Map();
for (const f of findings) {
const key = `${f.file} ${f.line} ${f.rule_key}`;
if (!seen.has(key)) seen.set(key, f);
}
return [...seen.values()];
}
/**
* Merge an array of reviewer raw-output texts into the coordinator's input.
* @param {Array<{label?: string, text: string}>} reviewerOutputs
* @returns {{ merged, validation, raw_finding_count, deduped_count }}
*/
export function mergeArmA(reviewerOutputs) {
const validation = [];
const allFindings = [];
for (const { label, text } of reviewerOutputs) {
const r = validateReviewerOutput(text);
validation.push({
label: label || null,
valid: r.valid,
finding_count: Array.isArray(r.parsed?.findings) ? r.parsed.findings.length : 0,
error_codes: (r.errors || []).map((e) => e.code),
warning_codes: (r.warnings || []).map((w) => w.code),
});
// Mirror Phase 5: only VALID reviewer output flows to the coordinator.
if (r.valid && Array.isArray(r.parsed?.findings)) {
allFindings.push(...r.parsed.findings);
}
}
const merged = dedupByTriplet(allFindings);
return {
merged,
validation,
raw_finding_count: allFindings.length,
deduped_count: merged.length,
};
}
// ---- CLI shim ----------------------------------------------------------------
if (isMain(import.meta.url)) {
const argv = process.argv.slice(2);
const asJson = argv.includes('--json');
const files = argv.filter((a) => !a.startsWith('--'));
if (files.length === 0) {
process.stderr.write('Usage: bakeoff-armA-merge.mjs [--json] <reviewer-output> ...\n');
process.exit(2);
}
const outputs = files.map((f) => {
if (!existsSync(f)) {
process.stderr.write(`bakeoff-armA-merge: file not found: ${f}\n`);
process.exit(2);
}
return { label: f, text: readFileSync(f, 'utf-8') };
});
const result = mergeArmA(outputs);
if (asJson) {
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
} else {
process.stdout.write(JSON.stringify(result.merged, null, 2) + '\n');
process.stderr.write(
`arm-A merge: ${result.raw_finding_count} raw → ${result.deduped_count} deduped; ` +
`validation ${result.validation.map((v) => `${v.label}:${v.valid ? 'OK' : v.error_codes.join('/')}`).join(' ')}\n`,
);
}
process.exit(0);
}