Kjell Tore Guttormsen
6b30483304
NW3 (CC-26 §6 PoC): delegate trekplan Phase 7 synthesis to a synthesis-agent, adopt only if Δ main-context ≥30% with no quality loss. Operator chose the deterministic-proof path (live ≥3-run bake-off was env-blocked: no API key; installed plugin is a cache copy so a new agent is invisible to `claude -p`). Decisive structural finding: trekplan Phase 5 runs the swarm FOREGROUND, so its outputs are already resident in main before Phase 7. Delegating only Phase 7 evicts nothing → Δ_faithful = 0% (BASE-independent). The ≥30% saving needs an out-of-scope Phase-5 redesign (swarm-writes-to-disk / nested orchestrator). VERDICT: DECLINED per measurement. - agents/synthesis-agent.md — dormant, schema-conformant deliverable (NOT wired) - lib/plan/synthesis-digest-schema.mjs — digest output contract (+ tests) - scripts/synthesis-measure.mjs — deterministic Δ-accounting core (+ tests) - tests/fixtures/synthesis/ — 7 exploration outputs + representative digest - docs/T1-synthesis-poc-results.md — measurement + verdict (reproducible) - CLAUDE.md — agent table row (doc-consistency: 24 agents) Tests 670 → 695 (693 pass / 2 skip / 0 fail). `claude plugin validate` clean (only the pre-existing root-CLAUDE.md warning). commands/trekplan.md untouched. [skip-docs] rationale: no user-facing feature ships (NW3 declined; agent dormant and unwired). The substantive doc is docs/T1-synthesis-poc-results.md; the README/CHANGELOG roll-up for NW1–NW3 is the S13 coordinated release per docs/W1-narrow-wins-plan.md §S13. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
38 lines
1.7 KiB
Markdown
38 lines
1.7 KiB
Markdown
# Test Strategy — output of test-strategist
|
|
|
|
Task: Add a per-wave concurrency cap to trekexecute headless launches.
|
|
|
|
## Existing patterns
|
|
|
|
- Framework: `node:test` + `node:assert/strict`. One file per module under
|
|
`tests/lib/` (mirrors `lib/`) and `tests/scripts/` for harness cores.
|
|
- Validators are tested as **pure functions**: feed a payload, assert
|
|
`{valid,errors}` + stable error codes (see findings-schema.test.mjs,
|
|
autonomy-gate.test.mjs).
|
|
- Template/prose invariants are pinned by **doc-consistency.test.mjs** (string
|
|
`assert.ok(text.includes(...))` against templates/commands).
|
|
|
|
## Coverage gaps for this task
|
|
|
|
1. No test asserts an upper bound on launch parallelism today (there is none).
|
|
2. headless-launch-template.md is pinned for a list of required needles
|
|
(`GIT_OPTIONAL_LOCKS`, `--max-turns`, `--max-budget-usd`, …) but NOT for any
|
|
concurrency mechanism.
|
|
|
|
## Recommended tests (TDD order)
|
|
|
|
1. **`lib/.../max-parallel-resolver.test.mjs`** (new) — pure resolver:
|
|
- flag `--max-parallel 3` wins over profile/default;
|
|
- missing flag → profile default → hard default;
|
|
- `0` / negative / non-integer → `{valid:false}` with a stable code;
|
|
- cap ≥ batchSize → returns batchSize (no-op clamp).
|
|
2. **doc-consistency extension** — assert headless-launch-template.md consumes
|
|
the resolved integer (e.g. includes `$MAX_PARALLEL` or `xargs -P`).
|
|
3. **arg-parser** — assert `--max-parallel` is recognised (extend
|
|
arg-parser.test.mjs / gates-flag-coverage pattern).
|
|
|
|
## Notes
|
|
|
|
The Bash fan-out itself is not unit-testable in `node:test`; rely on the
|
|
template-needle pin (#2) + keeping arithmetic in the JS resolver (#1). This
|
|
matches how the project already tests prose-driven behavior.
|