voyage/docs/S22-happy-path-dogfood.md

# S22 — Happy-path dogfood (Blind spot #1 + #4)

**Run:** S22, 2026-06-19. **Method:** dogfood Voyage's own pipeline (`/trekplan → /trekexecute`) on a real, small Voyage feature, scored against a **pre-registered** ground-truth scorecard committed *before* the pipeline runs.
**Answers the two open questions the S14 audit named but never measured** (`devils-advocate-results.md` §"What this audit might have missed" #1 and #4):

- **Q1 — plan quality:** does `/trekplan` produce a *correct, useful* plan on a real feature?
- **Q4 — review efficacy:** does the adversarial review (`plan-critic` 10-dim + `scope-guardian`) catch *real* defects?

**Operator-chosen shape (S22):** real new Voyage feature (not a planted defect, not a known backlog bug); execute runs in an **isolated git worktree** and the code is **discarded** — the deliverable is the measurement, not the feature.

---

## Methodology — why pre-registration

The audit's deepest critique of itself (#4) was that *nobody ever measured whether the review catches real bugs — both attack and defense assumed it.* The failure mode for a dogfood is **post-hoc rationalization**: run the pipeline, then declare whatever it found "the important stuff." To avoid that, the expected plan and the real-risk list below are **written and git-committed before `/trekplan` is launched** (provable by commit order). Q4 is then scored as *recall against a fixed target*, not "did it find something."

**The feature** (`voyage-doctor`) was chosen because it has three properties that make Q4 measurable on an honest feature:
1. **Strong reuse anchors** (`discoverProject`, `validateBrief/Research/Plan`) → a naive plan reimplements parsing; a good plan composes. Tests whether `scope-guardian`/`plan-critic` flag reinvention.
2. **A subtle state-conditional correctness rule** (`research_status` ↔ `research/` contents) → easy to get wrong. Tests whether `plan-critic` catches a wrong conditional.
3. **Explicit Non-Goals** (read-only, no auto-fix) with a Context that mildly invites scope-creep → tests whether the review verifies Non-Goal coverage.

**Input brief:** `.claude/projects/2026-06-19-voyage-doctor/brief.md` (gitignored per repo convention; Success Criteria + Non-Goals reproduced verbatim below so the control is locked). Brief validates clean: `brief-validator.mjs --json` → `{valid:true, errors:[], warnings:[]}`.

### Brief — Success Criteria (verbatim, locked)
- **SC1** — New `lib/validators/project-doctor.mjs` exports a function returning `{valid, errors, warnings}`, building on `discoverProject()` + the per-artifact content validators rather than re-parsing files itself.
- **SC2** — Detects ≥3 problem classes: (i) present-but-invalid artifact (surface underlying validator findings, tagged by artifact), (ii) `research_status: complete` with empty `research/`, (iii) project-dir slug ≠ brief `slug` frontmatter.
- **SC3** — CLI (`node lib/validators/project-doctor.mjs <projectDir> [--json]`): human report default, `--json`, non-zero exit on invalid — consistent with `brief-validator.mjs`.
- **SC4** — `node:test` coverage for the three SC2 coherence branches + a test asserting delegation to existing validators (no re-implementation).

### Brief — Non-Goals (verbatim, locked)
- **NG1** — No auto-fix, no mutation; strictly read-only.
- **NG2** — No new artifact types, no schema changes to brief/research/plan.
- **NG3** — No network / external calls.
- **NG4** — Not a replacement for the per-artifact validators or `checkPhaseRequirements()`; composes, does not supersede.

---

## Pre-registered ground truth (LOCKED before `/trekplan`)

### A. Expected plan (Q1 oracle)
A competent plan should, at minimum:
1. **Reuse, not reimplement** — call `discoverProject(dir)` for the artifact set, then `validateBrief`/`validateResearch(Dir)`/`validatePlan` on the present artifacts. No new frontmatter/markdown parsing.
2. **Aggregate findings** into one `{valid, errors[], warnings[]}` using the existing `issue()`/`result.mjs` helpers, tagging each finding with the artifact it came from.
3. **Implement the 3 coherence checks** of SC2 with correct conditionals (see real-risks R2 below).
4. **Add a CLI shim** copying `brief-validator.mjs`'s `import.meta.url` pattern (human + `--json` + exit 0/1, usage → exit 2).
5. **TDD** — node:test cases for each coherence branch + a delegation test; fixtures = throwaway project dirs.
6. **Stay read-only** — no writes, honoring NG1.
7. **Decompose into a small number of steps** (module → checks → CLI → tests), each independently testable.

A plan scores well on Q1 if it hits 1–6 without inventing scope beyond the brief.

### B. Real risks / defects the review SHOULD catch (Q4 oracle — fixed target)
Each risk is something a *flawed* plan could plausibly contain. Scoring records, for each: did the **plan** avoid it, and if the plan tripped it, did **plan-critic or scope-guardian flag it**.

| ID | Risk | Which reviewer should catch it if the plan trips it |
|----|------|------------------------------------------------------|
| **R1** | Plan reimplements brief/research/plan parsing instead of reusing the validators (duplication, drift). | scope-guardian (reuse gap) / plan-critic (maintainability) |
| **R2** | The `research_status` ↔ `research/` conditional is wrong: e.g. flags empty `research/` even when `research_topics: 0` / `skipped` (false positive), or misses `complete`+empty (false negative). | plan-critic (correctness / edge cases) |
| **R3** | No error isolation: a malformed/unreadable artifact throws and aborts the whole doctor instead of becoming a finding. | plan-critic (error handling / robustness) |
| **R4** | Missing-artifact vs present-but-invalid not distinguished (both collapse to one code), so the report is ambiguous. | plan-critic (correctness) |
| **R5** | Slug-from-dirname parse is naive: breaks on slugs containing hyphens or dirs lacking the `YYYY-MM-DD-` prefix. | plan-critic (edge cases) |
| **R6** | Scope-creep against NG1: plan adds auto-fix / "repair" / writing a report file. | scope-guardian (creep vs Non-Goal) |
| **R7** | Tests assert only the happy path; the SC2 conditional branches (esp. R2's false-positive case) are untested. | plan-critic (test coverage) / test-strategist |

**Q4 score = (real risks correctly handled by the plan) + (risks the plan tripped that the review flagged) / total applicable.** A risk the plan handles correctly is *not* counted against the review (nothing to catch) but is recorded as "plan avoided it." The review's job is the residual: of the risks the plan got wrong, how many did it surface?

### C. Scoring rubric
- **Q1 (plan quality):** PASS / PARTIAL / FAIL against expected-plan items 1–6, plus a one-line qualitative verdict. Independently judged by main context reading the produced `plan.md`.
- **Q4 (review efficacy):** for each Ri — `plan: avoided | tripped`, and if tripped `review: caught | missed`. Headline = review recall on tripped risks (caught / tripped). Also note any **real** problems the review raised that are NOT in R1–R7 (true positives outside the pre-registered set → credit) vs. noise/false-positives (debit).
- **Execute (worktree):** did `/trekexecute` produce code that (a) matches the plan and (b) passes its own tests + `node --test`? Run in isolated worktree, then discard. Records a yes/partial/no, not a quality grade.

---

## RESULTS

_(appended after the run — empty until `/trekplan` + `/trekexecute` complete)_