feat(review): the review is a gate - verdict in the file, CC engine, project memory
Ordre 5 (wave 1, P11 + P5), review_version 1.1. - K7: `verdict: ALLOW | WARN | BLOCK` is required in review.md (strict error, soft warning for 1.0 reviews). New CLI lib/review/verdict-gate.mjs: 0 = ALLOW/WARN, 1 = BLOCK, 2 = could not decide (never green); a verdict greener than the file's own BLOCKERs or JSON block is exit 2. Realpath main guard (the old file:// guard exits 0 under a symlink - measured). - No finding is struck for length: succinctness trims only MINOR/SUGGESTION text (catalogue tier), keeps BLOCKER/MAJOR whole, counts trims; the coordinator lists what it declined to judge. - Claude Code's /code-review and /security-review run as the engine via the Skill tool (never --fix/--comment/--post/ultra); lib/review/engine-findings.mjs turns their findings into a reviewer payload; two catalogue keys (CODE_REVIEW_DEFECT, SECURITY_REVIEW_VULNERABILITY), catalogue 12 -> 14. An engine that did not run forbids ALLOW. - Every BLOCKER is written as a numbered `### Step R<n>` with its source finding (strict error REVIEW_MISSING_REMEDIATION_STEP). - P5: lib/review/recurrence.mjs - a finding class in two review.md files (current + git history, no new store) becomes one proposal, routed by the catalogue's memory_route; written only between Voyage markers in .claude/rules/voyage-review-lessons.md under a size budget, CLAUDE.md gets a pointer only; the operator approves every line (Phase 8.7). - Workflow substrate: catalogue enum in step; a missing reviewer or engine forbids ALLOW there too (it returned ALLOW on zero findings before). - Fixtures review-run-A/B said WARN over a BLOCKER; now BLOCK. Borrowed and credited in code: Superpowers (declined-to-judge), ai-native-sdlc (cap only nits; mistake/fix/check), Spec Kit (managed block; converge steps), Pocock (retro routing), Kiro (verdict file + gate; evidence per lesson), Cherny (engine -> brief control order; review -> memory; budget). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
2c89a6b580
commit
d88245c06e
28 changed files with 2052 additions and 110 deletions
|
|
@ -14,7 +14,7 @@ Each artifact carries an explicit version field. Schema bumps are coordinated:
|
|||
| `research/*.md` | (implicit; tracked via `type: trekresearch-brief`) | unversioned |
|
||||
| `plan.md` | `plan_version` (plan header line) | `1.7` |
|
||||
| `progress.json` | `schema_version` (top-level) | `"1"` |
|
||||
| `review.md` | `review_version` (frontmatter) | `1.0` |
|
||||
| `review.md` | `review_version` (frontmatter) | `1.1` |
|
||||
| `.session-state.local.json` | `schema_version` (top-level) | `1` (number) |
|
||||
|
||||
## Breaking-change protocol
|
||||
|
|
@ -334,7 +334,9 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
|
|||
|
||||
**Producer:** `/trekreview` Phase 7 (write `review.md` after coordinator dedup + verdict).
|
||||
|
||||
**Consumer:** `/trekplan` Phase 1 when `--brief review.md` is supplied and the consumer detects `type: trekreview` in frontmatter. The plan command branches into a remediation-plan path: BLOCKER + MAJOR findings become plan goals, the produced `plan.md` carries a `source_findings: [<id>, ...]` frontmatter list as the audit trail back to the consumed findings. MINOR + SUGGESTION are skipped for v1.0 plan-input.
|
||||
**Consumer:** `/trekplan` Phase 1 when `--brief review.md` is supplied and the consumer detects `type: trekreview` in frontmatter. The plan command branches into a remediation-plan path: BLOCKER + MAJOR findings become plan goals, the produced `plan.md` carries a `source_findings: [<id>, ...]` frontmatter list as the audit trail back to the consumed findings. MINOR + SUGGESTION are skipped for plan-input. Since 1.1 every BLOCKER is also written as a numbered `### Step R<n>` under `## Remediation Steps` (Files · Source finding · Test first · Verify · On failure), so a consumer can append it as a plan step without rewording it.
|
||||
|
||||
**Second consumer — the verdict gate (K7, 1.1):** `lib/review/verdict-gate.mjs <review.md>` reads the frontmatter `verdict` and exits 0 = ALLOW/WARN, 1 = BLOCK, 2 = could not decide. Exit 2 is never green: a missing file, a missing or unknown verdict, and a file whose trailing JSON block or BLOCKER findings contradict a greener verdict all land there. Whatever stops a merge or a release on a review reads this exit code, never a summary. Borrowed: the verdict goes to a file and a command gates on it (Kiro `feature-pipeline`); the executor reads the verdict but never writes it (Spec Kit).
|
||||
|
||||
**Path conventions:**
|
||||
- Project-dir mode (recommended): `{project_dir}/review.md` (one per review iteration; subsequent runs overwrite atomically).
|
||||
|
|
@ -352,10 +354,10 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
|
|||
| `brief_path` | string | yes | path to consumed `brief.md` | Audit trail back to brief |
|
||||
| `scope_sha_end` | string | yes | git sha of HEAD at review time | Defines "after" boundary |
|
||||
| `reviewed_files_count` | number | yes | ≥ 0 | From triage gate Coverage |
|
||||
| `findings` | list | yes | block-style YAML list of 40-char hex IDs | Flat array; full objects in body |
|
||||
| `findings` | list | yes | block-style YAML list of 40-char hex IDs; `[]` when empty | Flat array; full objects in body. A bare `findings:` is null and fails |
|
||||
| `verdict` | string | yes (1.1; strict at write, warning at read) | `BLOCK \| WARN \| ALLOW` | Coordinator output; equals the trailing JSON block's `verdict` (K7) |
|
||||
| `created` | date | optional | YYYY-MM-DD | |
|
||||
| `scope_sha_start` | string | optional | git sha at review start | `null` if mtime fallback used |
|
||||
| `verdict` | string | optional | `BLOCK \| WARN \| ALLOW` | Coordinator output |
|
||||
|
||||
`findings:` is a flat array of finding-IDs (40-char hex from `lib/parsers/finding-id.mjs`). The full finding objects (severity, location, message, evidence, fix) live in the body as `### <id>` subsections under per-severity `## Findings (...)` headings — same pattern as brief-reviewer to avoid frontmatter-parser fragility on lists of dicts.
|
||||
|
||||
|
|
@ -364,7 +366,12 @@ The strongest validation in the entire pipeline. Phase 5.5 (planning-orchestrato
|
|||
- `## Coverage`
|
||||
- `## Remediation Summary`
|
||||
|
||||
Optional but standard sections: `## Findings (BLOCKER)`, `## Findings (MAJOR)`, `## Findings (MINOR)`, `## Findings (SUGGESTION)`. The `## Coverage` section enumerates which files were deep-reviewed, summary-only, or skipped (with reason) — this is how the triage gate stays honest and avoids Copilot-style silent skips.
|
||||
Optional but standard sections: `## Findings (BLOCKER)`, `## Findings (MAJOR)`, `## Findings (MINOR)`, `## Findings (SUGGESTION)`. The `## Coverage` section enumerates which files were deep-reviewed, summary-only, or skipped (with reason) — this is how the triage gate stays honest and avoids Copilot-style silent skips; engine findings on files outside the review's SHA range are listed there too.
|
||||
|
||||
Since 1.1 (ordre 5):
|
||||
- `## Remediation Steps` — required when there is a BLOCKER: one `### Step R<n>` per BLOCKER carrying `Source finding: <id>` (strict error `REVIEW_MISSING_REMEDIATION_STEP`, soft warning). Steps are appended and numbered, never reworded across iterations (Spec Kit `/converge`).
|
||||
- `## Declined to Judge` — written by the coordinator: every finding set aside without a ruling, with its reason, plus `Trimmed: N`. **No finding is struck for length**: only MINOR/SUGGESTION text is trimmed, and counted (`trimmed` in the JSON block); a long BLOCKER/MAJOR keeps every character.
|
||||
- Findings from Claude Code's own `/code-review` and `/security-review` enter under the catalogue keys `CODE_REVIEW_DEFECT` (MAJOR) and `SECURITY_REVIEW_VULNERABILITY` (BLOCKER), or a narrower key, with `reviewer` `claude-code-review` / `claude-security-review`.
|
||||
|
||||
**Validation strategy:**
|
||||
|
||||
|
|
@ -374,12 +381,14 @@ Optional but standard sections: `## Findings (BLOCKER)`, `## Findings (MAJOR)`,
|
|||
| Required fields | every read | All `REVIEW_REQUIRED_FRONTMATTER` present |
|
||||
| Type discriminator | every read | `type === "trekreview"` |
|
||||
| Findings shape | every read | Array of strings, each matching `^[0-9a-f]{40}$` |
|
||||
| Verdict present | strict: error · soft: warning | `verdict` ∈ `ALLOW \| WARN \| BLOCK`; an out-of-enum value is an error in both modes |
|
||||
| Remediation step per BLOCKER | strict: error · soft: warning | `### Step R<n>` with `Source finding: <id>` under `## Remediation Steps` |
|
||||
| Body sections | strict only | `Executive Summary`, `Coverage`, `Remediation Summary` |
|
||||
| Version format | every read | `review_version` matches `N.M`; warning otherwise |
|
||||
|
||||
The validator (`lib/validators/review-validator.mjs`) exposes the same CLI as the others: `node lib/validators/review-validator.mjs --json <review.md>`. Strict mode is the default; `--soft` downgrades section-missing errors to warnings. `/trekreview` Phase 8 runs `--strict`. `/trekplan` Phase 1 (when consuming `--brief review.md`) runs `--soft` so a partially-valid review can still seed a plan.
|
||||
|
||||
**Versioning:** current is `1.0`. There are no live `0.x` reviews. Future schema changes follow the breaking-change protocol above.
|
||||
**Versioning:** current is `1.1` (ordre 5: `verdict` required, remediation steps, Declined to Judge, two engine rule keys). N-1 window: a `1.0` review without `verdict` still validates in soft mode (warning) and still seeds a plan; it fails strict mode and the verdict gate (exit 2), because a missing verdict is never read as ALLOW. Future schema changes follow the breaking-change protocol above.
|
||||
|
||||
**Failure modes:**
|
||||
- `REVIEW_NOT_FOUND` → consumer halts with usage message
|
||||
|
|
@ -391,6 +400,9 @@ The validator (`lib/validators/review-validator.mjs`) exposes the same CLI as th
|
|||
- `REVIEW_BAD_FINDING_ID` → an ID is not 40-char hex; halt
|
||||
- `REVIEW_MISSING_SECTION` → strict halt; soft-mode warning
|
||||
- `REVIEW_VERSION_FORMAT` → warning only; review_version not in `N.M` form
|
||||
- `REVIEW_MISSING_VERDICT` → strict halt; soft-mode warning (1.0 reviews)
|
||||
- `REVIEW_BAD_VERDICT` → halt; verdict outside `ALLOW | WARN | BLOCK`
|
||||
- `REVIEW_MISSING_REMEDIATION_STEP` → strict halt; soft-mode warning
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,8 @@ Imported from `CLAUDE.md` via pointer.
|
|||
|
||||
- `lib/util/{frontmatter,result,atomic-write,autonomy-gate}.mjs` — shared YAML-frontmatter parser + Result helpers + `atomicWriteJson(path, obj)` for tmp+rename writes + autonomy-gate state machine (v3.4.0)
|
||||
- `lib/parsers/{plan-schema,manifest-yaml,project-discovery,arg-parser,bash-normalize,jaccard,finding-id}.mjs` — pure parsers (no I/O), unit-tested. `manifest-yaml` extended in v3.4.0 with additive `skip_commit_check` + `memory_write` flags (forward-compat: unknown keys ignored)
|
||||
- `lib/review/{rule-catalogue,plan-review-dedup}.mjs` — version-pinned rule catalogue (12 keys) + Phase 9 inline dedup helpers (v3.4.0)
|
||||
- `lib/review/{rule-catalogue,plan-review-dedup}.mjs` — version-pinned rule catalogue (14 keys) + Phase 9 inline dedup helpers (v3.4.0)
|
||||
- `lib/review/{verdict-gate,engine-findings,recurrence}.mjs` — the review gate (K7: 0 ALLOW/WARN · 1 BLOCK · 2 never green), Claude Code `/code-review` + `/security-review` findings → reviewer payload, recurring finding classes → operator-approved project-memory lines (review_version 1.1)
|
||||
- `lib/stats/event-emit.mjs` — single-source stats event emitter for autonomy-gate transitions and main-merge-gate (v3.4.0)
|
||||
- `lib/validators/{brief,research,plan,progress,session-state}-validator.mjs` — schema validators with CLI shims (`node lib/validators/X.mjs --json <path>`)
|
||||
- `lib/validators/architecture-discovery.mjs` — drift-WARN external-contract discovery for `architecture/overview.md`
|
||||
|
|
|
|||
|
|
@ -85,11 +85,11 @@ If `{project_dir}/architecture/overview.md` exists (typically produced by an opt
|
|||
|
||||
| Flag | Behavior |
|
||||
|------|----------|
|
||||
| _(default)_ | Run brief-conformance + code-correctness reviewers in parallel, coordinator dedup + verdict, write `{project_dir}/review.md` |
|
||||
| _(default)_ | Claude Code `/code-review` + `/security-review` (engine, via the Skill tool, never `--fix`), then brief-conformance + code-correctness reviewers in parallel, coordinator dedup + verdict, write `{project_dir}/review.md` with the required `verdict:`, decide with `lib/review/verdict-gate.mjs` (0 ALLOW/WARN · 1 BLOCK · 2 never green), propose recurring finding classes as project memory (operator approves each line) |
|
||||
| `--project <dir>` | **Required.** Path to trekplan project folder containing `brief.md`. Review is written to `{dir}/review.md` |
|
||||
| `--since <ref>` | Override "before" SHA for the diff range. Validated via `git rev-parse --verify` |
|
||||
| `--quick` | Skip brief-conformance reviewer; skip coordinator's reasonableness filter — fast correctness-only pass |
|
||||
| `--validate` | Schema-only check on existing `{dir}/review.md`. No LLM calls |
|
||||
| `--quick` | `/code-review` + code-correctness reviewer only; skip brief-conformance reviewer, `/security-review` and the coordinator's reasonableness filter — fast correctness-only pass |
|
||||
| `--validate` | Schema-only check on existing `{dir}/review.md` plus the verdict gate. No LLM calls. A 1.0 review without `verdict:` fails here (it still seeds a plan in soft mode) |
|
||||
| `--dry-run` | Print discovered scope + triage map; skip writes |
|
||||
| `--fg` | No-op alias (foreground is default) |
|
||||
| `--workflow` | (opt-in, NW2) Run Phase 5–6 on the bake-off-validated Workflow substrate (`scripts/trekreview-armB.workflow.mjs`) instead of the default prose path. Default stays prose; requires **Claude Code 2.1.154+** (raises the consumer floor — opt-in for portability). Fidelity-equivalent per `docs/T2-bakeoff-results.md` |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue