diff --git a/CLAUDE.md b/CLAUDE.md index c6a0c7a..3235a32 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -90,6 +90,25 @@ If `{project_dir}/architecture/overview.md` exists (typically produced by the se | contrarian-researcher | sonnet | Counter-evidence, overlooked alternatives | | gemini-bridge | sonnet | Gemini Deep Research second opinion (conditional) | +## Quality infrastructure (v3.1.0) + +`lib/` contains zero-dep validators and parsers wired into the four commands: + +- `lib/util/{frontmatter,result}.mjs` — shared YAML-frontmatter parser + Result helpers +- `lib/parsers/{plan-schema,manifest-yaml,project-discovery,arg-parser,bash-normalize}.mjs` — pure parsers (no I/O), unit-tested +- `lib/validators/{brief,research,plan,progress}-validator.mjs` — schema validators with CLI shims (`node lib/validators/X.mjs --json `) +- `lib/validators/architecture-discovery.mjs` — drift-WARN external-contract discovery for `architecture/overview.md` + +Wiring points (replaces previous prose-grep instructions): +- `/ultrabrief-local` Phase 4g → `brief-validator` (post-write sanity check) +- `/ultraplan-local` Phase 1 → `brief-validator --soft`, `research-validator --dir`, `architecture-discovery` +- `planning-orchestrator` Phase 5.5 → `plan-validator --strict` (replaces 3 `grep -cE` calls) +- `/ultraexecute-local --validate` → `plan-validator --strict` + `progress-validator` + +Tests under `tests/**/*.test.mjs` (109 tests, 0 deps). `npm test` is the fork-readiness gate. + +Doc-consistency test at `tests/lib/doc-consistency.test.mjs` pins agent-table count, command-table coverage, plan_version invariant, and settings.json scope cleanliness. + ## Architecture **Brief:** 7-phase workflow: Parse mode → Create project dir → Phase 3 completeness loop (section-driven, no question cap) → Phase 4 draft/review/revise with `brief-reviewer` as stop-gate (max 3 iterations; gate = all dimensions ≥ 4 and research plan = 5) → Finalize (`brief.md` on pass, or `brief_quality: partial` on cap/force-stop) → Manual/auto opt-in → Stats. Always interactive. Auto mode runs research + plan inline in the main context (v2.4.0). diff --git a/README.md b/README.md index ff6b1c6..d7ac23a 100644 --- a/README.md +++ b/README.md @@ -473,6 +473,31 @@ v2.0.0 is a **breaking release**. See [MIGRATION.md](MIGRATION.md) for a step-by | Requires GitHub | Yes | Yes | No | **No** | | Cross-platform | Web only | Web only | Desktop | **Mac, Linux, Windows** | +## Quality infrastructure (since v3.1.0) + +The plugin ships with `node:test`-based unit tests and a `lib/` directory of pure-JS validators wired into the commands. Forking the plugin for internal use? Run `npm test` to confirm the parsers, validators, and doc-consistency invariants still hold: + +```bash +cd plugins/ultraplan-local +npm test # runs all tests under tests/**/*.test.mjs +``` + +Validators (zero npm deps, hand-rolled YAML subset): + +| Module | Purpose | +|---|---| +| `lib/validators/brief-validator.mjs` | brief.md frontmatter + state machine (research_topics + status coherence) + body sections | +| `lib/validators/research-validator.mjs` | research-brief frontmatter (confidence ∈ [0,1], dimensions ≥ 1) + body sections; `--dir` mode validates a whole `research/` folder | +| `lib/validators/plan-validator.mjs` | wraps plan-schema + manifest-yaml; enforces v1.7 step heading, manifest count match, and forbidden-narrative-form denylist (`### Fase/Phase/Stage/Steg N`) — replaces the Phase 5.5 grep checks | +| `lib/validators/progress-validator.mjs` | progress.json shape (schema_version, status enum, current_step in range) + resume-readiness check | +| `lib/validators/architecture-discovery.mjs` | EXTERNAL CONTRACT — drift-WARN, never drift-FAIL. Discovers `architecture/overview.md` (owned by the separate `ultra-cc-architect` plugin) and tolerates non-canonical filenames with warnings. | + +Each module exposes a CLI: `node lib/validators/.mjs --json ` returns structured `{valid, errors, warnings, parsed}`. Commands invoke the CLI as their schema check. + +A doc-consistency test (`tests/lib/doc-consistency.test.mjs`) pins prose-vs-source invariants — the agent table in `CLAUDE.md` must match the `agents/*.md` file count, every command's frontmatter `name:` must match its filename, and `templates/plan-template.md` must declare `plan_version: 1.7`. + +Borrowed pattern from `llm-security` (commit `97c5c9d`); extending the plugin should preserve the invariants the test pins. + ## Known limitations **Infrastructure-as-code (IaC) gets reduced value.** The exploration agents are designed for application code. Terraform, Helm, Pulumi, CDK projects will get a plan, but agents like `architecture-mapper` and `test-strategist` produce less useful output for IaC. Use ultraplan-local for the structural plan, then supplement IaC-specific steps manually. diff --git a/agents/planning-orchestrator.md b/agents/planning-orchestrator.md index 134c532..2e88029 100644 --- a/agents/planning-orchestrator.md +++ b/agents/planning-orchestrator.md @@ -320,28 +320,33 @@ If any validation fails, fix the plan before handing to Phase 6 review. ### Phase 5.5 — Schema self-check (REQUIRED before Phase 6) After writing the plan file, verify the output conforms to the executor's -parser BEFORE handing to plan-critic. Use Bash to grep the plan file: +parser BEFORE handing to plan-critic. Run the plan validator: ```bash -# Count canonical step headings -grep -c '^### Step [0-9]\+: ' "$plan_path" - -# Count manifest blocks -grep -c '^ manifest:' "$plan_path" - -# Detect forbidden narrative formats -grep -cE '^(##|###) (Fase|Phase|Stage) [0-9]' "$plan_path" +node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json "$plan_path" ``` -**Pass criteria:** -- Step count ≥ 1 -- Manifest count == Step count -- Forbidden narrative count == 0 +**Pass criteria:** validator exits 0 with `valid: true` in its JSON output. +Internally the validator enforces (same checks as before, now in one place): +- Step count ≥ 1, numbering is 1..N contiguous +- Per-step Manifest YAML present, parses, and `commit_message_pattern` compiles +- Step count == manifest count +- Zero forbidden narrative headings (`### Fase N`, `### Phase N`, `### Stage N`, + `### Steg N`) +- `plan_version: 1.7` declared (warning only if older / missing) -**If the plan fails schema self-check:** rewrite the Implementation Plan -section using the exact literal template shown earlier in Phase 5. Do NOT -proceed to Phase 6 with a schema-failing plan — plan-critic cannot repair -format drift, only content issues. +Each error has a `code` field — read these to localize the fix. Common codes: +- `PLAN_FORBIDDEN_HEADING` — narrative drift; rewrite the section using the + literal template from Phase 5 +- `PLAN_MANIFEST_COUNT_MISMATCH` — at least one step lost its manifest block +- `MANIFEST_PATTERN_INVALID` — a `commit_message_pattern` does not compile; + check escaping (use `\\(` not `\(` in YAML double-quoted strings) +- `PLAN_STEP_NUMBERING` — steps skip a number; renumber sequentially + +**If the plan fails schema self-check:** rewrite the offending section using +the exact literal template shown earlier in Phase 5. Do NOT proceed to Phase 6 +with a schema-failing plan — plan-critic cannot repair format drift, only +content issues. ### Failure recovery (REQUIRED for every step) diff --git a/commands/ultrabrief-local.md b/commands/ultrabrief-local.md index 89c29ac..e66e6e5 100644 --- a/commands/ultrabrief-local.md +++ b/commands/ultrabrief-local.md @@ -460,11 +460,25 @@ After the loop exits (pass, cap, or force-stop), ensure: Populate the "How to continue" footer with the actual project path and topic questions. +**Schema sanity check (since v3.1.0):** before reporting, run the brief +validator. This catches frontmatter typos and state-machine inconsistencies +the brief-reviewer rubric does not check (e.g. `research_status: skipped` +with `research_topics: 3` and no `brief_quality: partial`). + +```bash +node ${CLAUDE_PLUGIN_ROOT}/lib/validators/brief-validator.mjs --json "{PROJECT_DIR}/brief.md" +``` + +If the validator returns errors, report them to the user and offer to +re-enter Phase 4 with the validator's hints in scope. If only warnings, +note them in the final report. + Report: ``` Brief written: {PROJECT_DIR}/brief.md Review iterations: {1..3} Final quality: {complete | partial} +Validator: {PASS | warnings(N)} Research topics identified: {N} ``` diff --git a/commands/ultraexecute-local.md b/commands/ultraexecute-local.md index a0e6f30..b411101 100644 --- a/commands/ultraexecute-local.md +++ b/commands/ultraexecute-local.md @@ -187,6 +187,23 @@ report. Do NOT run security scan, do NOT touch progress files, do NOT execute any steps. This gives the user a fast sanity-check of plan schema compliance without side effects. +**Preferred path (since v3.1.0):** invoke the plan validator directly. It +returns the same diagnostic info Phase 2 derives in prose, with stable +error codes for downstream tooling: + +```bash +node ${CLAUDE_PLUGIN_ROOT}/lib/validators/plan-validator.mjs --strict --json "{path}" + +# When --project is in scope and progress.json exists, also validate it: +[ -f "{project_dir}/progress.json" ] && \ + node ${CLAUDE_PLUGIN_ROOT}/lib/validators/progress-validator.mjs --json "{project_dir}/progress.json" +``` + +Map the validator's `code` field to the error templates below (e.g. +`PLAN_FORBIDDEN_HEADING` → "Detected heading format" branch). When both +calls exit 0, render the READY report. Otherwise render FAIL with the +validator's first error code + message. + If Phase 2 parsing succeeded (no fatal errors, every step has a valid Manifest block in strict mode, or synthesized manifests in legacy mode): diff --git a/commands/ultraplan-local.md b/commands/ultraplan-local.md index 6bc6e5e..6f11ce9 100644 --- a/commands/ultraplan-local.md +++ b/commands/ultraplan-local.md @@ -61,11 +61,27 @@ Parse `$ARGUMENTS` for mode flags. Order of precedence: Missing: {dir}/brief.md ``` - Set **project_dir = {dir}**, **brief_path = {dir}/brief.md**. + - **Validate inputs** (soft mode — warnings do not block, errors do): + ```bash + # Brief schema sanity check (frontmatter + state machine, soft on body sections) + node ${CLAUDE_PLUGIN_ROOT}/lib/validators/brief-validator.mjs --soft --json "{dir}/brief.md" + + # Research briefs (if any) — drift-warn only, none of these block the run + [ -d "{dir}/research" ] && \ + node ${CLAUDE_PLUGIN_ROOT}/lib/validators/research-validator.mjs --soft --dir "{dir}/research" --json + + # Architecture note discovery (EXTERNAL CONTRACT — drift-WARN, never drift-FAIL) + node ${CLAUDE_PLUGIN_ROOT}/lib/validators/architecture-discovery.mjs --json "{dir}" + ``` + Each call exits 0 on success or with a structured JSON error report on stderr. + Surface any warnings in the user-facing summary at Phase 3, but do not abort. - Set **has_research_brief = true** if `{dir}/research/*.md` matches ≥ 1 file. - - Set **has_architecture_note = true** if `{dir}/architecture/overview.md` exists. - If set, **architecture_note_path = {dir}/architecture/overview.md**. Produced by - the optional `/ultra-cc-architect-local` command from the separate `ultra-cc-architect` - plugin. Missing file is fine — this is additive discovery, not a requirement. + - Read the architecture-discovery JSON output: set **has_architecture_note = true** + if `found == true`. The discovery module emits warnings if the file lives at a + non-canonical path (e.g. `architecture-overview.md`); preserve them for the + user-facing summary. If set, **architecture_note_path = {result.overview}**. + Produced by the optional `/ultra-cc-architect-local` command from the separate + `ultra-cc-architect` plugin. Missing file is fine — additive discovery, not required. 4. **`--brief `** — extract the brief path. If the file does not exist: ``` diff --git a/lib/util/frontmatter.mjs b/lib/util/frontmatter.mjs index f3b8e8f..e74ddf9 100644 --- a/lib/util/frontmatter.mjs +++ b/lib/util/frontmatter.mjs @@ -79,10 +79,30 @@ export function parseFrontmatter(yamlText) { while (j < lines.length) { const next = lines[j]; if (next.trim() === '') { j++; continue; } - const m2 = next.match(/^\s+-\s+(.*)$/); - if (!m2) break; - list.push(parseScalar(m2[1])); - j++; + const itemMatch = next.match(/^(\s+)-\s+(.*)$/); + if (!itemMatch) break; + const itemIndent = itemMatch[1].length; + const firstContent = itemMatch[2]; + const dictKeyMatch = firstContent.match(/^([A-Za-z_][A-Za-z0-9_-]*)\s*:\s*(.*)$/); + if (dictKeyMatch) { + const item = {}; + item[dictKeyMatch[1]] = parseScalar(dictKeyMatch[2]); + let k = j + 1; + while (k < lines.length) { + const cont = lines[k]; + if (cont.trim() === '') { k++; continue; } + const contMatch = cont.match(/^(\s+)([A-Za-z_][A-Za-z0-9_-]*)\s*:\s*(.*)$/); + if (!contMatch) break; + if (contMatch[1].length <= itemIndent + 1) break; + item[contMatch[2]] = parseScalar(contMatch[3]); + k++; + } + list.push(item); + j = k; + } else { + list.push(parseScalar(firstContent)); + j++; + } } if (list.length > 0) { out[key] = list; diff --git a/tests/lib/manifest-yaml.test.mjs b/tests/lib/manifest-yaml.test.mjs index 58fcaf0..bd6a68e 100644 --- a/tests/lib/manifest-yaml.test.mjs +++ b/tests/lib/manifest-yaml.test.mjs @@ -99,6 +99,34 @@ test('parseManifest — commit_message_pattern compiles via new RegExp', () => { assert.ok(!re.test('chore: not it')); }); +test('parseManifest — must_contain list-of-dicts (real-world template form)', () => { + const body = `### Step 1: Real +- Manifest: + \`\`\`yaml + manifest: + expected_paths: + - a.json + - b.md + min_file_count: 2 + commit_message_pattern: "^chore:" + bash_syntax_check: [] + forbidden_paths: + - CHANGELOG.md + must_contain: + - path: a.json + pattern: '"version": "2\\.3\\.0"' + - path: b.md + pattern: "version-blue" + \`\`\` +`; + const r = parseManifest(body); + assert.equal(r.valid, true, JSON.stringify(r.errors)); + assert.equal(r.parsed.must_contain.length, 2); + assert.equal(r.parsed.must_contain[0].path, 'a.json'); + assert.equal(r.parsed.must_contain[1].path, 'b.md'); + assert.equal(r.parsed.forbidden_paths[0], 'CHANGELOG.md'); +}); + test('validateAllManifests — aggregates per-step issues', () => { const steps = [ { n: 1, body: STEP_BODY_GOOD },