feat(ultraplan-local): M1 — profile recommendation flow in ultrabrief

Adds the profile recommendation step to /ultrabrief-local Phase 4. The
brief stays universal (same questions, same template); the new step is
purely a processing-decision layer that records which profile downstream
commands should apply.

What lands:
- agents/profile-recommender.md — new sonnet agent that scores available
  profiles against the finalized brief (keyword + NFR-signal matching,
  axis bumps, hallucination gate that forbids inventing profile names).
  Emits a fenced JSON block with ranked entries.
- templates/ultrabrief-template.md — frontmatter gains
  recommended_profile, profile_match, profile_rationale (default values
  applied when only `default` is available — true at M1).
- commands/ultrabrief-local.md — Phase 4 gains Step 4h with explicit
  branches: short-circuit when only `default` exists; AskUserQuestion
  confirmation when top score ≥ 0.7; explicit fallback message when below
  threshold; manual selection sub-question on user override. Persists the
  three frontmatter fields to brief.md after user confirmation. JSON
  parser failure falls back to `default` with `profile_match: fallback`
  rather than blocking — silent fallback is the worst outcome, but a
  *visible* fallback is acceptable.
- scripts/profile-loader.mjs — adds selectRecommendation(ranked, opts) +
  RECOMMENDATION_THRESHOLD=0.7 export. Single source of truth for the
  threshold logic so the command spec and the helper agree.
- scripts/profile-loader.test.mjs — 10 new tests for selectRecommendation
  (default-only, empty/malformed input, above/below threshold, custom
  threshold, max-by-score, missing fields). Total now 36/36.
- README.md / CLAUDE.md / marketplace landing — docs reflect M0 + M1
  shipped, M2 + M3 still pending.

In practice nothing changes for users at M1 because only `default` is
available — Step 4h takes the short-circuit path and writes
`profile_match: default-only`. M2 ships the additional profiles that
make the recommender meaningful.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-04-30 14:21:54 +02:00
commit 7e2d9e151e
8 changed files with 609 additions and 15 deletions

View file

@ -468,6 +468,191 @@ Final quality: {complete | partial}
Research topics identified: {N}
```
### Step 4h — Profile recommendation (M1+)
After the brief is finalized on disk, recommend an ultraplan-local profile
that fits the brief. The profile drives which exploration/review agents
`/ultraplan-local` will spawn, which catalog filter the architect will use,
and which adversarial regime applies. Profiles live in
`${CLAUDE_PLUGIN_ROOT}/profiles/`; the loader is
`${CLAUDE_PLUGIN_ROOT}/scripts/profile-loader.mjs`.
The brief stays universal — Step 4h does not change what the brief contains,
only which processing profile downstream commands should apply to it.
**Step 4h.1 — Discover available profiles**
Run:
```
node ${CLAUDE_PLUGIN_ROOT}/scripts/profile-loader.mjs list
```
Capture the newline-separated profile names into a variable
`AVAILABLE_PROFILES`. If the loader exits non-zero or returns zero names,
skip Step 4h entirely and write `recommended_profile: default`,
`profile_match: default-only` to the brief frontmatter.
**Step 4h.2 — Short-circuit when only `default` exists**
If `AVAILABLE_PROFILES == ["default"]` (M1 ships only `default`), do not
spawn the recommender or ask the user. Write to the brief frontmatter:
```yaml
recommended_profile: default
profile_match: default-only
profile_rationale: "Only the default profile is available; recommendation skipped."
```
Report:
```
Profile: default (only profile available; recommendation skipped)
```
Proceed to Phase 5. The rest of Step 4h applies once M2 ships additional
profiles.
**Step 4h.3 — Build profile manifest for the recommender**
For each profile in `AVAILABLE_PROFILES`, load it:
```
node ${CLAUDE_PLUGIN_ROOT}/scripts/profile-loader.mjs load <name>
```
Extract `name`, `description`, `axes`, and `triggers` from each. Build a
JSON array of profile manifests in memory.
**Step 4h.4 — Spawn `profile-recommender`**
Launch the `profile-recommender` agent (foreground, blocking). Prompt:
> "Read the finalized brief at `{PROJECT_DIR}/brief.md` and rank these
> profiles by fit. Profiles JSON: `{paste manifest JSON array}`. Output the
> ranked list in your standard JSON block. Do not invent profile names — only
> rank what is in the JSON array."
Capture the agent's output. Locate the **last** fenced ```json``` block and
parse it. Expected shape:
```json
{
"ranked": [
{"name": "<profile>", "score": 0.0, "match_quality": "exact|partial|fallback", "rationale": "..."},
...
]
}
```
**JSON fallback:** if the JSON block is missing, malformed, or empty, treat
this as a recommender failure. Write to the brief:
```yaml
recommended_profile: default
profile_match: fallback
profile_rationale: "profile-recommender output could not be parsed; using default."
```
Report the failure to the user as plain text and proceed to Phase 5. Do not
ask AskUserQuestion in this branch — the silent fallback is the explicit
fallback.
**Step 4h.5 — Apply recommendation threshold**
Use the `selectRecommendation()` helper logic (see
`scripts/profile-loader.mjs`):
- If the top-ranked profile has `score >= 0.7`, that is the recommendation.
- If the top score is `< 0.7`, the recommendation is `default` with
`match: fallback`.
This logic is also exported as a helper for tests; use the same threshold.
**Step 4h.6 — Confirm with the user via AskUserQuestion**
If the recommended profile is **non-default** (i.e., a profile scored ≥ 0.7),
present:
```
Question: "Based on the brief, the {recommended} profile fits best.
Use it for /ultraplan-local?"
Options:
1. "Use {recommended}" — apply the recommendation. (Recommended)
2. "Use default" — fall back to the baseline profile.
3. "Choose another" — pick from the full list of available profiles.
```
If the user picks option 1: write `recommended_profile: {recommended}`,
`profile_match: {match_quality}` (from the agent's output),
`profile_rationale: {rationale}` (from the agent's output).
If the user picks option 2: write `recommended_profile: default`,
`profile_match: user-override`, `profile_rationale: "User chose default
over the {recommended} recommendation."`
If the user picks option 3: present a sub-question listing every profile in
`AVAILABLE_PROFILES` with its description. The user picks one. Write
`recommended_profile: {chosen}`, `profile_match: user-override`,
`profile_rationale: "User selected {chosen} manually over the {recommended}
recommendation."`
**Step 4h.7 — Fallback: top score below threshold**
If the top score is `< 0.7`, surface the fallback explicitly:
```
No profile matched this brief strongly enough for an automatic recommendation.
Available profiles:
- default: {description}
- {profile-2}: {description}
- {profile-N}: {description}
Using fallback: default. You can override with:
/ultraplan-local --project {PROJECT_DIR} --profile <name>
```
Then ask via `AskUserQuestion` whether the user wants to pick a profile now
or accept the default fallback:
```
Question: "No profile matched strongly. Pick one now, or use default?"
Options:
1. "Use default (fallback)" — write profile_match: fallback. (Recommended)
2. "Choose a profile manually" — sub-question with full list.
```
If the user picks option 1: write `recommended_profile: default`,
`profile_match: fallback`, `profile_rationale: "{top-score and reason from
agent's output}"`. If the user picks option 2: same flow as Step 4h.6 option 3,
but `profile_match: user-override`.
**Step 4h.8 — Persist to brief frontmatter**
Edit `{PROJECT_DIR}/brief.md` and replace the placeholder values:
```yaml
recommended_profile: <chosen>
profile_match: <exact|partial|fallback|user-override|default-only>
profile_rationale: "<one sentence>"
```
Use Edit with exact matches against the template's placeholder values
(`recommended_profile: default`, `profile_match: default-only`,
`profile_rationale: "Single profile available; no recommendation made."`)
to avoid clobbering other frontmatter.
**Step 4h.9 — Report**
```
Profile: {chosen} (match: {match_quality}, source: {recommended | user-override | fallback})
Rationale: {rationale}
```
If the brief did not write the placeholder defaults (older briefs from before
M1), insert the three lines below the existing frontmatter — never above
`---`. Old briefs without the fields stay valid; downstream consumers default
to `default`.
## Phase 5 — Auto-orchestration opt-in (if research_topics > 0)
**Skip this phase if research_topics = 0.** Proceed directly to Phase 6.