From 5dd7e8447ca2469f6a27d35b7924f29918b34141 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Tue, 7 Apr 2026 22:12:53 +0200 Subject: [PATCH] =?UTF-8?q?fix(ultraplan-local):=20CRITICAL=20=E2=80=94=20?= =?UTF-8?q?worktree=20isolation=20for=20parallel=20sessions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 2.6 previously launched parallel claude -p sessions in the same working directory, causing git race conditions and repository corruption. Changes: - Add Phase 2.55 (pre-flight safety checks): clean tree, plan file tracking, scope fence overlap validation, stale worktree cleanup - Rewrite Phase 2.6 with git worktree isolation: each parallel session gets its own worktree and branch, merged back sequentially - Add merge conflict detection and abort (no silent data loss) - Add unconditional worktree cleanup (even on failure) - Add hard rules 11-13 (worktree mandatory, cleanup, sequential merge) - Session-scoped progress file naming for --session mode - Update headless launch template with worktree support and cleanup trap - Bump version to 1.5.0 Co-Authored-By: Claude Opus 4.6 --- .../.claude-plugin/plugin.json | 2 +- plugins/ultraplan-local/CHANGELOG.md | 31 ++ .../commands/ultraexecute-local.md | 280 ++++++++++++++++-- .../templates/headless-launch-template.md | 72 ++++- 4 files changed, 356 insertions(+), 29 deletions(-) diff --git a/plugins/ultraplan-local/.claude-plugin/plugin.json b/plugins/ultraplan-local/.claude-plugin/plugin.json index 7598e52..485d789 100644 --- a/plugins/ultraplan-local/.claude-plugin/plugin.json +++ b/plugins/ultraplan-local/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "ultraplan-local", "description": "Deep implementation planning with interview, specialized agent swarms, external research, adversarial review, session decomposition, and headless execution support.", - "version": "1.4.0", + "version": "1.5.0", "author": { "name": "Kjell Tore Guttormsen" }, diff --git a/plugins/ultraplan-local/CHANGELOG.md b/plugins/ultraplan-local/CHANGELOG.md index 6249641..f34e88f 100644 --- a/plugins/ultraplan-local/CHANGELOG.md +++ b/plugins/ultraplan-local/CHANGELOG.md @@ -4,6 +4,37 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +## [1.5.0] - 2026-04-07 + +### Fixed + +- **CRITICAL: Parallel session data loss** — Phase 2.6 ran parallel `claude -p` sessions + in the same working directory, causing git race conditions and repository corruption. + Each parallel session now runs in its own git worktree with isolated branch, index, + and working files. Branches are merged back sequentially after each wave completes. + +### Added + +- **Phase 2.55 (Pre-flight safety checks)** — validates clean working tree, committed + plan file, no scope fence overlaps between parallel sessions, and no stale worktrees + before launching parallel execution. +- **Git worktree isolation** for all parallel sessions — one branch per session + (`ultraplan/{slug}/session-{N}`), merged with `--no-ff` after wave completion. +- **Merge conflict detection** — if merging a session branch produces conflicts, the merge + is aborted and conflicting files are reported. No silent data loss. +- **Unconditional worktree cleanup** — worktrees and session branches are always removed, + even on failure. Manual cleanup commands are reported if automated cleanup fails. +- **Hard rules 11-13** — worktree isolation mandatory, cleanup unconditional, merge + sequentially with conflict abort. +- **Session-scoped progress file naming** — `--session N` uses + `.ultraexecute-progress-{slug}-session-{N}.json` to prevent merge conflicts. + +### Changed + +- Headless launch template uses git worktrees with `cleanup_worktrees` trap on EXIT, + clean-tree pre-flight check, and sequential merge after each wave. +- Phase 2.6 rewritten with 5-step worktree lifecycle: create → launch → wait → merge → cleanup. + ## [1.4.0] - 2026-04-06 ### Renamed diff --git a/plugins/ultraplan-local/commands/ultraexecute-local.md b/plugins/ultraplan-local/commands/ultraexecute-local.md index b4235e4..6aeb916 100644 --- a/plugins/ultraplan-local/commands/ultraexecute-local.md +++ b/plugins/ultraplan-local/commands/ultraexecute-local.md @@ -145,11 +145,101 @@ Report: Strategy: {single session | N sessions (M parallel, K sequential)} ``` -## Phase 2.6 — Multi-session orchestration +## Phase 2.55 — Pre-flight safety checks -**Only runs for multi-session execution.** This phase launches headless child -sessions and collects results. After this phase, jump directly to Phase 8 -(final report). +**Only runs for multi-session parallel execution.** These checks prevent the +catastrophic data loss that occurs when parallel sessions share a working directory. + +### Check 1 — Clean working tree + +Run `git status --porcelain`. If there are ANY uncommitted or untracked changes: + +``` +Error: working tree is not clean. Parallel execution requires a clean git state. +Uncommitted changes are invisible to worktrees and will be lost during merge. + +Untracked/modified files: +{output of git status --porcelain} + +Commit or stash your changes, then re-run. +To run sequentially instead: /ultraexecute-local --fg {plan-path} +``` + +Stop execution. Update progress with `status: "stopped"`. + +### Check 2 — Plan file is tracked by git + +Run `git ls-files --error-unmatch {plan-path} 2>/dev/null`. If the plan file is +untracked (exit code != 0): + +```bash +git add {plan-path} +git commit -m "chore: track plan file for parallel execution" +``` + +Report: `Plan file committed for worktree visibility.` + +This ensures every worktree created from HEAD will have the plan file. + +### Check 3 — Scope fence overlap validation + +For each wave that has 2+ sessions, validate that no file appears in the Touch +list of two different sessions in the same wave: + +1. For each session in the wave, extract the "Touch" list from the Execution Strategy. +2. For each pair of sessions (A, B) in the same wave, compute the intersection + of their Touch lists. +3. If any intersection is non-empty: + +``` +Error: scope fence overlap detected in Wave {W}. +Sessions {A} and {B} both touch: {overlapping files} +These sessions cannot safely run in parallel. + +Fix the Execution Strategy in the plan, or use --fg for sequential execution. +``` + +Stop execution. This is a defense-in-depth check — the planning-orchestrator +should have prevented this, but verifying at execution time catches plans +that were manually edited or have bugs. + +### Check 4 — Stale worktree cleanup + +Run `git worktree list`. If any worktrees with paths containing +`ultraplan-sessions/{slug}/worktrees/` exist from a previous failed run: + +```bash +git worktree remove --force {stale-path} 2>/dev/null +git worktree prune +``` + +Also check for stale branches: +```bash +git branch --list "ultraplan/{slug}/*" | while read b; do + git branch -D "$b" 2>/dev/null +done +``` + +Report: `Cleaned {N} stale worktrees and {N} branches from previous run.` + +If cleanup fails, report the manual commands and stop. + +After all 4 checks pass: +``` +Pre-flight: PASS (clean tree, plan tracked, no overlaps, no stale worktrees) +``` + +## Phase 2.6 — Multi-session orchestration (worktree-isolated) + +**Only runs for multi-session execution.** This phase creates isolated git +worktrees for each parallel session, launches headless child sessions in their +own worktrees, merges results back sequentially, and cleans up. After this +phase, jump directly to Phase 8 (final report). + +**CRITICAL SAFETY RULE:** Every parallel `claude -p` session MUST run in its own +git worktree. Never launch two sessions in the same working directory. This rule +exists because parallel git operations in a shared worktree cause index corruption, +race conditions, and repository destruction. ### Step 0 — Billing safety check (MANDATORY) @@ -186,39 +276,74 @@ and stop. If `ANTHROPIC_API_KEY` is NOT set: proceed silently to Step 1. -### Step 1 — Create session log directory +### Step 1 — Create session infrastructure ```bash -mkdir -p .claude/ultraplan-sessions/{slug}/logs +REPO_ROOT="$(git rev-parse --show-toplevel)" +SESSION_DIR="$REPO_ROOT/.claude/ultraplan-sessions/{slug}" +WORKTREE_DIR="$SESSION_DIR/worktrees" +LOG_DIR="$SESSION_DIR/logs" +mkdir -p "$WORKTREE_DIR" "$LOG_DIR" +ORIGINAL_BRANCH="$(git rev-parse --abbrev-ref HEAD)" ``` -### Step 2 — Execute waves +Record `REPO_ROOT`, `WORKTREE_DIR`, `LOG_DIR`, and `ORIGINAL_BRANCH` for use +in subsequent steps. All paths must be absolute. + +### Step 2 — Execute waves with worktree isolation For each wave (in order): -**Launch sessions in this wave:** - -For each session in the wave, launch a headless `claude -p` process: +**2a. Create worktrees for this wave's sessions:** +For each session N in this wave: ```bash -claude -p "/ultraexecute-local --session {N} {plan-path}" \ - > .claude/ultraplan-sessions/{slug}/logs/session-{N}.log 2>&1 & +BRANCH_NAME="ultraplan/{slug}/session-{N}" +WORKTREE_PATH="$WORKTREE_DIR/session-{N}" +git worktree add -b "$BRANCH_NAME" "$WORKTREE_PATH" HEAD ``` -If the wave has only 1 session, run it without `&` (no background needed). +If `git worktree add` fails (e.g., branch exists from a crashed run): +```bash +git branch -D "$BRANCH_NAME" 2>/dev/null +git worktree add -b "$BRANCH_NAME" "$WORKTREE_PATH" HEAD +``` -Track PIDs for parallel sessions. +If it still fails, report the error, mark this session as failed, and skip it. -**Wait for wave completion:** +Report: +``` +Worktree created: session-{N} → {WORKTREE_PATH} (branch: {BRANCH_NAME}) +``` + +**2b. Launch sessions in this wave (each in its own worktree):** + +For each session N in the wave: +```bash +cd "$WORKTREE_PATH" && claude -p "/ultraexecute-local --session {N} {plan-path}" \ + --dangerously-skip-permissions \ + > "$LOG_DIR/session-{N}.log" 2>&1 & +``` + +Key rules: +- `$WORKTREE_PATH` is the absolute path to the session's worktree +- `$LOG_DIR` is an absolute path in the main worktree (NOT inside the session worktree) +- `{plan-path}` is the same relative path — it works because the worktree has + the same repo content from HEAD +- If the wave has only 1 session, run without `&` (no background needed) +- Track PIDs for parallel sessions + +**2c. Wait for wave completion:** ```bash wait {PID1} {PID2} ... ``` -**Check results after each wave:** +**2d. Check results after each wave:** -For each session in the wave, read its log file and grep for -`"ultraexecute_summary"`. Parse the JSON to determine: +For each session in the wave, read its log file (in `$LOG_DIR`, always accessible +from the main worktree) and grep for `"ultraexecute_summary"`. Parse the JSON to +determine: - Did the session complete? (`result: "completed"`) - Did it fail? (`result: "failed"` or `"stopped"`) @@ -226,18 +351,100 @@ If ANY session in the wave failed: ``` Wave {W} FAILED: Session {N} failed at step {S}. Stopping — later waves depend on this wave. -See log: .claude/ultraplan-sessions/{slug}/logs/session-{N}.log +See log: {LOG_DIR}/session-{N}.log ``` -Do NOT start later waves. Jump to Phase 8 with partial results. +Do NOT merge. Do NOT start later waves. Jump to Step 4 (cleanup), then Phase 8. -If all sessions in the wave passed: continue to the next wave. +If all sessions in the wave passed: continue to Step 2e. + +**2e. Merge session branches back (SEQUENTIAL, one at a time):** + +Return to the main worktree: +```bash +cd "$REPO_ROOT" +``` + +For each session N in the wave (in order): +```bash +git merge --no-ff "ultraplan/{slug}/session-{N}" \ + -m "merge: ultraplan session {N} — {session-title}" +``` + +If the merge succeeds (exit code 0): continue to next session. + +If the merge fails (conflict): +```bash +CONFLICTS="$(git diff --name-only --diff-filter=U)" +git merge --abort +``` + +Report: +``` +Wave {W} MERGE CONFLICT: Session {N} branch conflicts with merged state. +Conflicting files: +{CONFLICTS} + +Session {N} log: {LOG_DIR}/session-{N}.log +Aborting further merges. Sessions already merged in this wave are preserved. +``` + +Mark remaining sessions as "merge-failed". Jump to Step 4 (cleanup), then Phase 8. + +**2f. Remove worktrees for completed wave:** + +After successful merge of all sessions in the wave: +```bash +for each session N in the wave: + git worktree remove "$WORKTREE_DIR/session-{N}" --force + git branch -d "ultraplan/{slug}/session-{N}" +done +git worktree prune +``` + +Report: `Wave {W}: {N} sessions merged, worktrees cleaned up.` + +Continue to the next wave. ### Step 3 — Run master verification -After all waves complete successfully, run the plan's `## Verification` section -commands to verify the integrated result. +After all waves complete and merge successfully, run the plan's `## Verification` +section commands to verify the integrated result. -### Step 4 — Aggregate results +### Step 4 — Cleanup (ALWAYS runs, even on failure) + +This step MUST execute regardless of how Step 2 exited — success, failure, or +merge conflict. It is the worktree equivalent of a `finally` block. + +```bash +cd "$REPO_ROOT" + +# Remove any remaining worktrees +for wt in "$WORKTREE_DIR"/session-*; do + [ -d "$wt" ] && git worktree remove "$wt" --force 2>/dev/null +done +git worktree prune + +# Remove session branches +git branch --list "ultraplan/{slug}/*" | while read branch; do + git branch -D "$branch" 2>/dev/null +done + +# Clean up empty directories +rmdir "$WORKTREE_DIR" 2>/dev/null +``` + +Report: +``` +Cleanup: {N} worktrees removed, {N} branches deleted. +``` + +If cleanup fails for any worktree, report but do not fail: +``` +Warning: failed to remove worktree {path}. Manual cleanup: + git worktree remove {path} --force && git worktree prune +``` + +### Step 5 — Aggregate results Collect all session summaries into an aggregated report. Jump to Phase 8. @@ -254,11 +461,19 @@ When mode = `session N`: This mode is used internally by Phase 2.6 when launching child sessions. It can also be used manually to re-run a specific session. +When `--session N` is invoked inside a git worktree (as done by Phase 2.6), all +git operations (add, commit) apply to the worktree's branch. The session does not +need to know it is in a worktree — git handles this transparently. + ## Phase 3 — Progress file setup The progress file lives at `{plan-dir}/.ultraexecute-progress-{slug}.json` where `{slug}` is the plan filename without extension. +**Session-scoped naming:** When `mode = session N`, use +`{plan-dir}/.ultraexecute-progress-{slug}-session-{N}.json` instead. This prevents +merge conflicts when parallel sessions each write their own progress file. + ### Progress file schema ```json @@ -645,3 +860,20 @@ Never let stats failures block the workflow. 10. **No sub-agents.** The executor reads and implements directly. No Agent tool, no TeamCreate, no delegation. + +11. **Worktree isolation is mandatory for parallel execution.** Every parallel + `claude -p` session MUST run in its own git worktree. Never launch two or + more sessions in the same working directory. This rule has no exceptions. + Sequential (single-session) execution does not require worktrees. + +12. **Worktree cleanup is unconditional.** Before producing the final report + (Phase 8), always remove all worktrees and session branches created during + this execution, even if the run failed or was stopped. Leaked worktrees + consume disk space and block future runs. If automated cleanup fails, + report the manual cleanup commands in the final report. + +13. **Merge sequentially, abort on conflict.** After a parallel wave completes, + merge each session's branch into the main branch one at a time with + `--no-ff`. If any merge produces a conflict, run `git merge --abort`, + report the conflicting files, and do not attempt further merges. Never use + `--force` or `--strategy-option theirs/ours` to silently resolve conflicts. diff --git a/plugins/ultraplan-local/templates/headless-launch-template.md b/plugins/ultraplan-local/templates/headless-launch-template.md index 943be95..6281ea2 100644 --- a/plugins/ultraplan-local/templates/headless-launch-template.md +++ b/plugins/ultraplan-local/templates/headless-launch-template.md @@ -17,23 +17,55 @@ set -euo pipefail # Prevent accidental API billing — remove this line if you intend to use API credits unset ANTHROPIC_API_KEY +REPO_ROOT="$(git rev-parse --show-toplevel)" PLAN_DIR="{session_dir}" LOG_DIR="{session_dir}/logs" -mkdir -p "$LOG_DIR" +WORKTREE_BASE="{session_dir}/worktrees" +mkdir -p "$LOG_DIR" "$WORKTREE_BASE" -echo "=== Ultraplan Headless Execution ===" +# Cleanup trap — always remove worktrees on exit (success or failure) +cleanup_worktrees() { + echo "" + echo "=== Cleaning up worktrees ===" + cd "$REPO_ROOT" + for wt in "$WORKTREE_BASE"/session-*; do + [ -d "$wt" ] && git worktree remove "$wt" --force 2>/dev/null && echo "Removed: $wt" + done + git worktree prune + git branch --list "ultraplan/{slug}/*" | while read b; do + git branch -D "$b" 2>/dev/null + done + rmdir "$WORKTREE_BASE" 2>/dev/null + echo "Cleanup complete." +} +trap cleanup_worktrees EXIT + +# Pre-flight: verify clean working tree +if [ -n "$(git status --porcelain)" ]; then + echo "ERROR: Working tree is not clean. Commit or stash changes before parallel execution." + git status --short + exit 1 +fi + +echo "=== Ultraplan Headless Execution (Worktree-Isolated) ===" echo "Plan: {plan_path}" echo "Sessions: {total_sessions}" +echo "Repo root: $REPO_ROOT" echo "" # --- Wave {N}: Parallel sessions (no dependencies) --- echo "--- Wave {N}: {description} ---" -{# For each parallel session in this wave: } -claude -p "$(cat "$PLAN_DIR/session-{n}-{slug}.md")" \ +{# For each parallel session in this wave, create worktree: } +git worktree add -b "ultraplan/{slug}/session-{n}" "$WORKTREE_BASE/session-{n}" HEAD +echo "Worktree created: session-{n} (branch: ultraplan/{slug}/session-{n})" + +{# Launch session in its worktree: } +cd "$WORKTREE_BASE/session-{n}" && claude -p "$(cat "$PLAN_DIR/session-{n}-{slug}.md")" \ --dangerously-skip-permissions \ > "$LOG_DIR/session-{n}.log" 2>&1 & PID_{n}=$! +cd "$REPO_ROOT" echo "Started session {n}: {title} (PID $PID_{n})" {# After all parallel sessions in this wave: } @@ -42,6 +74,25 @@ wait $PID_{n1} $PID_{n2} echo "Wave {N} complete." echo "" +# --- Merge wave results (sequential) --- +echo "--- Merging Wave {N} ---" +cd "$REPO_ROOT" +{# For each session in the wave, merge its branch: } +git merge --no-ff "ultraplan/{slug}/session-{n}" \ + -m "merge: ultraplan session {n} — {title}" +if [ $? -ne 0 ]; then + echo "MERGE CONFLICT: session {n}. Conflicting files:" + git diff --name-only --diff-filter=U + git merge --abort + echo "Aborting. Earlier sessions in this wave are already merged." + exit 1 +fi +git worktree remove "$WORKTREE_BASE/session-{n}" --force +git branch -d "ultraplan/{slug}/session-{n}" +echo "Merged and cleaned: session {n}" + +git worktree prune + # --- Verify wave results --- echo "--- Verifying Wave {N} ---" {# For each session in the wave, run its exit condition commands } @@ -78,3 +129,16 @@ When generating a launch script from this template: 10. **Billing preamble.** Prepend `unset ANTHROPIC_API_KEY` with a comment at the top of the script to prevent accidental API billing. Users who intend to use API credits can remove this line. +11. **Worktree isolation is mandatory.** Every parallel wave MUST use git + worktrees. Each session gets its own worktree and branch. Never launch + parallel `claude -p` sessions in the same working directory. +12. **Cleanup trap on EXIT.** The generated script MUST include a `trap` on + EXIT that removes all worktrees (`git worktree remove --force`) and prunes + branches, even if the script fails or is interrupted. +13. **Sequential merge after each wave.** After all sessions in a wave complete, + merge their branches back to the main branch one at a time. Abort on merge + conflict — do not force-resolve. +14. **Clean working tree before worktrees.** Add a `git status --porcelain` + check at the top of the script. Fail if the working tree is dirty. +15. **Absolute paths for logs.** Log file paths must be absolute (resolved from + `$REPO_ROOT`), not relative to any worktree.