diff --git a/README.md b/README.md index 65383322..525e3f2d 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,7 @@ Then orchestrate execution and review from the CLI: ```bash productize tasks run # execute the task list via the daemon +productize tasks run --concurrent 3 # dependency-aware parallel worktrees productize reviews fetch --pr 123 # pull PR review feedback productize reviews fix # auto-remediate review issues productize archive # file away completed work @@ -241,7 +242,10 @@ access_mode = "full" auto_commit = false [tasks.run] +concurrent = 1 include_completed = false +output_format = "text" +verify_command = "make verify" # Route certain task types to a specific agent/model: task_runtime_rules = [ { type = "frontend", ide = "codex", model = "gpt-5.5" }, diff --git a/docs/cli-reference.md b/docs/cli-reference.md index e158bc97..8fbc2673 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -211,18 +211,42 @@ productize tasks run [flags] The CLI resolves workspace defaults locally, validates task metadata, auto-starts the daemon when needed, and starts the workflow through daemon transport. +Concurrency defaults to one. `--detach` controls only whether the client follows +the run; it does not change task scheduling or isolation. | Flag | Default | Description | | ---- | ------- | ----------- | | `--name` | | Workflow slug, defaults to positional slug | +| `--concurrent` | `1` | Maximum dependency-ready tasks to execute concurrently; must be positive | | `--include-completed` | `false` | Re-run completed tasks | +| `--format` | `text` | Output contract: `text`, `json`, or `raw-json` | | `--skip-validation` | `false` | Skip task metadata preflight; use only when validation already ran elsewhere | | `--force` | `false` | Continue after task metadata validation fails in non-interactive mode | -| `--attach` | `auto` | Attach mode: `auto`, `stream`, or `detach`; legacy `ui` maps to `stream` | -| `--ui` | `false` | Deprecated alias for `--stream` | +| `--attach` | `auto` | Attach mode: `auto`, `stream`, or `detach` | | `--stream` | `false` | Force textual stream attach mode | | `--detach` | `false` | Start the run without attaching a client | | `--task-runtime` | | Per-task runtime override rule such as `type=...`, `id=...`, `ide=...`, `model=...` | +| `--verify-command` | | Shell command used to verify integrated parallel task changes | + +Text mode preserves the human-readable run summary and watcher. JSON emits the +lean workflow event stream as JSONL, while raw JSON emits full canonical event +envelopes. A local parallel dry-run emits one schema-versioned plan object for +both machine-readable formats. + +Task batch size remains fixed at one. Parallel runs require a verification +command unless Productize discovers a Make `verify` target, a Node `verify` or +`test` script, a Go module, or a Rust workspace. Productize executes that command +directly after every merged wave and once more before finalization, records its +exit code and logs, and rejects verifier changes to the integration checkout. + +Parallel mode requires a clean Git-root checkout. Each ready task runs in a +Productize-owned worktree, successful branches merge deterministically, and the +starting branch advances only by safe fast-forward. Failed task worktrees are +retained; verified successful tasks are finalized so a rerun skips them. +Parallel task runs reject `--add-dir` because shared external directories are +not isolated by the repository worktrees. +`--dry-run --concurrent N` performs only local read-only planning and does not +start the daemon, create a run, worktree, branch, or repository file. ## `productize reviews` diff --git a/docs/configuration.md b/docs/configuration.md index 465f1cfd..4ca6f9fa 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -37,8 +37,10 @@ retry_backoff_multiplier = 1.5 types = ["frontend", "backend", "docs", "test", "infra", "refactor", "chore", "bugfix"] [tasks.run] +concurrent = 1 include_completed = false output_format = "text" +verify_command = "make verify" [exec] output_format = "text" @@ -91,7 +93,8 @@ marker remains intact across refreshes. - `[tasks]` for the allowed task `type` list used by `create-tasks` and `productize tasks validate` - `[tasks.run]` for workflow-run defaults used by `productize tasks run`, such - as `include_completed`, `output_format`, and `task_runtime_rules` + as `concurrent`, `include_completed`, `output_format`, `task_runtime_rules`, + and `verify_command` - `[fix_reviews]` for `concurrent`, `batch_size`, `include_resolved`, and `output_format` - `[fetch_reviews]` for `provider` and `nitpicks` @@ -109,6 +112,12 @@ marker remains intact across refreshes. - Both `~/.productize/config.toml` and `.productize/config.toml` are optional. If both are absent, Productize keeps the built-in defaults. - `.productize/tasks` remains the fixed workflow root in this version. +- `tasks.run.concurrent` must be greater than zero. The default is `1`, which + preserves ordered sequential execution; task batch size remains fixed at one. +- `tasks.run.verify_command` is optional. Parallel execution uses it to verify + integrated changes when Productize cannot discover a Make `verify` target, + Node `verify`/`test` script, Go module, or Rust workspace. The command runs + directly after each merged wave and before the final fast-forward. - Unknown keys and invalid value types are rejected during config loading. - Relative `add_dirs` are resolved against the owning config scope: the user home directory for `~/.productize/config.toml` and the workspace root for diff --git a/docs/events.md b/docs/events.md index fa5f987a..f44cc6cc 100644 --- a/docs/events.md +++ b/docs/events.md @@ -14,7 +14,7 @@ Every line in `events.jsonl` is one `events.Event` object: | `run_id` | `string` | Stable identifier for the workflow or exec run that emitted the event. | | `seq` | `uint64` | Monotonic sequence number within a run. | | `ts` | `RFC3339 timestamp` | Event timestamp in UTC. | -| `kind` | `string` | One of the 51 public event kinds below. | +| `kind` | `string` | One of the 52 public event kinds below. | | `payload` | `object` | Kind-specific payload from `pkg/productize/events/kinds`. | ## Run Events @@ -350,6 +350,16 @@ Payload type: `kinds.TaskMemoryUpdatedPayload` - `mode` - `bytes_written` +### `task.scheduler_updated` + +Payload type: `kinds.TaskSchedulerUpdatedPayload` + +Emitted after each durable parallel-task scheduler checkpoint. The payload +contains the manifest path, scheduler status, concurrency, starting and +integration Git state, dependency waves, per-task run/branch/worktree/commit +state, enforced verification command results and logs, finalization state, and +sorted next actions. + ## Artifact Events ### `artifact.updated` diff --git a/docs/plans/2026-08-05-parallel-task-worktrees-design.md b/docs/plans/2026-08-05-parallel-task-worktrees-design.md new file mode 100644 index 00000000..9124dad1 --- /dev/null +++ b/docs/plans/2026-08-05-parallel-task-worktrees-design.md @@ -0,0 +1,108 @@ +# Parallel Task Worktrees + +## Goal + +Allow `productize tasks run --concurrent N` to execute dependency-ready +PRD tasks concurrently without allowing multiple coding agents to mutate the +same checkout. + +Sequential execution remains the default and preserves its current behavior. + +## Architecture + +Productize will treat task frontmatter dependencies as a directed acyclic +graph. Before a run starts, it validates missing references, self-dependencies, +and cycles. The scheduler selects a deterministic wave of pending tasks whose +dependencies have completed and caps active tasks at the requested concurrency. + +For concurrent runs, the daemon creates a run integration branch and a +Productize-owned integration worktree. Each ready task receives its own branch +and worktree rooted at the current integration commit. Worktrees live under the +Productize home instead of inside the user's repository. + +Each task executes through the existing preparation, agent, journal, and result +pipeline. Successful task changes are committed on the task branch. Productize +then merges successful branches into the integration branch in task-number +order. A new wave begins only after the preceding wave is integrated and +verified. + +## Git Safety + +Parallel mode requires a Git repository and a clean starting worktree. It never +stashes, resets, discards, or overwrites user changes. + +Productize records the starting branch and commit. After final verification it +fast-forwards the starting checkout only when it is still clean, remains on the +same branch, and still points at the starting commit. Otherwise, it leaves the +verified integration branch intact and reports the exact merge command. + +Merge conflicts are aborted in the Productize-owned integration worktree. The +task branch and diagnostic worktree are retained for inspection. No automatic +conflict resolution or force update is attempted. + +## Workflow Memory + +Parallel workers read the shared memory snapshot present at the beginning of a +wave but write only their per-task memory files. After task branches merge, the +coordinator rebuilds the shared memory deterministically from the integrated +per-task records before scheduling dependent tasks. This removes shared-file +write races while preserving durable workflow context. + +## CLI and Configuration + +`productize tasks run` gains `--concurrent`. Values greater than one enable the +worktree scheduler; one retains ordered execution. Task batch size remains one. + +`[tasks.run].concurrent` provides the workspace default. `--detach` remains a +presentation choice and does not change isolation semantics. + +Dry-run output includes the validated dependency graph, execution waves, +concurrency, branch names, worktree locations, merge order, and finalization +strategy, without creating Git refs, directories, runs, or daemon mutations. + +## Run State and Recovery + +The existing run journal and daemon read models will record scheduler state, +including task attempts, dependencies, waves, branches, worktrees, commits, +merge outcomes, verification outcomes, blocked tasks, and next actions. + +Cancellation stops and drains owned agent processes and prevents new tasks from +starting. Worktrees containing failed or uncommitted changes are retained; +clean completed worktrees may be removed. Verified successful tasks are safely +finalized even when a sibling fails, so a normal rerun uses their completed task +metadata and does not duplicate completed work. Interrupted daemon processes +retain their checkpoint, branches, and worktrees for manual recovery. + +Independent tasks may finish after another task fails, but dependents of a +failed or unmerged task remain blocked. The run reports partial progress rather +than pretending the workflow completed. + +## Verification + +Each isolated task continues to use the existing task verification contract. +The coordinator verifies every integrated wave and performs final verification +before advancing the starting branch. + +Tests cover graph validation, deterministic waves, bounded concurrency, task +failure, merge conflict, cancellation, retained-resource recovery, dirty repositories, +changed starting branches, worktree cleanup and retention, dry-run purity, +workflow-memory folding, daemon transport, and race safety. Repository +acceptance requires `make verify`. + +## Implementation Plan + +1. Add graph parsing, validation, deterministic wave selection, and scheduler + state types around the existing task metadata. +2. Add a focused Git worktree lifecycle package with injectable command + execution and real-repository integration tests. +3. Extend task-run CLI/config/transport contracts with concurrency and stable + scheduler reporting. +4. Integrate parallel scheduling with the existing planner, executor, daemon, + journal, cancellation, and result paths without creating a second agent + execution stack. +5. Make parallel workflow memory task-local during a wave and rebuild shared + memory deterministically after integration. +6. Add safe merge, verification, final fast-forward, recovery, and cleanup + behavior. +7. Update runtime guidance and user documentation, then run the full + verification gate. diff --git a/docs/workflow.md b/docs/workflow.md index 3916a7d6..8a42fc9d 100644 --- a/docs/workflow.md +++ b/docs/workflow.md @@ -113,10 +113,20 @@ productize tasks run The daemon executes each task by driving your chosen agent over ACP. Useful flags: +- `--concurrent N` to run up to `N` dependency-ready tasks in parallel. The default is `1`; task batch size remains one. - `--attach stream` to watch live, or `--detach` to run in the background (re-attach later with `productize runs watch `). +- `--verify-command "make verify"` to specify the integration verification command when Productize cannot discover one. - `--include-completed` to re-run tasks already marked done; `--skip-validation` or `--force` to bypass the metadata preflight. - `--task-runtime type=frontend,ide=codex,model=gpt-5.5` to route a task type to a specific agent/model. Defaults also live in `[tasks.run].task_runtime_rules` in config. +With concurrency greater than one, Productize validates the task dependency +graph, requires a clean Git-root checkout, and gives every ready task its own +branch and Productize-owned worktree. It merges successful tasks in stable task +order, rebuilds shared workflow memory, and executes the verification command +after every wave and before a safe final fast-forward. Failed worktrees remain +available for inspection; verified partial successes are finalized so the next +run does not repeat them. Parallel `--dry-run` is local and read-only. + --- ## 5. Review & remediate — `productize reviews` diff --git a/internal/api/client/reviews_exec_test.go b/internal/api/client/reviews_exec_test.go index 6aa5169b..b9080a8a 100644 --- a/internal/api/client/reviews_exec_test.go +++ b/internal/api/client/reviews_exec_test.go @@ -298,13 +298,29 @@ func TestClientReviewRequestsEncodeDaemonPathsAndBodies(t *testing.T) { req.URL.EscapedPath(), ) } + body, err := io.ReadAll(req.Body) + if err != nil { + t.Fatalf("read task run request body: %v", err) + } + var payload map[string]any + if err := json.Unmarshal(body, &payload); err != nil { + t.Fatalf("decode task run request body: %v", err) + } + runtimeOverrides, ok := payload["runtime_overrides"].(map[string]any) + if !ok || runtimeOverrides["concurrent"] != float64(3) { + t.Fatalf("runtime_overrides = %#v, want concurrent=3", payload["runtime_overrides"]) + } + if runtimeOverrides["verification_command"] != "make verify" { + t.Fatalf("runtime_overrides = %#v, want verification_command", runtimeOverrides) + } return jsonResponse(http.StatusCreated, `{"run":{"run_id":"task-run-1","mode":"task"}}`), nil }), }, } run, err := client.StartTaskRun(context.Background(), " demo alpha/beta ", apicore.TaskRunRequest{ - Workspace: "/tmp/workspace", + Workspace: "/tmp/workspace", + RuntimeOverrides: json.RawMessage(`{"concurrent":3,"verification_command":"make verify"}`), }) if err != nil { t.Fatalf("StartTaskRun() error = %v", err) diff --git a/internal/api/core/openapi_contract_test.go b/internal/api/core/openapi_contract_test.go index 8142fecf..b07ad092 100644 --- a/internal/api/core/openapi_contract_test.go +++ b/internal/api/core/openapi_contract_test.go @@ -168,6 +168,17 @@ func TestOpenAPIContractKeepsWorkspaceContextAndProblemSemantics(t *testing.T) { if schemaRequires(taskRunSchema, "workspace") { t.Fatal("TaskRunRequest must not require workspace") } + taskRunProperties := getMap(t, taskRunSchema, "properties") + taskRuntimeOverrides := getMap(t, taskRunProperties, "runtime_overrides") + taskRuntimeProperties := getMap(t, taskRuntimeOverrides, "properties") + concurrent := getMap(t, taskRuntimeProperties, "concurrent") + if concurrent["type"] != "integer" || concurrent["minimum"] != float64(1) { + t.Fatalf("TaskRunRequest concurrent schema = %#v, want positive integer", concurrent) + } + verificationCommand := getMap(t, taskRuntimeProperties, "verification_command") + if verificationCommand["type"] != "string" || verificationCommand["minLength"] != float64(1) { + t.Fatalf("TaskRunRequest verification command schema = %#v, want non-empty string", verificationCommand) + } reviewRunSchema := getSchema(t, spec, "ReviewRunRequest") if schemaRequires(reviewRunSchema, "workspace") { t.Fatal("ReviewRunRequest must not require workspace") diff --git a/internal/cli/commands_test.go b/internal/cli/commands_test.go index 37a8eccc..dba18799 100644 --- a/internal/cli/commands_test.go +++ b/internal/cli/commands_test.go @@ -98,6 +98,13 @@ func TestNewTasksRunCommandDefaultsAttachModeToAuto(t *testing.T) { t.Parallel() cmd := newTasksRunCommandWithDefaults(nil, defaultCommandStateDefaults()) + formatFlag := cmd.Flags().Lookup("format") + if formatFlag == nil { + t.Fatal("expected --format flag") + } + if formatFlag.DefValue != string(core.OutputFormatText) { + t.Fatalf("expected --format default %q, got %q", core.OutputFormatText, formatFlag.DefValue) + } flag := cmd.Flags().Lookup("attach") if flag == nil { t.Fatal("expected --attach flag") diff --git a/internal/cli/daemon_commands.go b/internal/cli/daemon_commands.go index 3d719f90..6c92a8f8 100644 --- a/internal/cli/daemon_commands.go +++ b/internal/cli/daemon_commands.go @@ -96,6 +96,8 @@ type daemonRuntimeOverrides struct { Timeout *string `json:"timeout,omitempty"` MaxRetries *int `json:"max_retries,omitempty"` RetryBackoffMultiplier *float64 `json:"retry_backoff_multiplier,omitempty"` + Concurrent *int `json:"concurrent,omitempty"` + VerificationCommand *string `json:"verification_command,omitempty"` Verbose *bool `json:"verbose,omitempty"` Persist *bool `json:"persist,omitempty"` IncludeCompleted *bool `json:"include_completed,omitempty"` @@ -258,8 +260,11 @@ func newTasksRunCommandWithDefaults(_ *kernel.Dispatcher, defaults commandStateD Long: `Start a task workflow through the shared home-scoped daemon. The CLI resolves the workspace root and attach mode locally, ensures the daemon -is running, and then sends the workflow request over the daemon transport.`, +is running, and then sends the workflow request over the daemon transport. +Concurrency controls execution scheduling independently of whether the client +streams the run or detaches.`, Example: ` productize tasks run my-feature + productize tasks run my-feature --concurrent 3 productize tasks run my-feature --stream productize tasks run my-feature --detach productize tasks run --name my-feature --dry-run`, @@ -268,9 +273,16 @@ is running, and then sends the workflow request over the daemon transport.`, }, } - addCommonFlags(cmd, state, commonFlagOptions{}) + addCommonFlags(cmd, state, commonFlagOptions{includeConcurrent: true}) + addWorkflowOutputFlags(cmd, state) cmd.Flags().StringVar(&state.name, "name", "", "Task workflow slug (defaults to the positional slug)") cmd.Flags().BoolVar(&state.includeCompleted, "include-completed", false, "Include completed tasks") + cmd.Flags().StringVar( + &state.verificationCommand, + "verify-command", + "", + "Shell command used to verify integrated parallel task changes", + ) cmd.Flags().BoolVar( &state.skipValidation, "skip-validation", @@ -306,6 +318,9 @@ func (s *commandState) runTaskWorkflow(cmd *cobra.Command, args []string) error if err := s.applyWorkspaceDefaults(ctx, cmd); err != nil { return withExitCode(2, fmt.Errorf("apply workspace defaults for %s: %w", cmd.CommandPath(), err)) } + if err := s.validateTaskRunOptions(cmd); err != nil { + return withExitCode(1, err) + } if len(args) == 0 && strings.TrimSpace(s.name) == "" { if err := s.maybeCollectInteractiveParams(cmd); err != nil { return err @@ -322,11 +337,10 @@ func (s *commandState) runTaskWorkflow(cmd *cobra.Command, args []string) error s.tasksDir = resolvedTasksDir s.explicitRuntime = captureExplicitRuntimeFlags(cmd) - cfg, err := s.buildConfig() - if err != nil { - return withExitCode(2, err) + if err := s.validateAndPreflightTaskRun(ctx, cmd); err != nil { + return err } - if err := s.preflightTaskMetadata(ctx, cmd, cfg); err != nil { + if handled, err := s.runParallelTaskDryPlan(ctx, cmd); handled { return err } @@ -352,7 +366,74 @@ func (s *commandState) runTaskWorkflow(cmd *cobra.Command, args []string) error if err != nil { return mapDaemonCommandError(err) } - return handleStartedTaskRun(ctx, cmd, client, run) + return s.observeStartedTaskRun(ctx, cmd, client, run) +} + +func (s *commandState) validateAndPreflightTaskRun(ctx context.Context, cmd *cobra.Command) error { + cfg, err := s.buildConfig() + if err != nil { + return withExitCode(2, err) + } + if err := cfg.Validate(); err != nil { + return withExitCode(1, err) + } + return s.preflightTaskMetadata(ctx, cmd, cfg) +} + +func (s *commandState) observeStartedTaskRun( + ctx context.Context, + cmd *cobra.Command, + client daemonCommandClient, + run apicore.Run, +) error { + switch strings.TrimSpace(s.outputFormat) { + case string(core.OutputFormatJSON): + return s.streamDaemonWorkflowEvents(ctx, cmd.OutOrStdout(), client, run.RunID, false) + case string(core.OutputFormatRawJSON): + return s.streamDaemonWorkflowEvents(ctx, cmd.OutOrStdout(), client, run.RunID, true) + default: + return handleStartedTaskRun(ctx, cmd, client, run) + } +} + +func (s *commandState) runParallelTaskDryPlan( + ctx context.Context, + cmd *cobra.Command, +) (bool, error) { + if !s.dryRun || s.concurrent <= 1 { + return false, nil + } + plan, err := buildParallelTaskDryRunPlan( + ctx, + s.workspaceRoot, + s.tasksDir, + s.name, + s.concurrent, + s.includeCompleted, + s.verificationCommand, + ) + if err != nil { + return true, withExitCode(1, fmt.Errorf("plan parallel task run: %w", err)) + } + if err := writeParallelTaskDryRunPlan(cmd.OutOrStdout(), s.outputFormat, plan); err != nil { + return true, withExitCode(2, fmt.Errorf("write parallel task plan: %w", err)) + } + return true, nil +} + +func (s *commandState) validateTaskRunOptions(cmd *cobra.Command) error { + if s.concurrent <= 0 { + return fmt.Errorf("--concurrent must be greater than zero (got %d)", s.concurrent) + } + if s.concurrent > 1 && len(s.addDirs) > 0 { + return errors.New( + "parallel task runs do not support --add-dir because shared external directories are not worktree-isolated", + ) + } + if commandFlagChanged(cmd, "verify-command") && strings.TrimSpace(s.verificationCommand) == "" { + return errors.New("--verify-command cannot be blank") + } + return nil } func handleStartedTaskRun( @@ -456,6 +537,15 @@ func (s *commandState) buildTaskRunRuntimeOverrides(cmd *cobra.Command) (json.Ra set(commandFlagChanged(cmd, "dry-run"), func() { overrides.DryRun = boolPointer(s.dryRun) }) set(commandFlagChanged(cmd, "auto-commit"), func() { overrides.AutoCommit = boolPointer(s.autoCommit) }) + set(commandFlagChanged(cmd, "concurrent") || s.concurrent != 1, func() { + overrides.Concurrent = intPointer(s.concurrent) + }) + set(commandFlagChanged(cmd, "verify-command") || s.verificationCommand != "", func() { + overrides.VerificationCommand = stringPointer(s.verificationCommand) + }) + set(commandFlagChanged(cmd, "format"), func() { + overrides.OutputFormat = stringPointer(s.outputFormat) + }) set(commandFlagChanged(cmd, "ide"), func() { overrides.IDE = stringPointer(s.ide) }) set(commandFlagChanged(cmd, "model"), func() { overrides.Model = stringPointer(s.model) }) set(commandFlagChanged(cmd, "add-dir"), func() { diff --git a/internal/cli/daemon_commands_test.go b/internal/cli/daemon_commands_test.go index 541b265f..44b2ec1c 100644 --- a/internal/cli/daemon_commands_test.go +++ b/internal/cli/daemon_commands_test.go @@ -21,6 +21,7 @@ import ( core "github.com/itseffi/productize/internal/core" "github.com/itseffi/productize/internal/core/model" "github.com/itseffi/productize/internal/daemon" + eventspkg "github.com/itseffi/productize/pkg/productize/events" "github.com/spf13/cobra" ) @@ -36,6 +37,19 @@ var ( type daemonCommandContextKey string +func TestValidateTaskRunOptionsRejectsSharedAdditionalDirectoriesInParallelMode(t *testing.T) { + t.Parallel() + + state := &commandState{runtimeConfig: runtimeConfig{ + concurrent: 2, + addDirs: []string{"../shared"}, + }} + err := state.validateTaskRunOptions(&cobra.Command{}) + if err == nil || !strings.Contains(err.Error(), "do not support --add-dir") { + t.Fatalf("validateTaskRunOptions() error = %v, want parallel add-dir rejection", err) + } +} + type stubDaemonCommandClient struct { target apiclient.Target health apicore.DaemonHealth @@ -1565,8 +1579,9 @@ func TestBuildTaskRunRuntimeOverridesIncludesOnlyExplicitFlags(t *testing.T) { state := newCommandState(commandKindTasksRun, "") cmd := newTaskRunPresentationCommand(state) - addCommonFlags(cmd, state, commonFlagOptions{}) + addCommonFlags(cmd, state, commonFlagOptions{includeConcurrent: true}) cmd.Flags().BoolVar(&state.includeCompleted, "include-completed", false, "include completed") + cmd.Flags().StringVar(&state.verificationCommand, "verify-command", "", "verify command") if err := cmd.Flags().Set("dry-run", "true"); err != nil { t.Fatalf("set dry-run: %v", err) @@ -1588,18 +1603,55 @@ func TestBuildTaskRunRuntimeOverridesIncludesOnlyExplicitFlags(t *testing.T) { if overrides.IncludeCompleted == nil || !*overrides.IncludeCompleted { t.Fatalf("expected explicit include-completed override, got %#v", overrides) } - if overrides.AutoCommit != nil || overrides.Model != nil || overrides.Timeout != nil { + if overrides.AutoCommit != nil || overrides.Concurrent != nil || overrides.Model != nil || + overrides.Timeout != nil || overrides.VerificationCommand != nil { t.Fatalf("expected unset flags to remain absent, got %#v", overrides) } } +func TestTasksRunRejectsInvalidParallelRunFlagsBeforeDaemonStartup(t *testing.T) { + tests := []struct { + name string + args []string + wantErr string + }{ + {name: "zero concurrency", args: []string{"demo", "--concurrent", "0"}, wantErr: "--concurrent"}, + {name: "negative concurrency", args: []string{"demo", "--concurrent", "-1"}, wantErr: "--concurrent"}, + {name: "blank verify command", args: []string{"demo", "--verify-command", " "}, wantErr: "--verify-command"}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + isolateCLIConfigHome(t) + chdirCLITest(t, t.TempDir()) + + cmd := newTasksRunCommandWithDefaults(nil, commandStateDefaults{ + commandStateCallbacks: commandStateCallbacks{ + isInteractive: func() bool { return false }, + }, + }) + cmd.SetArgs(tt.args) + err := cmd.Execute() + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("Execute() error = %v, want %q", err, tt.wantErr) + } + var exitErr interface{ ExitCode() int } + if !errors.As(err, &exitErr) || exitErr.ExitCode() != 1 { + t.Fatalf("Execute() error = %v, want exit code 1", err) + } + }) + } +} + func TestBuildTaskRunRuntimeOverridesIncludesAllExplicitRuntimeFlags(t *testing.T) { t.Parallel() state := newCommandState(commandKindTasksRun, "") cmd := newTaskRunPresentationCommand(state) - addCommonFlags(cmd, state, commonFlagOptions{}) + addCommonFlags(cmd, state, commonFlagOptions{includeConcurrent: true}) + addWorkflowOutputFlags(cmd, state) cmd.Flags().BoolVar(&state.includeCompleted, "include-completed", false, "include completed") + cmd.Flags().StringVar(&state.verificationCommand, "verify-command", "", "verify command") cmd.Flags().Var( newTaskRuntimeFlagValue(&state.executionTaskRuntimeRules), "task-runtime", @@ -1615,6 +1667,12 @@ func TestBuildTaskRunRuntimeOverridesIncludesAllExplicitRuntimeFlags(t *testing. mustSetFlag("auto-commit", "true") state.autoCommit = true + mustSetFlag("concurrent", "3") + state.concurrent = 3 + mustSetFlag("verify-command", "make verify") + state.verificationCommand = "make verify" + mustSetFlag("format", "json") + state.outputFormat = "json" mustSetFlag("ide", "claude") state.ide = "claude" mustSetFlag("model", "gpt-5.5") @@ -1644,6 +1702,15 @@ func TestBuildTaskRunRuntimeOverridesIncludesAllExplicitRuntimeFlags(t *testing. if overrides.AutoCommit == nil || !*overrides.AutoCommit { t.Fatalf("expected auto-commit override, got %#v", overrides) } + if overrides.Concurrent == nil || *overrides.Concurrent != 3 { + t.Fatalf("expected concurrent override, got %#v", overrides) + } + if overrides.VerificationCommand == nil || *overrides.VerificationCommand != "make verify" { + t.Fatalf("expected verification command override, got %#v", overrides) + } + if overrides.OutputFormat == nil || *overrides.OutputFormat != "json" { + t.Fatalf("expected output format override, got %#v", overrides) + } if overrides.IDE == nil || *overrides.IDE != "claude" { t.Fatalf("expected ide override, got %#v", overrides) } @@ -1683,6 +1750,160 @@ func TestBuildTaskRunRuntimeOverridesIncludesAllExplicitRuntimeFlags(t *testing. } } +func TestTasksRunCommandUsesWorkflowOutputFormat(t *testing.T) { + tests := []struct { + name string + config string + args []string + wantFormat string + wantTextObserver bool + wantOverride bool + }{ + { + name: "text uses the human observer", + wantFormat: "text", + wantTextObserver: true, + }, + { + name: "explicit json streams lean events", + args: []string{"--format", "json"}, + wantFormat: "json", + wantOverride: true, + }, + { + name: "explicit raw json streams canonical events", + args: []string{"--format", "raw-json"}, + wantFormat: "raw-json", + wantOverride: true, + }, + { + name: "workspace json streams lean events", + config: ` +[tasks.run] +output_format = "json" +`, + wantFormat: "json", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + isolateCLIConfigHome(t) + workspaceRoot, tasksDir := makeValidateTasksWorkspace(t, "demo") + writeRawTaskFileForCLI(t, tasksDir, "task_01.md", cliTaskMarkdown( + []string{ + "status: pending", + "title: Demo Task", + "type: backend", + "complexity: low", + }, + "# Task 1: Demo Task", + )) + writeCLIWorkspaceConfig(t, workspaceRoot, tt.config) + chdirCLITest(t, workspaceRoot) + + stream := newStaticClientRunStream() + stream.items <- apiclient.RunStreamItem{Event: &eventspkg.Event{ + SchemaVersion: eventspkg.SchemaVersion, + RunID: "task-run-format", + Seq: 1, + Kind: eventspkg.EventKindRunStarted, + Timestamp: time.Date(2026, 8, 6, 12, 0, 0, 0, time.UTC), + }} + stream.items <- apiclient.RunStreamItem{Event: &eventspkg.Event{ + SchemaVersion: eventspkg.SchemaVersion, + RunID: "task-run-format", + Seq: 2, + Kind: eventspkg.EventKindRunCompleted, + Timestamp: time.Date(2026, 8, 6, 12, 0, 1, 0, time.UTC), + }} + close(stream.items) + + client := &stubDaemonCommandClient{ + target: apiclient.Target{SocketPath: "/tmp/productize-task-format.sock"}, + health: apicore.DaemonHealth{Ready: true}, + startRun: apicore.Run{ + RunID: "task-run-format", + Mode: string(core.ModePRDTasks), + Status: "running", + PresentationMode: attachModeStream, + }, + stream: stream, + } + installTestCLIReadyDaemonBootstrap(t, client) + + textObserverCalled := false + installTestCLIRunObservers( + t, + nil, + func(_ context.Context, dst io.Writer, _ daemonCommandClient, runID string) error { + textObserverCalled = true + _, err := fmt.Fprintf(dst, "watched %s\n", runID) + return err + }, + ) + + defaults := defaultCommandStateDefaults() + defaults.isInteractive = func() bool { return false } + args := []string{"demo", "--skip-validation"} + args = append(args, tt.args...) + stdout, stderr, err := executeCommandCapturingProcessIO( + t, + newTasksRunCommandWithDefaults(nil, defaults), + nil, + args..., + ) + if err != nil { + t.Fatalf("execute tasks run: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr) + } + if textObserverCalled != tt.wantTextObserver { + t.Fatalf("text observer called=%t, want %t", textObserverCalled, tt.wantTextObserver) + } + + if tt.wantTextObserver { + if !containsAll(stdout, "task run started: task-run-format", "watched task-run-format") { + t.Fatalf("unexpected text output:\n%s", stdout) + } + } else { + events := decodeExecJSONLEvents(t, stdout) + if len(events) != 2 { + t.Fatalf("event count=%d, want 2\noutput:\n%s", len(events), stdout) + } + if strings.Contains(stdout, "task run started:") { + t.Fatalf("machine output contains human prefix:\n%s", stdout) + } + switch tt.wantFormat { + case "json": + if events[0]["type"] != string(eventspkg.EventKindRunStarted) || + events[1]["type"] != string(eventspkg.EventKindRunCompleted) { + t.Fatalf("unexpected lean events: %#v", events) + } + if _, exists := events[0]["kind"]; exists { + t.Fatalf("lean event contains canonical kind: %#v", events[0]) + } + case "raw-json": + if events[0]["kind"] != string(eventspkg.EventKindRunStarted) || + events[1]["kind"] != string(eventspkg.EventKindRunCompleted) { + t.Fatalf("unexpected canonical events: %#v", events) + } + if events[0]["schema_version"] != eventspkg.SchemaVersion { + t.Fatalf("unexpected raw schema version: %#v", events[0]) + } + } + } + + overrides := decodeTaskRunOverrides(t, client.startRequest.RuntimeOverrides) + if tt.wantOverride { + if overrides.OutputFormat == nil || *overrides.OutputFormat != tt.wantFormat { + t.Fatalf("output format override = %#v, want %q", overrides.OutputFormat, tt.wantFormat) + } + } else if overrides.OutputFormat != nil { + t.Fatalf("unexpected configured output format override: %#v", overrides.OutputFormat) + } + }) + } +} + func TestHelpOnlyDaemonCommandRootsReturnHelp(t *testing.T) { t.Parallel() diff --git a/internal/cli/parallel_task_plan.go b/internal/cli/parallel_task_plan.go new file mode 100644 index 00000000..28062299 --- /dev/null +++ b/internal/cli/parallel_task_plan.go @@ -0,0 +1,154 @@ +package cli + +import ( + "context" + "encoding/json" + "fmt" + "io" + "strings" + + "github.com/itseffi/productize/internal/core/model" + taskworktree "github.com/itseffi/productize/internal/core/run" + "github.com/itseffi/productize/internal/core/tasks" +) + +type parallelTaskDryRunPlan struct { + SchemaVersion int `json:"schema_version"` + Status string `json:"status"` + DryRun bool `json:"dry_run"` + WorkspaceRoot string `json:"workspace_root"` + Workflow string `json:"workflow"` + Concurrent int `json:"concurrent"` + StartingBranch string `json:"starting_branch"` + StartingCommit string `json:"starting_commit"` + IntegrationBranch string `json:"integration_branch"` + IntegrationWorktree string `json:"integration_worktree"` + VerificationCommand string `json:"verification_command,omitempty"` + Waves [][]string `json:"waves"` + Tasks []parallelTaskDryRunTask `json:"tasks"` + MergeOrder []string `json:"merge_order"` + Finalization string `json:"finalization"` +} + +type parallelTaskDryRunTask struct { + TaskID string `json:"task_id"` + Dependencies []string `json:"dependencies,omitempty"` + Status string `json:"status"` + Branch string `json:"branch,omitempty"` + WorktreePath string `json:"worktree_path,omitempty"` +} + +func buildParallelTaskDryRunPlan( + ctx context.Context, + workspaceRoot string, + tasksDir string, + workflow string, + concurrent int, + includeCompleted bool, + verificationCommand string, +) (parallelTaskDryRunPlan, error) { + graph, err := tasks.ReadDependencyGraph(tasksDir) + if err != nil { + return parallelTaskDryRunPlan{}, err + } + waves, err := graph.WavesWithOptions(concurrent, includeCompleted) + if err != nil { + return parallelTaskDryRunPlan{}, err + } + nodes := graph.Nodes() + pending := make([]string, 0, len(nodes)) + for _, node := range nodes { + if includeCompleted || !tasks.IsTaskCompleted(model.TaskEntry{Status: node.Status}) { + pending = append(pending, node.ID) + } + } + lifecycle, err := taskworktree.Preflight(ctx, taskworktree.Options{ + WorkspaceRoot: workspaceRoot, + RunID: "dry-run-" + strings.TrimSpace(workflow), + TaskIDs: pending, + }) + if err != nil { + return parallelTaskDryRunPlan{}, err + } + start := lifecycle.StartState() + layout := lifecycle.Layout() + verification := strings.TrimSpace(verificationCommand) + if verification == "" { + verification = "auto-discover" + } + plan := parallelTaskDryRunPlan{ + SchemaVersion: 1, + Status: "planned", + DryRun: true, + WorkspaceRoot: start.WorkspaceRoot, + Workflow: strings.TrimSpace(workflow), + Concurrent: concurrent, + StartingBranch: start.Branch, + StartingCommit: start.HEAD, + IntegrationBranch: layout.IntegrationBranch, + IntegrationWorktree: layout.IntegrationWorktree, + VerificationCommand: verification, + Waves: make([][]string, 0, len(waves)), + Tasks: make([]parallelTaskDryRunTask, 0, len(nodes)), + Finalization: "fast-forward the unchanged starting branch after final verification", + } + for _, wave := range waves { + ids := make([]string, 0, len(wave)) + for _, node := range wave { + ids = append(ids, node.ID) + plan.MergeOrder = append(plan.MergeOrder, node.ID) + } + plan.Waves = append(plan.Waves, ids) + } + for _, node := range nodes { + task := parallelTaskDryRunTask{ + TaskID: node.ID, + Dependencies: append([]string(nil), node.Dependencies...), + Status: node.Status, + } + if planned, taskErr := lifecycle.TaskWorktree(node.ID); taskErr == nil { + task.Branch = planned.Branch + task.WorktreePath = planned.WorktreePath + } + plan.Tasks = append(plan.Tasks, task) + } + return plan, nil +} + +func writeParallelTaskDryRunPlan( + output io.Writer, + format string, + plan parallelTaskDryRunPlan, +) error { + switch model.OutputFormat(strings.TrimSpace(format)) { + case model.OutputFormatJSON, model.OutputFormatRawJSON: + encoder := json.NewEncoder(output) + encoder.SetIndent("", " ") + return encoder.Encode(plan) + default: + if _, err := fmt.Fprintf( + output, + "parallel task plan: %s (concurrent=%d)\n", + plan.Workflow, + plan.Concurrent, + ); err != nil { + return err + } + for index, wave := range plan.Waves { + if _, err := fmt.Fprintf(output, "wave %d: %s\n", index+1, strings.Join(wave, ", ")); err != nil { + return err + } + } + if _, err := fmt.Fprintf(output, "verification: %s\n", plan.VerificationCommand); err != nil { + return err + } + _, err := fmt.Fprintf( + output, + "integration: %s at %s\nfinalization: %s\nno changes were made\n", + plan.IntegrationBranch, + plan.IntegrationWorktree, + plan.Finalization, + ) + return err + } +} diff --git a/internal/cli/parallel_task_plan_test.go b/internal/cli/parallel_task_plan_test.go new file mode 100644 index 00000000..b36f90a0 --- /dev/null +++ b/internal/cli/parallel_task_plan_test.go @@ -0,0 +1,103 @@ +package cli + +import ( + "bytes" + "encoding/json" + "os" + "os/exec" + "path/filepath" + "reflect" + "strings" + "testing" +) + +func TestBuildParallelTaskDryRunPlanIsReadOnlyAndDeterministic(t *testing.T) { + repo := t.TempDir() + runParallelPlanGit(t, repo, "init", "-q", "-b", "main") + runParallelPlanGit(t, repo, "config", "user.email", "parallel-plan@example.com") + runParallelPlanGit(t, repo, "config", "user.name", "Parallel Plan Test") + tasksDir := filepath.Join(repo, ".productize", "tasks", "demo") + if err := os.MkdirAll(tasksDir, 0o755); err != nil { + t.Fatalf("mkdir tasks: %v", err) + } + writeParallelPlanTask(t, tasksDir, "task_01.md", "pending", nil) + writeParallelPlanTask(t, tasksDir, "task_02.md", "pending", nil) + writeParallelPlanTask(t, tasksDir, "task_03.md", "pending", []string{"task_01", "task_02"}) + runParallelPlanGit(t, repo, "add", ".productize") + runParallelPlanGit(t, repo, "commit", "--no-gpg-sign", "-m", "tasks") + + refsBefore := runParallelPlanGit(t, repo, "show-ref", "--heads") + first, err := buildParallelTaskDryRunPlan(t.Context(), repo, tasksDir, "demo", 2, false, "make verify") + if err != nil { + t.Fatalf("buildParallelTaskDryRunPlan() error = %v", err) + } + second, err := buildParallelTaskDryRunPlan(t.Context(), repo, tasksDir, "demo", 2, false, "make verify") + if err != nil { + t.Fatalf("second buildParallelTaskDryRunPlan() error = %v", err) + } + if !reflect.DeepEqual(first, second) { + t.Fatalf("dry-run plans differ\nfirst: %#v\nsecond: %#v", first, second) + } + if got, want := first.Waves, [][]string{{"task_01", "task_02"}, {"task_03"}}; !reflect.DeepEqual(got, want) { + t.Fatalf("waves = %#v, want %#v", got, want) + } + if refsAfter := runParallelPlanGit(t, repo, "show-ref", "--heads"); refsAfter != refsBefore { + t.Fatalf("dry-run changed refs\nbefore: %s\nafter: %s", refsBefore, refsAfter) + } + if status := runParallelPlanGit(t, repo, "status", "--porcelain"); status != "" { + t.Fatalf("dry-run changed repository: %q", status) + } + + var output bytes.Buffer + if err := writeParallelTaskDryRunPlan(&output, "json", first); err != nil { + t.Fatalf("write JSON plan: %v", err) + } + var decoded parallelTaskDryRunPlan + if err := json.Unmarshal(output.Bytes(), &decoded); err != nil { + t.Fatalf("decode JSON plan: %v", err) + } + if !reflect.DeepEqual(decoded, first) { + t.Fatalf("decoded plan mismatch\nwant: %#v\ngot: %#v", first, decoded) + } +} + +func writeParallelPlanTask( + t *testing.T, + tasksDir string, + name string, + status string, + dependencies []string, +) { + t.Helper() + lines := []string{ + "---", + "status: " + status, + "title: " + name, + "type: backend", + "complexity: low", + "dependencies:", + } + if len(dependencies) == 0 { + lines[len(lines)-1] = "dependencies: []" + } else { + for _, dependency := range dependencies { + lines = append(lines, " - "+dependency) + } + } + lines = append(lines, "---", "", "# "+name, "") + if err := os.WriteFile(filepath.Join(tasksDir, name), []byte(strings.Join(lines, "\n")), 0o600); err != nil { + t.Fatalf("write task %s: %v", name, err) + } +} + +func runParallelPlanGit(t *testing.T, dir string, args ...string) string { + t.Helper() + cmd := exec.CommandContext(t.Context(), "git", args...) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GIT_TERMINAL_PROMPT=0") + output, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("git %s: %v: %s", strings.Join(args, " "), err, output) + } + return string(output) +} diff --git a/internal/cli/root_command_execution_test.go b/internal/cli/root_command_execution_test.go index 3685553a..7183bf0a 100644 --- a/internal/cli/root_command_execution_test.go +++ b/internal/cli/root_command_execution_test.go @@ -861,6 +861,10 @@ default_attach_mode = "stream" writeCLIWorkspaceConfig(t, workspaceRoot, ` [runs] default_attach_mode = "detach" + +[tasks.run] +concurrent = 3 +verify_command = "make verify" `) nestedDir := filepath.Join(workspaceRoot, "pkg", "feature") @@ -940,7 +944,6 @@ default_attach_mode = "detach" "tasks", "run", "demo", - "--dry-run", "--include-completed", ) if err != nil { @@ -972,12 +975,76 @@ default_attach_mode = "detach" t.Fatalf("unexpected presentation mode: %q", readyClient.startRequest.PresentationMode) } overrides := decodeTaskRunOverrides(t, readyClient.startRequest.RuntimeOverrides) - if overrides.DryRun == nil || !*overrides.DryRun { - t.Fatalf("expected dry-run override in request, got %#v", overrides) + if overrides.DryRun != nil { + t.Fatalf("did not expect dry-run override in request, got %#v", overrides) } if overrides.IncludeCompleted == nil || !*overrides.IncludeCompleted { t.Fatalf("expected include-completed override in request, got %#v", overrides) } + if overrides.Concurrent == nil || *overrides.Concurrent != 3 { + t.Fatalf("expected configured concurrency in detached request, got %#v", overrides) + } + if overrides.VerificationCommand == nil || *overrides.VerificationCommand != "make verify" { + t.Fatalf("expected configured verification command in detached request, got %#v", overrides) + } +} + +func TestTasksRunParallelDryRunDoesNotBootstrapDaemon(t *testing.T) { + isolateCLIConfigHome(t) + workspaceRoot, tasksDir := makeValidateTasksWorkspace(t, "demo") + writeRawTaskFileForCLI(t, tasksDir, "task_01.md", cliTaskMarkdown( + []string{ + "status: pending", + "title: Demo Task", + "type: backend", + "complexity: low", + "dependencies: []", + }, + "# Task 1: Demo Task", + )) + runParallelPlanGit(t, workspaceRoot, "init", "-q", "-b", "main") + runParallelPlanGit(t, workspaceRoot, "config", "user.email", "dry-run@example.com") + runParallelPlanGit(t, workspaceRoot, "config", "user.name", "Dry Run Test") + runParallelPlanGit(t, workspaceRoot, "add", ".productize") + runParallelPlanGit(t, workspaceRoot, "commit", "--no-gpg-sign", "-m", "tasks") + chdirCLITest(t, workspaceRoot) + + bootstrapCalled := false + installTestCLIDaemonBootstrap(t, cliDaemonBootstrap{ + resolveHomePaths: func() (productizeconfig.HomePaths, error) { + bootstrapCalled = true + return productizeconfig.HomePaths{}, errors.New("daemon bootstrap must not run") + }, + }) + defaults := allowBundledSkillsForExecutionTests() + defaults.isInteractive = func() bool { return false } + cmd := newRootCommandWithDefaults(newLazyRootDispatcher(), defaults) + stdout, stderr, err := executeCommandCapturingProcessIO( + t, + cmd, + nil, + "tasks", + "run", + "demo", + "--dry-run", + "--concurrent", + "2", + "--verify-command", + "make verify", + ) + if err != nil { + t.Fatalf("execute parallel dry-run: %v\nstdout:\n%s\nstderr:\n%s", err, stdout, stderr) + } + if bootstrapCalled { + t.Fatal("parallel dry-run bootstrapped the daemon") + } + if !strings.Contains(stdout, "parallel task plan: demo") || + !strings.Contains(stdout, "no changes were made") { + t.Fatalf("unexpected dry-run output: %q", stdout) + } + if status := runParallelPlanGit(t, workspaceRoot, "status", "--porcelain"); status != "" { + t.Fatalf("parallel dry-run changed repository: %q", status) + } } func TestTasksRunCommandAutoModeResolvesToStreamInNonInteractiveExecution(t *testing.T) { diff --git a/internal/cli/root_test.go b/internal/cli/root_test.go index 4e4d5ef5..9e7b7a09 100644 --- a/internal/cli/root_test.go +++ b/internal/cli/root_test.go @@ -414,7 +414,9 @@ func TestTasksRunHelpShowsDaemonTaskFlagsOnly(t *testing.T) { required := []string{ "--attach", + "--concurrent", "--detach", + "--format", "--stream", "--name", "--include-completed", @@ -423,6 +425,7 @@ func TestTasksRunHelpShowsDaemonTaskFlagsOnly(t *testing.T) { "--force", "Continue after task metadata validation fails in non-interactive mode", "--task-runtime", + "--verify-command", } for _, snippet := range required { if !strings.Contains(output, snippet) { @@ -435,8 +438,6 @@ func TestTasksRunHelpShowsDaemonTaskFlagsOnly(t *testing.T) { "--provider", "--reviews-dir", "--batch-size", - "--concurrent", - "--format", "--grouped", "--include-resolved", "--tasks-dir", @@ -1987,7 +1988,9 @@ func TestCommandStateDefaultsWithFallbacksPreservesExplicitFunctions(t *testing. func newTestCommand(state *commandState) *cobra.Command { cmd := &cobra.Command{Use: "test"} - addCommonFlags(cmd, state, commonFlagOptions{includeConcurrent: state.kind == commandKindFixReviews}) + addCommonFlags(cmd, state, commonFlagOptions{ + includeConcurrent: state.kind == commandKindTasksRun || state.kind == commandKindFixReviews, + }) if state.kind == commandKindTasksRun || state.kind == commandKindFixReviews { addWorkflowOutputFlags(cmd, state) } diff --git a/internal/cli/state.go b/internal/cli/state.go index 1327f7a7..3445c716 100644 --- a/internal/cli/state.go +++ b/internal/cli/state.go @@ -33,6 +33,7 @@ type runtimeConfig struct { autoCommit bool concurrent int batchSize int + verificationCommand string attachMode string untilClean bool maxRounds int @@ -198,7 +199,7 @@ func addCommonFlags(cmd *cobra.Command, state *commandState, opts commonFlagOpti "Include automatic commit instructions at task/batch completion", ) if opts.includeConcurrent { - cmd.Flags().IntVar(&state.concurrent, "concurrent", 1, "Number of batches to process in parallel") + cmd.Flags().IntVar(&state.concurrent, "concurrent", 1, "Maximum number of jobs to process concurrently") } cmd.Flags().StringVar( &state.ide, @@ -309,21 +310,22 @@ func (s *commandState) buildConfig() (core.Config, error) { ReviewsDir: s.reviewsDir, TasksDir: s.tasksDir, - DryRun: s.dryRun, - AutoCommit: s.autoCommit, - Concurrent: s.concurrent, - BatchSize: s.batchSize, - AgentName: s.agentName, - IDE: core.IDE(s.ide), - Model: s.model, - AddDirs: core.NormalizeAddDirs(s.addDirs), - TailLines: s.tailLines, - ReasoningEffort: s.reasoningEffort, - AccessMode: s.accessMode, - ExplicitRuntime: s.explicitRuntime, - TaskRuntimeRules: s.taskRuntimeRules(), - IncludeCompleted: s.includeCompleted, - IncludeResolved: s.includeResolved, + DryRun: s.dryRun, + AutoCommit: s.autoCommit, + Concurrent: s.concurrent, + BatchSize: s.batchSize, + VerificationCommand: s.verificationCommand, + AgentName: s.agentName, + IDE: core.IDE(s.ide), + Model: s.model, + AddDirs: core.NormalizeAddDirs(s.addDirs), + TailLines: s.tailLines, + ReasoningEffort: s.reasoningEffort, + AccessMode: s.accessMode, + ExplicitRuntime: s.explicitRuntime, + TaskRuntimeRules: s.taskRuntimeRules(), + IncludeCompleted: s.includeCompleted, + IncludeResolved: s.includeResolved, Mode: s.mode, OutputFormat: core.OutputFormat(s.outputFormat), diff --git a/internal/cli/tasks_run_built_command_test.go b/internal/cli/tasks_run_built_command_test.go new file mode 100644 index 00000000..23efd560 --- /dev/null +++ b/internal/cli/tasks_run_built_command_test.go @@ -0,0 +1,367 @@ +package cli + +import ( + "encoding/json" + "os" + "path/filepath" + "reflect" + "sort" + "strconv" + "strings" + "testing" +) + +func TestBuiltTasksRunParallelDryRunOutputFormatsAndPrecedence(t *testing.T) { + tests := []struct { + name string + config string + args []string + wantJSON bool + wantConcurrent int + wantVerificationCommand string + }{ + { + name: "default text output", + args: []string{"--concurrent", "2", "--verify-command", "make verify"}, + wantConcurrent: 2, + wantVerificationCommand: "make verify", + }, + { + name: "explicit json output", + args: []string{ + "--concurrent", + "2", + "--verify-command", + "make verify", + "--format", + "json", + }, + wantJSON: true, + wantConcurrent: 2, + wantVerificationCommand: "make verify", + }, + { + name: "workspace json output", + config: ` +[tasks.run] +concurrent = 2 +verify_command = "make workspace-verify" +output_format = "json" +`, + wantJSON: true, + wantConcurrent: 2, + wantVerificationCommand: "make workspace-verify", + }, + { + name: "explicit text overrides workspace json", + config: ` +[tasks.run] +concurrent = 2 +verify_command = "make workspace-verify" +output_format = "json" +`, + args: []string{"--format", "text"}, + wantConcurrent: 2, + wantVerificationCommand: "make workspace-verify", + }, + { + name: "explicit runtime flags override workspace values", + config: ` +[tasks.run] +concurrent = 4 +verify_command = "make workspace-verify" +output_format = "text" +`, + args: []string{ + "--concurrent", + "2", + "--verify-command", + "go test ./...", + "--format", + "json", + }, + wantJSON: true, + wantConcurrent: 2, + wantVerificationCommand: "go test ./...", + }, + { + name: "raw json emits the structured local plan", + args: []string{ + "--concurrent", + "2", + "--verify-command", + "make verify", + "--format", + "raw-json", + }, + wantJSON: true, + wantConcurrent: 2, + wantVerificationCommand: "make verify", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + workspaceRoot, homeDir := makeBuiltTasksRunParallelFixture(t, tt.config) + before := snapshotBuiltTasksRunDryPlanState(t, workspaceRoot, homeDir) + + args := []string{"tasks", "run", "demo", "--dry-run"} + args = append(args, tt.args...) + stdout, stderr, exitCode := runCLICommand(t, workspaceRoot, args...) + if exitCode != 0 { + t.Fatalf("tasks run exit=%d\nstdout:\n%s\nstderr:\n%s", exitCode, stdout, stderr) + } + + if tt.wantJSON { + var plan parallelTaskDryRunPlan + if err := json.Unmarshal([]byte(stdout), &plan); err != nil { + t.Fatalf("decode task plan: %v\nstdout:\n%s", err, stdout) + } + assertBuiltTaskRunPlan(t, plan, tt.wantConcurrent, tt.wantVerificationCommand) + if strings.Contains(stdout, "parallel task plan:") { + t.Fatalf("JSON output contains human-readable prefix: %q", stdout) + } + } else { + if !strings.Contains(stdout, "parallel task plan: demo") || + !strings.Contains(stdout, "concurrent="+strconv.Itoa(tt.wantConcurrent)) || + !strings.Contains(stdout, "verification: "+tt.wantVerificationCommand) { + t.Fatalf("unexpected text plan:\n%s", stdout) + } + var plan parallelTaskDryRunPlan + if err := json.Unmarshal([]byte(stdout), &plan); err == nil { + t.Fatalf("text output unexpectedly decoded as JSON: %#v", plan) + } + } + + after := snapshotBuiltTasksRunDryPlanState(t, workspaceRoot, homeDir) + if !reflect.DeepEqual(after, before) { + t.Fatalf("parallel dry-run mutated state\nbefore: %#v\nafter: %#v", before, after) + } + }) + } +} + +func TestBuiltTasksRunRejectsInvalidFormatsBeforeMutation(t *testing.T) { + for _, format := range []string{"yaml", "JSON"} { + t.Run(format, func(t *testing.T) { + workspaceRoot, homeDir := makeBuiltTasksRunParallelFixture(t, "") + before := snapshotBuiltTasksRunDryPlanState(t, workspaceRoot, homeDir) + + stdout, stderr, exitCode := runCLICommand( + t, + workspaceRoot, + "tasks", + "run", + "demo", + "--dry-run", + "--concurrent", + "2", + "--format", + format, + ) + if exitCode != 1 { + t.Fatalf("invalid format exit=%d, want 1\nstdout:\n%s\nstderr:\n%s", exitCode, stdout, stderr) + } + if stdout != "" || !strings.Contains(stderr, "invalid output format") { + t.Fatalf("unexpected invalid-format output\nstdout:\n%s\nstderr:\n%s", stdout, stderr) + } + + after := snapshotBuiltTasksRunDryPlanState(t, workspaceRoot, homeDir) + if !reflect.DeepEqual(after, before) { + t.Fatalf("invalid format mutated state\nbefore: %#v\nafter: %#v", before, after) + } + }) + } +} + +func TestBuiltTasksRunRejectsInvalidParallelFlagsAndConfiguration(t *testing.T) { + tests := []struct { + name string + config string + args []string + wantExit int + wantStderr string + }{ + { + name: "explicit zero concurrency", + args: []string{"--concurrent", "0"}, + wantExit: 1, + wantStderr: "--concurrent must be greater than zero", + }, + { + name: "explicit blank verification command", + args: []string{"--concurrent", "2", "--verify-command", " "}, + wantExit: 1, + wantStderr: "--verify-command cannot be blank", + }, + { + name: "configured zero concurrency", + config: ` +[tasks.run] +concurrent = 0 +`, + wantExit: 2, + wantStderr: "tasks.run.concurrent must be greater than zero", + }, + { + name: "configured blank verification command", + config: ` +[tasks.run] +verify_command = " " +`, + wantExit: 2, + wantStderr: "tasks.run.verify_command cannot be blank", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + workspaceRoot, homeDir := makeBuiltTasksRunParallelFixture(t, tt.config) + before := snapshotBuiltTasksRunDryPlanState(t, workspaceRoot, homeDir) + + args := []string{"tasks", "run", "demo", "--dry-run"} + args = append(args, tt.args...) + stdout, stderr, exitCode := runCLICommand(t, workspaceRoot, args...) + if exitCode != tt.wantExit { + t.Fatalf("tasks run exit=%d, want %d\nstdout:\n%s\nstderr:\n%s", exitCode, tt.wantExit, stdout, stderr) + } + if stdout != "" || !strings.Contains(stderr, tt.wantStderr) { + t.Fatalf("unexpected validation output\nstdout:\n%s\nstderr:\n%s", stdout, stderr) + } + + after := snapshotBuiltTasksRunDryPlanState(t, workspaceRoot, homeDir) + if !reflect.DeepEqual(after, before) { + t.Fatalf("invalid input mutated state\nbefore: %#v\nafter: %#v", before, after) + } + }) + } +} + +type builtTasksRunDryPlanState struct { + GitStatus string + GitRefs string + GitWorktrees string + HomeEntries []string + RunRootPresent bool +} + +func makeBuiltTasksRunParallelFixture(t *testing.T, config string) (string, string) { + t.Helper() + + workspaceRoot, tasksDir := makeValidateTasksWorkspace(t, "demo") + writeRawTaskFileForCLI(t, tasksDir, "task_01.md", cliTaskMarkdown( + []string{ + "status: pending", + "title: First Task", + "type: backend", + "complexity: low", + "dependencies: []", + }, + "# Task 1: First Task", + )) + writeRawTaskFileForCLI(t, tasksDir, "task_02.md", cliTaskMarkdown( + []string{ + "status: pending", + "title: Second Task", + "type: backend", + "complexity: low", + "dependencies:", + " - task_01", + }, + "# Task 2: Second Task", + )) + writeCLIWorkspaceConfig(t, workspaceRoot, config) + + runParallelPlanGit(t, workspaceRoot, "init", "-q", "-b", "main") + runParallelPlanGit(t, workspaceRoot, "config", "user.email", "tasks-run-built@example.com") + runParallelPlanGit(t, workspaceRoot, "config", "user.name", "Tasks Run Built Test") + runParallelPlanGit(t, workspaceRoot, "add", ".productize") + runParallelPlanGit(t, workspaceRoot, "commit", "--no-gpg-sign", "-m", "fixture") + + homeDir := t.TempDir() + t.Setenv("HOME", homeDir) + t.Setenv("XDG_CONFIG_HOME", filepath.Join(homeDir, "xdg")) + return workspaceRoot, homeDir +} + +func snapshotBuiltTasksRunDryPlanState( + t *testing.T, + workspaceRoot string, + homeDir string, +) builtTasksRunDryPlanState { + t.Helper() + + return builtTasksRunDryPlanState{ + GitStatus: runParallelPlanGit(t, workspaceRoot, "status", "--porcelain"), + GitRefs: runParallelPlanGit(t, workspaceRoot, "show-ref", "--heads"), + GitWorktrees: runParallelPlanGit(t, workspaceRoot, "worktree", "list", "--porcelain"), + HomeEntries: relativeDirectoryEntriesForCLITest(t, homeDir), + RunRootPresent: pathExistsForCLITest(filepath.Join(workspaceRoot, ".productize", "runs")), + } +} + +func relativeDirectoryEntriesForCLITest(t *testing.T, root string) []string { + t.Helper() + + entries := make([]string, 0) + err := filepath.WalkDir(root, func(path string, entry os.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if path == root { + return nil + } + relative, err := filepath.Rel(root, path) + if err != nil { + return err + } + kind := "file" + if entry.IsDir() { + kind = "dir" + } + entries = append(entries, filepath.ToSlash(relative)+":"+kind) + return nil + }) + if err != nil { + t.Fatalf("snapshot directory %s: %v", root, err) + } + sort.Strings(entries) + return entries +} + +func pathExistsForCLITest(path string) bool { + _, err := os.Stat(path) + return err == nil +} + +func assertBuiltTaskRunPlan( + t *testing.T, + plan parallelTaskDryRunPlan, + wantConcurrent int, + wantVerificationCommand string, +) { + t.Helper() + + if plan.SchemaVersion != 1 || plan.Status != "planned" || !plan.DryRun { + t.Fatalf("unexpected plan envelope: %#v", plan) + } + if plan.Workflow != "demo" || plan.Concurrent != wantConcurrent { + t.Fatalf("unexpected plan target: %#v", plan) + } + if plan.VerificationCommand != wantVerificationCommand { + t.Fatalf("verification command = %q, want %q", plan.VerificationCommand, wantVerificationCommand) + } + wantWaves := [][]string{{"task_01"}, {"task_02"}} + if !reflect.DeepEqual(plan.Waves, wantWaves) { + t.Fatalf("waves = %#v, want %#v", plan.Waves, wantWaves) + } + if len(plan.Tasks) != 2 || plan.Tasks[0].TaskID != "task_01" || plan.Tasks[1].TaskID != "task_02" { + t.Fatalf("unexpected tasks: %#v", plan.Tasks) + } + if !reflect.DeepEqual(plan.Tasks[1].Dependencies, []string{"task_01"}) { + t.Fatalf("task_02 dependencies = %#v", plan.Tasks[1].Dependencies) + } + if !reflect.DeepEqual(plan.MergeOrder, []string{"task_01", "task_02"}) { + t.Fatalf("merge order = %#v", plan.MergeOrder) + } +} diff --git a/internal/cli/testdata/tasks_run_help.golden b/internal/cli/testdata/tasks_run_help.golden index b76f60a4..86f1d18b 100644 --- a/internal/cli/testdata/tasks_run_help.golden +++ b/internal/cli/testdata/tasks_run_help.golden @@ -2,12 +2,15 @@ Start a task workflow through the shared home-scoped daemon. The CLI resolves the workspace root and attach mode locally, ensures the daemon is running, and then sends the workflow request over the daemon transport. +Concurrency controls execution scheduling independently of whether the client +streams the run or detaches. Usage: productize tasks run [slug] [flags] Examples: productize tasks run my-feature + productize tasks run my-feature --concurrent 3 productize tasks run my-feature --stream productize tasks run my-feature --detach productize tasks run --name my-feature --dry-run @@ -17,9 +20,11 @@ Flags: --add-dir strings Additional directory to allow for ACP runtimes that support extra writable roots (currently claude and codex; repeatable or comma-separated) --attach string Attach mode: auto, stream, or detach (default "auto") --auto-commit Include automatic commit instructions at task/batch completion + --concurrent int Maximum number of jobs to process concurrently (default 1) --detach Start the run without attaching a client --dry-run Only generate prompts; do not run IDE tool --force Continue after task metadata validation fails in non-interactive mode + --format string Output format: text, json, or raw-json (default "text") -h, --help help for run --ide string ACP runtime to use. Built-in and enabled extension runtimes are validated against the active runtime catalog. (default "codex") --include-completed Include completed tasks @@ -33,3 +38,4 @@ Flags: --tail-lines int Maximum number of log lines to retain per job in run snapshots (0 = full history) --task-runtime task-runtime Per-task runtime override rule for task runs (repeatable). Use key=value pairs such as type=frontend,ide=codex,model=gpt-5.5 or id=task_01,reasoning-effort=xhigh --timeout string Activity timeout duration (e.g., 5m, 30s). Job canceled if no output received within this period. (default "10m") + --verify-command string Shell command used to verify integrated parallel task changes diff --git a/internal/cli/workspace_config.go b/internal/cli/workspace_config.go index cb22e450..8facc08f 100644 --- a/internal/cli/workspace_config.go +++ b/internal/cli/workspace_config.go @@ -129,6 +129,10 @@ func (s *commandState) applyProjectConfig(cmd *cobra.Command, cfg workspace.Proj switch s.kind { case commandKindTasksRun: applyConfig(cmd, "attach", cfg.Runs.DefaultAttachMode, func(val string) { s.attachMode = val }) + applyConfig(cmd, "concurrent", cfg.Tasks.Run.Concurrent, func(val int) { s.concurrent = val }) + applyConfig(cmd, "verify-command", cfg.Tasks.Run.VerifyCommand, func(val string) { + s.verificationCommand = val + }) applyConfig(cmd, "format", cfg.Tasks.Run.OutputFormat, func(val string) { s.outputFormat = val }) s.configuredTaskRuntimeRules = model.CloneTaskRuntimeRules( derefTaskRuntimeRulesConfig(cfg.Tasks.Run.TaskRuntimeRules), diff --git a/internal/cli/workspace_config_test.go b/internal/cli/workspace_config_test.go index 57176806..d84f4051 100644 --- a/internal/cli/workspace_config_test.go +++ b/internal/cli/workspace_config_test.go @@ -36,12 +36,15 @@ timeout = "5m" add_dirs = ["../shared", "../docs"] [tasks.run] +concurrent = 4 include_completed = true +verify_command = "make verify" `) state := newCommandState(commandKindTasksRun, core.ModePRDTasks) cmd := newTestCommand(state) cmd.Flags().Bool("include-completed", false, "include completed") + cmd.Flags().StringVar(&state.verificationCommand, "verify-command", "", "verify command") chdirCLITest(t, startDir) @@ -64,6 +67,12 @@ include_completed = true if !state.includeCompleted { t.Fatalf("expected includeCompleted=true") } + if state.concurrent != 4 { + t.Fatalf("expected tasks.run.concurrent=4, got %d", state.concurrent) + } + if state.verificationCommand != "make verify" { + t.Fatalf("expected tasks.run.verify_command to apply, got %q", state.verificationCommand) + } resolvedRoot := mustEvalSymlinksCLITest(t, root) wantDirs := []string{ filepath.Join(filepath.Dir(resolvedRoot), "shared"), @@ -119,6 +128,37 @@ batch_size = 4 } } +func TestApplyWorkspaceDefaultsPreservesExplicitTaskConcurrency(t *testing.T) { + root := t.TempDir() + isolateCLIConfigHome(t) + writeCLIWorkspaceConfig(t, root, ` +[tasks.run] +concurrent = 4 +verify_command = "make verify" +`) + + state := newCommandState(commandKindTasksRun, core.ModePRDTasks) + cmd := newTestCommand(state) + cmd.Flags().StringVar(&state.verificationCommand, "verify-command", "", "verify command") + chdirCLITest(t, root) + + if err := cmd.Flags().Set("concurrent", "2"); err != nil { + t.Fatalf("set concurrent: %v", err) + } + if err := cmd.Flags().Set("verify-command", "go test ./..."); err != nil { + t.Fatalf("set verify-command: %v", err) + } + if err := state.applyWorkspaceDefaults(context.Background(), cmd); err != nil { + t.Fatalf("apply workspace defaults: %v", err) + } + if state.concurrent != 2 { + t.Fatalf("expected explicit --concurrent to win, got %d", state.concurrent) + } + if state.verificationCommand != "go test ./..." { + t.Fatalf("expected explicit --verify-command to win, got %q", state.verificationCommand) + } +} + func TestApplyWorkspaceDefaultsCanDisableAutomaticRetries(t *testing.T) { isolateCLIConfigHome(t) root := t.TempDir() @@ -194,7 +234,7 @@ verbose = true } func TestApplyWorkspaceDefaultsUsesStartPresentationOverrides(t *testing.T) { - t.Parallel() + isolateCLIConfigHome(t) root := t.TempDir() startDir := filepath.Join(root, "pkg", "feature") diff --git a/internal/core/api.go b/internal/core/api.go index ed966e63..706f2646 100644 --- a/internal/core/api.go +++ b/internal/core/api.go @@ -105,6 +105,7 @@ type Config struct { AutoCommit bool Concurrent int BatchSize int + VerificationCommand string IDE IDE Model string AddDirs []string @@ -355,6 +356,7 @@ func (cfg Config) RuntimeConfig() *model.RuntimeConfig { AutoCommit: cfg.AutoCommit, Concurrent: cfg.Concurrent, BatchSize: cfg.BatchSize, + VerificationCommand: cfg.VerificationCommand, IDE: string(cfg.IDE), Model: cfg.Model, AddDirs: NormalizeAddDirs(cfg.AddDirs), diff --git a/internal/core/extension/host_helpers.go b/internal/core/extension/host_helpers.go index be258e48..c90939e4 100644 --- a/internal/core/extension/host_helpers.go +++ b/internal/core/extension/host_helpers.go @@ -141,6 +141,7 @@ type RunConfig struct { AutoCommit bool `json:"auto_commit,omitempty"` Concurrent int `json:"concurrent,omitempty"` BatchSize int `json:"batch_size,omitempty"` + VerificationCommand string `json:"verification_command,omitempty"` IDE string `json:"ide,omitempty"` Model string `json:"model,omitempty"` AddDirs []string `json:"add_dirs,omitempty"` diff --git a/internal/core/extension/host_writes.go b/internal/core/extension/host_writes.go index abe90f9b..c7b8e3b9 100644 --- a/internal/core/extension/host_writes.go +++ b/internal/core/extension/host_writes.go @@ -277,6 +277,7 @@ func (cfg RunConfig) toRuntimeConfig(workspaceRoot string, parentRunID string) * AutoCommit: cfg.AutoCommit, Concurrent: cfg.Concurrent, BatchSize: cfg.BatchSize, + VerificationCommand: strings.TrimSpace(cfg.VerificationCommand), IDE: strings.TrimSpace(cfg.IDE), Model: strings.TrimSpace(cfg.Model), AddDirs: append([]string(nil), cfg.AddDirs...), diff --git a/internal/core/kernel/commands/commands_test.go b/internal/core/kernel/commands/commands_test.go index d6c6d398..a1b1194c 100644 --- a/internal/core/kernel/commands/commands_test.go +++ b/internal/core/kernel/commands/commands_test.go @@ -351,6 +351,9 @@ func assertRuntimeConfig(t *testing.T, got *model.RuntimeConfig, want core.Confi if got.BatchSize != want.BatchSize { t.Fatalf("unexpected batch size: %d", got.BatchSize) } + if got.VerificationCommand != want.VerificationCommand { + t.Fatalf("unexpected verification command: %q", got.VerificationCommand) + } if got.IDE != string(want.IDE) { t.Fatalf("unexpected ide: %q", got.IDE) } @@ -441,6 +444,7 @@ func testCoreConfig() core.Config { AutoCommit: true, Concurrent: 2, BatchSize: 1, + VerificationCommand: "make verify", IDE: core.IDECodex, Model: "gpt-5.5", AddDirs: []string{"docs", "src"}, diff --git a/internal/core/memory/store.go b/internal/core/memory/store.go index d76519bc..a74d4106 100644 --- a/internal/core/memory/store.go +++ b/internal/core/memory/store.go @@ -4,6 +4,7 @@ import ( "fmt" "os" "path/filepath" + "sort" "strings" ) @@ -20,6 +21,8 @@ const ( "Do not duplicate facts that are obvious from the repository, PRD documents, or git history." taskGuidanceLine = "Keep only task-local execution context here. " + "Do not duplicate facts that are obvious from the repository, task file, PRD documents, or git history." + parallelIndexStart = "" + parallelIndexEnd = "" ) type WriteMode string @@ -152,6 +155,76 @@ func Prepare(tasksDir, taskFileName string) (Context, error) { }, nil } +// RebuildWorkflowIndex refreshes the generated index of integrated task-memory +// documents without rewriting human-authored shared-memory sections. +func RebuildWorkflowIndex(tasksDir string) (Document, error) { + workflow, err := ReadDocument(tasksDir, "") + if err != nil { + return Document{}, err + } + if !workflow.Exists { + if err := os.MkdirAll(Directory(tasksDir), 0o755); err != nil { + return Document{}, fmt.Errorf("prepare workflow memory dir: %w", err) + } + if err := writeIfMissing(WorkflowPath(tasksDir), workflowTemplate()); err != nil { + return Document{}, fmt.Errorf("bootstrap workflow memory: %w", err) + } + workflow, err = ReadDocument(tasksDir, "") + if err != nil { + return Document{}, err + } + } + + names, err := integratedTaskMemoryNames(Directory(tasksDir)) + if err != nil { + return Document{}, err + } + content := stripParallelMemoryIndex(workflow.Content) + if len(names) > 0 { + var generated strings.Builder + generated.WriteString(parallelIndexStart) + generated.WriteString("\n## Integrated Task Memory\n\n") + generated.WriteString("Read the completed dependency records relevant to the next task.\n\n") + for _, name := range names { + fmt.Fprintf(&generated, "- `%s`\n", name) + } + generated.WriteString(parallelIndexEnd) + content = strings.TrimRight(content, "\n") + "\n\n" + generated.String() + "\n" + } + document, _, err := WriteDocument(tasksDir, "", content, WriteModeReplace) + return document, err +} + +func integratedTaskMemoryNames(directory string) ([]string, error) { + entries, err := os.ReadDir(directory) + if err != nil { + return nil, fmt.Errorf("read workflow memory directory: %w", err) + } + names := make([]string, 0, len(entries)) + for _, entry := range entries { + name := entry.Name() + if !entry.Type().IsRegular() || name == WorkflowFileName || filepath.Ext(name) != ".md" { + continue + } + names = append(names, name) + } + sort.Strings(names) + return names, nil +} + +func stripParallelMemoryIndex(content string) string { + start := strings.Index(content, parallelIndexStart) + if start < 0 { + return content + } + endRelative := strings.Index(content[start:], parallelIndexEnd) + if endRelative < 0 { + return strings.TrimRight(content[:start], "\n") + "\n" + } + end := start + endRelative + len(parallelIndexEnd) + return strings.TrimRight(content[:start]+content[end:], "\n") + "\n" +} + func writeIfMissing(path, content string) error { if _, err := os.Stat(path); err == nil { return nil diff --git a/internal/core/memory/store_test.go b/internal/core/memory/store_test.go index 0cf24e06..c4867eb8 100644 --- a/internal/core/memory/store_test.go +++ b/internal/core/memory/store_test.go @@ -43,6 +43,44 @@ func TestPrepareBootstrapsWorkflowAndTaskMemory(t *testing.T) { } } +func TestRebuildWorkflowIndexPreservesSharedContentAndSortsTaskMemory(t *testing.T) { + t.Parallel() + + tasksDir := t.TempDir() + if _, _, err := WriteDocument( + tasksDir, + "", + "# Workflow Memory\n\n## Shared Decisions\n\nKeep this.\n", + WriteModeReplace, + ); err != nil { + t.Fatalf("write shared memory: %v", err) + } + for _, name := range []string{"task_10.md", "task_02.md"} { + if _, _, err := WriteDocument(tasksDir, name, "# "+name+"\n", WriteModeReplace); err != nil { + t.Fatalf("write %s: %v", name, err) + } + } + + first, err := RebuildWorkflowIndex(tasksDir) + if err != nil { + t.Fatalf("RebuildWorkflowIndex() error = %v", err) + } + if !strings.Contains(first.Content, "Keep this.") { + t.Fatalf("shared content was not preserved:\n%s", first.Content) + } + if strings.Index(first.Content, "task_02.md") > strings.Index(first.Content, "task_10.md") { + t.Fatalf("task-memory index is not sorted:\n%s", first.Content) + } + + second, err := RebuildWorkflowIndex(tasksDir) + if err != nil { + t.Fatalf("second RebuildWorkflowIndex() error = %v", err) + } + if second.Content != first.Content { + t.Fatalf("rebuild is not idempotent\nfirst:\n%s\nsecond:\n%s", first.Content, second.Content) + } +} + func TestPreparePreservesExistingMemoryFiles(t *testing.T) { t.Parallel() diff --git a/internal/core/model/runtime_config.go b/internal/core/model/runtime_config.go index 9f3b2ad4..06d1fe0a 100644 --- a/internal/core/model/runtime_config.go +++ b/internal/core/model/runtime_config.go @@ -27,6 +27,7 @@ type RuntimeConfig struct { AutoCommit bool Concurrent int BatchSize int + VerificationCommand string IDE string Model string AddDirs []string @@ -36,6 +37,8 @@ type RuntimeConfig struct { AgentName string ExplicitRuntime ExplicitRuntimeFlags TaskRuntimeRules []TaskRuntimeRule + TaskIDs []string + TaskMemoryLocalOnly bool Mode ExecutionMode OutputFormat OutputFormat Verbose bool diff --git a/internal/core/model/task_runtime.go b/internal/core/model/task_runtime.go index 57f5c7d8..ffb826b2 100644 --- a/internal/core/model/task_runtime.go +++ b/internal/core/model/task_runtime.go @@ -93,6 +93,7 @@ func (cfg *RuntimeConfig) Clone() *RuntimeConfig { cloned := *cfg cloned.AddDirs = append([]string(nil), cfg.AddDirs...) cloned.TaskRuntimeRules = CloneTaskRuntimeRules(cfg.TaskRuntimeRules) + cloned.TaskIDs = append([]string(nil), cfg.TaskIDs...) return &cloned } diff --git a/internal/core/plan/prepare.go b/internal/core/plan/prepare.go index 89dee349..c45e808e 100644 --- a/internal/core/plan/prepare.go +++ b/internal/core/plan/prepare.go @@ -253,7 +253,50 @@ func resolvePreparedEntries( if err != nil { return nil, err } - return postDiscover.Entries, nil + return selectTaskEntries(postDiscover.Entries, cfg) +} + +func selectTaskEntries(entries []model.IssueEntry, cfg *model.RuntimeConfig) ([]model.IssueEntry, error) { + if cfg == nil || cfg.Mode != model.ExecutionModePRDTasks || len(cfg.TaskIDs) == 0 { + return entries, nil + } + + selectedIDs := make(map[string]struct{}, len(cfg.TaskIDs)) + for _, taskID := range cfg.TaskIDs { + normalized := normalizeSelectedTaskID(taskID) + if normalized == "" { + return nil, errors.New("selected task id must not be empty") + } + selectedIDs[normalized] = struct{}{} + } + + selected := make([]model.IssueEntry, 0, len(selectedIDs)) + found := make(map[string]struct{}, len(selectedIDs)) + for _, entry := range entries { + normalized := normalizeSelectedTaskID(entry.Name) + if _, ok := selectedIDs[normalized]; !ok { + continue + } + selected = append(selected, entry) + found[normalized] = struct{}{} + } + + missing := make([]string, 0) + for taskID := range selectedIDs { + if _, ok := found[taskID]; !ok { + missing = append(missing, taskID) + } + } + sort.Strings(missing) + if len(missing) > 0 { + return nil, fmt.Errorf("selected tasks not found or already completed: %s", strings.Join(missing, ", ")) + } + return selected, nil +} + +func normalizeSelectedTaskID(taskID string) string { + trimmed := strings.TrimSpace(taskID) + return strings.TrimSuffix(trimmed, filepath.Ext(trimmed)) } func dispatchPlanPreDiscover( @@ -811,6 +854,14 @@ func prepareBatchTaskContext( TaskPath: memoryCtx.Task.Path, WorkflowNeedsCompaction: memoryCtx.Workflow.NeedsCompaction, TaskNeedsCompaction: memoryCtx.Task.NeedsCompaction, + TaskLocalOnly: cfg.TaskMemoryLocalOnly, + } + for _, dependency := range taskData.Dependencies { + dependencyFile := strings.TrimSuffix(dependency, filepath.Ext(dependency)) + ".md" + params.Memory.DependencyPaths = append( + params.Memory.DependencyPaths, + memory.TaskPath(cfg.TasksDir, dependencyFile), + ) } return taskData, nil } diff --git a/internal/core/plan/prepare_test.go b/internal/core/plan/prepare_test.go index ad397fdf..5a4da267 100644 --- a/internal/core/plan/prepare_test.go +++ b/internal/core/plan/prepare_test.go @@ -9,6 +9,7 @@ import ( "os" "path/filepath" "reflect" + "slices" "sort" "strings" "testing" @@ -1719,6 +1720,76 @@ func TestResolveInputsRejectsLegacyTasksDirInference(t *testing.T) { } } +func TestSelectTaskEntries(t *testing.T) { + t.Parallel() + + entries := []model.IssueEntry{ + {Name: "task_01.md"}, + {Name: "task_02.md"}, + {Name: "task_03.md"}, + } + tests := []struct { + name string + cfg *model.RuntimeConfig + want []string + wantErr string + }{ + { + name: "keeps every entry without an internal selection", + cfg: &model.RuntimeConfig{Mode: model.ExecutionModePRDTasks}, + want: []string{"task_01.md", "task_02.md", "task_03.md"}, + }, + { + name: "selects normalized task ids in source order", + cfg: &model.RuntimeConfig{ + Mode: model.ExecutionModePRDTasks, + TaskIDs: []string{"task_03.md", "task_01"}, + }, + want: []string{"task_01.md", "task_03.md"}, + }, + { + name: "rejects missing selections", + cfg: &model.RuntimeConfig{ + Mode: model.ExecutionModePRDTasks, + TaskIDs: []string{"task_99"}, + }, + wantErr: "selected tasks not found or already completed: task_99", + }, + { + name: "does not filter other execution modes", + cfg: &model.RuntimeConfig{ + Mode: model.ExecutionModePRReview, + TaskIDs: []string{"task_01"}, + }, + want: []string{"task_01.md", "task_02.md", "task_03.md"}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + got, err := selectTaskEntries(entries, tt.cfg) + if tt.wantErr != "" { + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("selectTaskEntries() error = %v, want containing %q", err, tt.wantErr) + } + return + } + if err != nil { + t.Fatalf("selectTaskEntries() error = %v", err) + } + gotNames := make([]string, 0, len(got)) + for _, entry := range got { + gotNames = append(gotNames, entry.Name) + } + if !slices.Equal(gotNames, tt.want) { + t.Fatalf("selectTaskEntries() = %v, want %v", gotNames, tt.want) + } + }) + } +} + func TestResolveInputsRejectsLegacyReviewsDirInference(t *testing.T) { t.Parallel() diff --git a/internal/core/projectknowledge/scan.go b/internal/core/projectknowledge/scan.go index fe0931d8..5c6f5c6e 100644 --- a/internal/core/projectknowledge/scan.go +++ b/internal/core/projectknowledge/scan.go @@ -208,12 +208,16 @@ func matchesBuiltinPrune(rel string, isDir bool) bool { if rel == ".productize" || strings.HasPrefix(rel, ".productize/") { return true } + base := path.Base(rel) + switch base { + case ".git", ".hg", ".svn": + return true + } if !isDir { return false } - base := path.Base(rel) switch base { - case ".git", ".hg", ".svn", "node_modules", "vendor", "dist", "build", "target", + case "node_modules", "vendor", "dist", "build", "target", "generated", ".next", "out", "coverage", ".cache", ".turbo", ".venv", "venv", "__pycache__", ".pytest_cache", ".mypy_cache", ".ruff_cache", ".tox", ".nox", "bin", "obj": return true diff --git a/internal/core/projectknowledge/scan_test.go b/internal/core/projectknowledge/scan_test.go index fb4af34a..77de8cf8 100644 --- a/internal/core/projectknowledge/scan_test.go +++ b/internal/core/projectknowledge/scan_test.go @@ -10,6 +10,19 @@ import ( "testing" ) +func TestMatchesBuiltinPruneIgnoresGitDirectoriesAndWorktreeFiles(t *testing.T) { + t.Parallel() + + for _, isDir := range []bool{false, true} { + if !matchesBuiltinPrune(".git", isDir) { + t.Fatalf(".git prune with isDir=%t = false, want true", isDir) + } + } + if matchesBuiltinPrune(".github", true) { + t.Fatal(".github must remain scannable") + } +} + func TestScanDetectsMixedLanguageMonorepo(t *testing.T) { t.Parallel() diff --git a/internal/core/prompt/prd.go b/internal/core/prompt/prd.go index 62badf26..5fc9f7a9 100644 --- a/internal/core/prompt/prd.go +++ b/internal/core/prompt/prd.go @@ -11,11 +11,13 @@ import ( ) type WorkflowMemoryContext struct { - Directory string `json:"directory,omitempty"` - WorkflowPath string `json:"workflow_path,omitempty"` - TaskPath string `json:"task_path,omitempty"` - WorkflowNeedsCompaction bool `json:"workflow_needs_compaction,omitempty"` - TaskNeedsCompaction bool `json:"task_needs_compaction,omitempty"` + Directory string `json:"directory,omitempty"` + WorkflowPath string `json:"workflow_path,omitempty"` + TaskPath string `json:"task_path,omitempty"` + DependencyPaths []string `json:"dependency_paths,omitempty"` + WorkflowNeedsCompaction bool `json:"workflow_needs_compaction,omitempty"` + TaskNeedsCompaction bool `json:"task_needs_compaction,omitempty"` + TaskLocalOnly bool `json:"task_local_only,omitempty"` } func buildPRDTaskPrompt(task model.IssueEntry, autoCommit bool, memory *WorkflowMemoryContext) string { @@ -132,11 +134,21 @@ func buildWorkflowMemorySection(memory *WorkflowMemoryContext) string { fmt.Fprintf(&sb, "- Memory directory: `%s`\n", memory.Directory) fmt.Fprintf(&sb, "- Shared workflow memory: `%s`\n", memory.WorkflowPath) fmt.Fprintf(&sb, "- Current task memory: `%s`\n", memory.TaskPath) + for _, dependencyPath := range memory.DependencyPaths { + fmt.Fprintf(&sb, "- Completed dependency memory: `%s`\n", dependencyPath) + } sb.WriteString("- Use installed `workflow-memory` before editing code and before finishing the task.\n") - sb.WriteString( - "- Read both memory files before implementation. " + - "Promote durable cross-task context only to shared workflow memory.\n", - ) + if memory.TaskLocalOnly { + sb.WriteString("- This task runs in an isolated parallel worktree. Read shared memory, but do not modify it.\n") + sb.WriteString( + "- Record every new decision, learning, touched surface, and correction only in current task memory.\n", + ) + } else { + sb.WriteString( + "- Read both memory files before implementation. " + + "Promote durable cross-task context only to shared workflow memory.\n", + ) + } sb.WriteString( "- Keep task-local decisions, learnings, touched surfaces, and corrections in the current task memory file.\n", ) @@ -220,7 +232,20 @@ func buildPRDSystemPromptAddendum(memory *WorkflowMemoryContext) string { "- shared workflow memory: `" + memory.WorkflowPath + "`", "- current task memory: `" + memory.TaskPath + "`", "Update task memory when objectives, decisions, learnings, touched surfaces, or corrections change.", - "Promote only durable cross-task context into shared workflow memory.", + } + for _, dependencyPath := range memory.DependencyPaths { + lines = append(lines, "- completed dependency memory: `"+dependencyPath+"`") + } + if memory.TaskLocalOnly { + lines = append( + lines, + "This task runs in an isolated parallel worktree.", + "Read shared workflow memory but DO NOT modify it in this task.", + "Write all new memory only to the current task memory file; "+ + "the coordinator will rebuild shared context after integration.", + ) + } else { + lines = append(lines, "Promote only durable cross-task context into shared workflow memory.") } if memory.WorkflowNeedsCompaction || memory.TaskNeedsCompaction { lines = append(lines, "Compact every flagged memory file before proceeding with implementation.") diff --git a/internal/core/run/executor/hooks.go b/internal/core/run/executor/hooks.go index 51ebf57e..757144b4 100644 --- a/internal/core/run/executor/hooks.go +++ b/internal/core/run/executor/hooks.go @@ -143,6 +143,7 @@ func hookRuntimeConfig(src *config) model.RuntimeConfig { AutoCommit: src.AutoCommit, Concurrent: src.Concurrent, BatchSize: src.BatchSize, + VerificationCommand: src.VerificationCommand, IDE: src.IDE, Model: src.Model, AddDirs: append([]string(nil), src.AddDirs...), @@ -181,6 +182,7 @@ func applyHookRuntimeConfig(dst *config, updated model.RuntimeConfig) { dst.AutoCommit = updated.AutoCommit dst.Concurrent = updated.Concurrent dst.BatchSize = updated.BatchSize + dst.VerificationCommand = updated.VerificationCommand dst.IDE = updated.IDE dst.Model = updated.Model dst.AddDirs = append([]string(nil), updated.AddDirs...) diff --git a/internal/core/run/internal/runshared/config.go b/internal/core/run/internal/runshared/config.go index 666b51b5..a5466d09 100644 --- a/internal/core/run/internal/runshared/config.go +++ b/internal/core/run/internal/runshared/config.go @@ -19,6 +19,7 @@ type Config struct { AutoCommit bool Concurrent int BatchSize int + VerificationCommand string IDE string Model string AddDirs []string @@ -117,6 +118,7 @@ func NewConfig(src *model.RuntimeConfig, runArtifacts model.RunArtifacts) *Confi AutoCommit: src.AutoCommit, Concurrent: src.Concurrent, BatchSize: src.BatchSize, + VerificationCommand: src.VerificationCommand, IDE: src.IDE, Model: src.Model, AddDirs: append([]string(nil), src.AddDirs...), diff --git a/internal/core/run/internal/taskworktree/git.go b/internal/core/run/internal/taskworktree/git.go new file mode 100644 index 00000000..6020b534 --- /dev/null +++ b/internal/core/run/internal/taskworktree/git.go @@ -0,0 +1,158 @@ +package taskworktree + +import ( + "bytes" + "context" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" +) + +type gitRunner interface { + output(context.Context, string, ...string) (string, error) + branchExists(context.Context, string, string) (bool, error) + refExists(context.Context, string, string) (bool, error) + isAncestor(context.Context, string, string, string) (bool, error) + worktrees(context.Context, string) (map[string]string, error) +} + +type commandGitRunner struct{} + +type gitError struct { + args []string + exitCode int + stderr string + err error +} + +func (e *gitError) Error() string { + if e == nil { + return "git command failed" + } + detail := strings.TrimSpace(e.stderr) + if detail == "" { + return fmt.Sprintf("git %s: %v", strings.Join(e.args, " "), e.err) + } + return fmt.Sprintf("git %s: %v (%s)", strings.Join(e.args, " "), e.err, detail) +} + +func (e *gitError) Unwrap() error { return e.err } + +func (commandGitRunner) output(ctx context.Context, dir string, args ...string) (string, error) { + cmd := exec.CommandContext(ctx, "git", args...) + cmd.Dir = dir + cmd.Env = append(sanitizedGitEnv(), "LC_ALL=C", "GIT_TERMINAL_PROMPT=0") + var stdout, stderr bytes.Buffer + cmd.Stdout = &stdout + cmd.Stderr = &stderr + err := cmd.Run() + if err == nil { + return stdout.String(), nil + } + if ctxErr := ctx.Err(); ctxErr != nil { + err = errors.Join(ctxErr, err) + } + exitCode := -1 + var exitErr *exec.ExitError + if errors.As(err, &exitErr) { + exitCode = exitErr.ExitCode() + } + return "", &gitError{ + args: append([]string(nil), args...), + exitCode: exitCode, + stderr: stderr.String(), + err: err, + } +} + +func (r commandGitRunner) branchExists(ctx context.Context, dir, branch string) (bool, error) { + return r.refExists(ctx, dir, "refs/heads/"+branch) +} + +func (r commandGitRunner) refExists(ctx context.Context, dir, ref string) (bool, error) { + _, err := r.output(ctx, dir, "rev-parse", "--verify", "--quiet", ref) + if err == nil { + return true, nil + } + if isExitCode(err, 1) { + return false, nil + } + return false, err +} + +func (r commandGitRunner) isAncestor(ctx context.Context, dir, ancestor, descendant string) (bool, error) { + _, err := r.output(ctx, dir, "merge-base", "--is-ancestor", ancestor, descendant) + if err == nil { + return true, nil + } + if isExitCode(err, 1) { + return false, nil + } + return false, err +} + +func (r commandGitRunner) worktrees(ctx context.Context, dir string) (map[string]string, error) { + output, err := r.output(ctx, dir, "worktree", "list", "--porcelain", "-z") + if err != nil { + return nil, fmt.Errorf("task worktree: list Git worktrees: %w", err) + } + result := make(map[string]string) + var path string + for _, line := range strings.Split(output, "\x00") { + switch { + case strings.HasPrefix(line, "worktree "): + path = strings.TrimPrefix(line, "worktree ") + canonical, canonicalErr := canonicalRegisteredPath(path) + if canonicalErr != nil { + return nil, canonicalErr + } + path = canonical + case strings.HasPrefix(line, "branch ") && path != "": + branch := strings.TrimPrefix(line, "branch refs/heads/") + result[path] = branch + case line == "": + path = "" + } + } + return result, nil +} + +func canonicalRegisteredPath(path string) (string, error) { + abs, err := filepath.Abs(path) + if err != nil { + return "", fmt.Errorf("task worktree: resolve registered worktree %q: %w", path, err) + } + resolved, err := filepath.EvalSymlinks(abs) + if err == nil { + return filepath.Clean(resolved), nil + } + if errors.Is(err, os.ErrNotExist) { + return filepath.Clean(abs), nil + } + return "", fmt.Errorf("task worktree: resolve registered worktree %q: %w", path, err) +} + +func isExitCode(err error, code int) bool { + var commandErr *gitError + return errors.As(err, &commandErr) && commandErr.exitCode == code +} + +func sanitizedGitEnv() []string { + env := os.Environ() + filtered := make([]string, 0, len(env)) + for _, kv := range env { + key := kv + if separator := strings.IndexByte(kv, '='); separator >= 0 { + key = kv[:separator] + } + switch key { + case "GIT_DIR", "GIT_WORK_TREE", "GIT_COMMON_DIR", "GIT_INDEX_FILE", "GIT_NAMESPACE": + continue + } + filtered = append(filtered, kv) + } + return filtered +} diff --git a/internal/core/run/internal/taskworktree/lifecycle.go b/internal/core/run/internal/taskworktree/lifecycle.go new file mode 100644 index 00000000..79f7b9c4 --- /dev/null +++ b/internal/core/run/internal/taskworktree/lifecycle.go @@ -0,0 +1,1261 @@ +// Package taskworktree manages Productize-owned Git worktrees used to isolate +// concurrently executed PRD tasks. +package taskworktree + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "os" + "path/filepath" + "sort" + "strings" + "time" + "unicode" + + productizeconfig "github.com/itseffi/productize/internal/config" +) + +var ( + // ErrDirtyWorkspace indicates that a caller-owned checkout has local changes. + ErrDirtyWorkspace = errors.New("task worktree: workspace is not clean") + // ErrDetachedHEAD indicates that the original checkout is not on a branch. + ErrDetachedHEAD = errors.New("task worktree: workspace has detached HEAD") + // ErrOriginalChanged indicates that finalization cannot safely update the + // original checkout because it changed since preflight. + ErrOriginalChanged = errors.New("task worktree: original checkout changed") + // ErrMergeConflict indicates that Git found conflicting paths while merging + // a task branch into the Productize-owned integration worktree. + ErrMergeConflict = errors.New("task worktree: merge conflict") +) + +const mergeCleanupTimeout = 15 * time.Second + +// Options identifies one parallel task run. WorktreesRoot is optional; when +// omitted, Productize uses ~/.productize/worktrees. +type Options struct { + WorkspaceRoot string + WorktreesRoot string + RunID string + TaskIDs []string +} + +// RepositoryState is the immutable original-checkout state captured by +// Preflight and required by Finalize. +type RepositoryState struct { + WorkspaceRoot string + Branch string + HEAD string +} + +// TaskLayout contains the deterministic branch and checkout assigned to one +// task. +type TaskLayout struct { + TaskID string + Branch string + WorktreePath string +} + +// Layout contains all Productize-owned paths and branches for one run. +type Layout struct { + RunRoot string + IntegrationBranch string + IntegrationWorktree string + Tasks []TaskLayout +} + +// Lifecycle owns the isolated Git resources for one parallel task run. +// Its plan is immutable after Preflight and its methods are safe to call from +// a coordinator that serializes Git lifecycle transitions. +type Lifecycle struct { + git gitRunner + start RepositoryState + layout Layout + tasks map[string]TaskLayout + taskBases map[string]string +} + +// CheckoutResult describes a created or resumed Productize-owned checkout. +type CheckoutResult struct { + Branch string + WorktreePath string + HEAD string + Current bool +} + +// CommitResult describes the task branch after CommitTask returns. +type CommitResult struct { + TaskID string + Commit string + Changed bool +} + +// IntegrationCommitResult describes the integration branch after +// CommitIntegration returns. +type IntegrationCommitResult struct { + Commit string + Changed bool +} + +// MergeResult describes one task branch merged into the integration branch. +type MergeResult struct { + TaskID string + TaskCommit string + IntegrationCommit string +} + +// MergeConflictError records the conflicting repository-relative paths. The +// integration worktree is clean again before this error is returned. +type MergeConflictError struct { + TaskID string + Paths []string +} + +// TaskMemoryIsolationError reports shared or cross-task memory files changed +// by one isolated task checkout. +type TaskMemoryIsolationError struct { + TaskID string + Paths []string +} + +func (e *TaskMemoryIsolationError) Error() string { + if e == nil { + return "task worktree: task memory isolation violated" + } + return fmt.Sprintf( + "task worktree: task %s changed memory outside its task-local file: %s", + e.TaskID, + strings.Join(e.Paths, ", "), + ) +} + +func (e *MergeConflictError) Error() string { + if e == nil { + return ErrMergeConflict.Error() + } + if len(e.Paths) == 0 { + return fmt.Sprintf("%s while merging %s", ErrMergeConflict, e.TaskID) + } + return fmt.Sprintf("%s while merging %s: %s", ErrMergeConflict, e.TaskID, strings.Join(e.Paths, ", ")) +} + +// Unwrap supports errors.Is(err, ErrMergeConflict). +func (*MergeConflictError) Unwrap() error { return ErrMergeConflict } + +// FinalizeResult describes the safe fast-forward of the original checkout. +type FinalizeResult struct { + PreviousHEAD string + FinalHEAD string + Current bool +} + +// TaskOutcome controls cleanup. Only successful, clean task worktrees are +// removed; failed and conflicted worktrees are retained for diagnosis. +type TaskOutcome string + +const ( + TaskSucceeded TaskOutcome = "succeeded" + TaskFailed TaskOutcome = "failed" + TaskConflicted TaskOutcome = "conflicted" +) + +// CleanupResult reports which resources were removed or retained. +type CleanupResult struct { + Removed []string + Retained []string +} + +// Preflight validates a clean, branch-attached Git repository and produces a +// deterministic Productize-owned worktree plan without writing anything. +func Preflight(ctx context.Context, opts Options) (*Lifecycle, error) { + return preflightWithRunner(ctx, opts, commandGitRunner{}) +} + +func preflightWithRunner(ctx context.Context, opts Options, runner gitRunner) (*Lifecycle, error) { + if runner == nil { + return nil, errors.New("task worktree: missing Git runner") + } + if strings.TrimSpace(opts.RunID) == "" { + return nil, errors.New("task worktree: run ID is required") + } + start, err := resolveStartState(ctx, runner, opts.WorkspaceRoot) + if err != nil { + return nil, err + } + layout, tasks, err := buildLayout(opts, start.WorkspaceRoot) + if err != nil { + return nil, err + } + lifecycle := &Lifecycle{ + git: runner, + start: start, + layout: layout, + tasks: tasks, + taskBases: make(map[string]string, len(tasks)), + } + if err := lifecycle.validateOwnedResources(ctx); err != nil { + return nil, err + } + return lifecycle, nil +} + +func resolveStartState(ctx context.Context, runner gitRunner, workspaceRoot string) (RepositoryState, error) { + root, err := canonicalDirectory(workspaceRoot) + if err != nil { + return RepositoryState{}, fmt.Errorf("task worktree: resolve workspace root: %w", err) + } + repoRoot, err := runner.output(ctx, root, "rev-parse", "--show-toplevel") + if err != nil { + return RepositoryState{}, fmt.Errorf("task worktree: resolve Git repository root: %w", err) + } + canonicalRepoRoot, err := canonicalDirectory(strings.TrimSpace(repoRoot)) + if err != nil { + return RepositoryState{}, fmt.Errorf("task worktree: resolve Git top level: %w", err) + } + if canonicalRepoRoot != root { + return RepositoryState{}, fmt.Errorf( + "task worktree: workspace root %q is not Git top level %q", + root, + canonicalRepoRoot, + ) + } + + branch, err := runner.output(ctx, root, "symbolic-ref", "--quiet", "--short", "HEAD") + if err != nil { + if isExitCode(err, 1) { + return RepositoryState{}, ErrDetachedHEAD + } + return RepositoryState{}, fmt.Errorf("task worktree: resolve current branch: %w", err) + } + branch = strings.TrimSpace(branch) + if branch == "" { + return RepositoryState{}, ErrDetachedHEAD + } + head, err := revParseHEAD(ctx, runner, root) + if err != nil { + return RepositoryState{}, err + } + if err := requireClean(ctx, runner, root); err != nil { + return RepositoryState{}, err + } + return RepositoryState{WorkspaceRoot: root, Branch: branch, HEAD: head}, nil +} + +func buildLayout(opts Options, root string) (Layout, map[string]TaskLayout, error) { + worktreesRoot, err := resolveWorktreesRoot(opts.WorktreesRoot) + if err != nil { + return Layout{}, nil, err + } + if pathWithin(worktreesRoot, root) { + return Layout{}, nil, fmt.Errorf( + "task worktree: worktrees root %q must be outside workspace %q", + worktreesRoot, + root, + ) + } + workspaceKey := safeComponent(filepath.Base(root)) + "-" + shortHash(root) + runKey := safeComponent(opts.RunID) + "-" + shortHash(opts.RunID) + runRoot := filepath.Join(worktreesRoot, workspaceKey, runKey) + branchPrefix := "productize/" + runKey + + taskIDs, err := normalizeTaskIDs(opts.TaskIDs) + if err != nil { + return Layout{}, nil, err + } + layout := Layout{ + RunRoot: runRoot, + IntegrationBranch: branchPrefix + "/integration", + IntegrationWorktree: filepath.Join(runRoot, "integration"), + Tasks: make([]TaskLayout, 0, len(taskIDs)), + } + tasks := make(map[string]TaskLayout, len(taskIDs)) + usedKeys := make(map[string]string, len(taskIDs)) + for _, taskID := range taskIDs { + taskKey := safeComponent(taskID) + if prior, exists := usedKeys[taskKey]; exists { + return Layout{}, nil, fmt.Errorf( + "task worktree: task IDs %q and %q map to the same safe name", + prior, + taskID, + ) + } + usedKeys[taskKey] = taskID + taskLayout := TaskLayout{ + TaskID: taskID, + Branch: branchPrefix + "/task/" + taskKey, + WorktreePath: filepath.Join(runRoot, "tasks", taskKey), + } + layout.Tasks = append(layout.Tasks, taskLayout) + tasks[taskID] = taskLayout + } + + return layout, tasks, nil +} + +// StartState returns the original checkout state captured during preflight. +func (l *Lifecycle) StartState() RepositoryState { return l.start } + +// Layout returns a copy of the deterministic run layout. +func (l *Lifecycle) Layout() Layout { + result := l.layout + result.Tasks = append([]TaskLayout(nil), l.layout.Tasks...) + return result +} + +// TaskWorktree returns the planned branch and worktree for taskID. +func (l *Lifecycle) TaskWorktree(taskID string) (TaskLayout, error) { + return l.task(taskID) +} + +// CreateIntegration creates or resumes the Productize-owned integration +// branch and worktree from the starting commit. +func (l *Lifecycle) CreateIntegration(ctx context.Context) (CheckoutResult, error) { + checkout, err := l.createOwnedWorktree( + ctx, + l.layout.IntegrationWorktree, + l.layout.IntegrationBranch, + l.start.HEAD, + ) + if err != nil { + return CheckoutResult{}, err + } + valid, err := l.git.isAncestor( + ctx, + l.layout.IntegrationWorktree, + l.start.HEAD, + checkout.HEAD, + ) + if err != nil { + return CheckoutResult{}, fmt.Errorf("task worktree: validate integration base: %w", err) + } + if !valid { + return CheckoutResult{}, fmt.Errorf( + "task worktree: integration HEAD %s does not descend from starting commit %s", + checkout.HEAD, + l.start.HEAD, + ) + } + return checkout, nil +} + +// CreateTask creates or resumes one Productize-owned task worktree from the +// integration branch's current HEAD. Call this after prior waves are merged. +func (l *Lifecycle) CreateTask(ctx context.Context, taskID string) (CheckoutResult, error) { + task, err := l.task(taskID) + if err != nil { + return CheckoutResult{}, err + } + if err := l.requireOwnedCheckout(ctx, l.layout.IntegrationWorktree, l.layout.IntegrationBranch); err != nil { + return CheckoutResult{}, fmt.Errorf("task worktree: integration checkout is not ready: %w", err) + } + base, err := revParseHEAD(ctx, l.git, l.layout.IntegrationWorktree) + if err != nil { + return CheckoutResult{}, err + } + checkout, err := l.createOwnedWorktree(ctx, task.WorktreePath, task.Branch, base) + if err != nil { + return CheckoutResult{}, err + } + valid, err := l.git.isAncestor(ctx, task.WorktreePath, base, checkout.HEAD) + if err != nil { + return CheckoutResult{}, fmt.Errorf("task worktree: validate base for %s: %w", taskID, err) + } + if !valid { + return CheckoutResult{}, fmt.Errorf( + "task worktree: task %s HEAD %s does not descend from wave base %s", + taskID, + checkout.HEAD, + base, + ) + } + l.taskBases[task.TaskID] = base + return checkout, nil +} + +// CommitTask commits all tracked and untracked changes in an owned task +// worktree. If the agent already committed its changes, CommitTask returns the +// current task HEAD without creating an empty commit. +func (l *Lifecycle) CommitTask(ctx context.Context, taskID, message string) (CommitResult, error) { + task, err := l.task(taskID) + if err != nil { + return CommitResult{}, err + } + if err := l.requireOwnedCheckout(ctx, task.WorktreePath, task.Branch); err != nil { + return CommitResult{}, err + } + if err := l.requireTaskBase(ctx, task); err != nil { + return CommitResult{}, err + } + commit, changed, err := l.commitOwnedCheckout(ctx, task.WorktreePath, message) + if err != nil { + return CommitResult{}, err + } + return CommitResult{TaskID: taskID, Commit: commit, Changed: changed}, nil +} + +// CommitIntegration commits coordinator-owned changes, such as a rebuilt +// shared-memory index, in the integration worktree. A clean integration +// checkout returns its current HEAD without creating an empty commit. +func (l *Lifecycle) CommitIntegration( + ctx context.Context, + message string, +) (IntegrationCommitResult, error) { + if err := l.requireOwnedCheckout( + ctx, + l.layout.IntegrationWorktree, + l.layout.IntegrationBranch, + ); err != nil { + return IntegrationCommitResult{}, err + } + commit, changed, err := l.commitOwnedCheckout(ctx, l.layout.IntegrationWorktree, message) + if err != nil { + return IntegrationCommitResult{}, err + } + return IntegrationCommitResult{Commit: commit, Changed: changed}, nil +} + +// RequireIntegrationClean verifies that the owned integration checkout has no +// tracked or untracked changes. +func (l *Lifecycle) RequireIntegrationClean(ctx context.Context) error { + if err := l.requireOwnedCheckout( + ctx, + l.layout.IntegrationWorktree, + l.layout.IntegrationBranch, + ); err != nil { + return err + } + if err := requireClean(ctx, l.git, l.layout.IntegrationWorktree); err != nil { + return fmt.Errorf("task worktree: integration checkout: %w", err) + } + return nil +} + +// IntegrationHEAD returns the current integration commit after validating the +// Productize-owned checkout. +func (l *Lifecycle) IntegrationHEAD(ctx context.Context) (string, error) { + if err := l.requireOwnedCheckout( + ctx, + l.layout.IntegrationWorktree, + l.layout.IntegrationBranch, + ); err != nil { + return "", err + } + return revParseHEAD(ctx, l.git, l.layout.IntegrationWorktree) +} + +// ValidateTaskMemoryIsolation rejects shared or cross-task memory changes in +// an isolated task checkout. memoryRoot must be repository-relative. +func (l *Lifecycle) ValidateTaskMemoryIsolation( + ctx context.Context, + taskID string, + memoryRoot string, +) error { + task, err := l.task(taskID) + if err != nil { + return err + } + if err := l.requireOwnedCheckout(ctx, task.WorktreePath, task.Branch); err != nil { + return err + } + base, err := l.taskBase(task.TaskID) + if err != nil { + return err + } + paths, err := l.changedTaskPaths(ctx, task, base) + if err != nil { + return err + } + root := strings.Trim(filepath.ToSlash(filepath.Clean(memoryRoot)), "/") + if root == "" || root == "." || strings.HasPrefix(root, "../") { + return fmt.Errorf("task worktree: invalid repository-relative memory root %q", memoryRoot) + } + allowed := root + "/" + task.TaskID + ".md" + violations := make([]string, 0) + for _, path := range paths { + if path != allowed && strings.HasPrefix(path, root+"/") { + violations = append(violations, path) + } + } + if len(violations) > 0 { + sort.Strings(violations) + return &TaskMemoryIsolationError{TaskID: task.TaskID, Paths: violations} + } + return nil +} + +// MergeTasks merges task branches into the owned integration branch in sorted +// task-ID order. A failed merge is aborted before the method returns. +func (l *Lifecycle) MergeTasks(ctx context.Context, taskIDs []string) ([]MergeResult, error) { + if err := l.requireOwnedCheckout(ctx, l.layout.IntegrationWorktree, l.layout.IntegrationBranch); err != nil { + return nil, err + } + if err := requireClean(ctx, l.git, l.layout.IntegrationWorktree); err != nil { + return nil, fmt.Errorf("task worktree: integration checkout: %w", err) + } + ordered, err := l.orderedTasks(taskIDs) + if err != nil { + return nil, err + } + results := make([]MergeResult, 0, len(ordered)) + for _, task := range ordered { + if err := l.requireOwnedCheckout(ctx, task.WorktreePath, task.Branch); err != nil { + return results, err + } + if err := requireClean(ctx, l.git, task.WorktreePath); err != nil { + return results, fmt.Errorf("task worktree: task %s: %w", task.TaskID, err) + } + if err := l.requireTaskBase(ctx, task); err != nil { + return results, err + } + taskCommit, err := revParseHEAD(ctx, l.git, task.WorktreePath) + if err != nil { + return results, err + } + _, mergeErr := l.git.output( + ctx, + l.layout.IntegrationWorktree, + "merge", + "--no-ff", + "--no-edit", + "--no-gpg-sign", + "-m", + "productize: merge "+task.TaskID, + task.Branch, + ) + if mergeErr != nil { + cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), mergeCleanupTimeout) + conflicts, conflictErr := l.conflictedPaths(cleanupCtx) + abortErr := l.abortMergeIfActive(cleanupCtx) + cancel() + if conflictErr != nil { + return results, errors.Join( + fmt.Errorf("task worktree: merge %s: %w", task.TaskID, mergeErr), + conflictErr, + abortErr, + ) + } + if len(conflicts) > 0 { + return results, errors.Join( + &MergeConflictError{TaskID: task.TaskID, Paths: conflicts}, + abortErr, + ) + } + return results, errors.Join( + fmt.Errorf("task worktree: merge %s: %w", task.TaskID, mergeErr), + abortErr, + ) + } + integrationCommit, err := revParseHEAD(ctx, l.git, l.layout.IntegrationWorktree) + if err != nil { + return results, err + } + results = append(results, MergeResult{ + TaskID: task.TaskID, + TaskCommit: taskCommit, + IntegrationCommit: integrationCommit, + }) + } + return results, nil +} + +// Finalize fast-forwards the original branch to the integration branch only +// when the original checkout is still clean, on the starting branch, and at +// the starting commit. +func (l *Lifecycle) Finalize(ctx context.Context) (FinalizeResult, error) { + if err := l.RequireIntegrationClean(ctx); err != nil { + return FinalizeResult{}, err + } + integrationHEAD, err := revParseHEAD(ctx, l.git, l.layout.IntegrationWorktree) + if err != nil { + return FinalizeResult{}, err + } + branch, err := l.git.output(ctx, l.start.WorkspaceRoot, "symbolic-ref", "--quiet", "--short", "HEAD") + if err != nil || strings.TrimSpace(branch) != l.start.Branch { + return FinalizeResult{}, fmt.Errorf("%w: expected branch %q", ErrOriginalChanged, l.start.Branch) + } + head, err := revParseHEAD(ctx, l.git, l.start.WorkspaceRoot) + if err != nil { + return FinalizeResult{}, fmt.Errorf("%w: resolve original HEAD: %v", ErrOriginalChanged, err) + } + if head != l.start.HEAD { + if head == integrationHEAD { + if err := requireClean(ctx, l.git, l.start.WorkspaceRoot); err != nil { + return FinalizeResult{}, fmt.Errorf("%w: %v", ErrOriginalChanged, err) + } + return FinalizeResult{PreviousHEAD: l.start.HEAD, FinalHEAD: integrationHEAD, Current: true}, nil + } + return FinalizeResult{}, fmt.Errorf("%w: expected HEAD %s, found %s", ErrOriginalChanged, l.start.HEAD, head) + } + if err := requireClean(ctx, l.git, l.start.WorkspaceRoot); err != nil { + return FinalizeResult{}, fmt.Errorf("%w: %v", ErrOriginalChanged, err) + } + if _, err := l.git.output( + ctx, + l.start.WorkspaceRoot, + "merge", + "--ff-only", + "--no-edit", + l.layout.IntegrationBranch, + ); err != nil { + return FinalizeResult{}, fmt.Errorf("task worktree: fast-forward original checkout: %w", err) + } + finalHEAD, err := revParseHEAD(ctx, l.git, l.start.WorkspaceRoot) + if err != nil { + return FinalizeResult{}, err + } + if finalHEAD != integrationHEAD { + return FinalizeResult{}, fmt.Errorf( + "task worktree: final HEAD %s does not match integration HEAD %s", + finalHEAD, + integrationHEAD, + ) + } + return FinalizeResult{PreviousHEAD: head, FinalHEAD: finalHEAD}, nil +} + +// Cleanup removes only successful clean task worktrees. It removes the clean +// integration worktree only after its commit is reachable from the starting +// branch. Dirty, failed, and conflicted resources are retained. +func (l *Lifecycle) Cleanup(ctx context.Context, outcomes map[string]TaskOutcome) (CleanupResult, error) { + result := CleanupResult{} + var cleanupErr error + retainIntegration := false + for _, task := range l.layout.Tasks { + outcome, exists := outcomes[task.TaskID] + if !exists || outcome != TaskSucceeded { + present, err := l.managedResourcePresent(ctx, task.WorktreePath) + if err != nil { + cleanupErr = errors.Join(cleanupErr, err) + result.Retained = append(result.Retained, task.WorktreePath) + retainIntegration = true + } else if present { + result.Retained = append(result.Retained, task.WorktreePath) + retainIntegration = true + } + continue + } + removed, err := l.removeCleanWorktree( + ctx, + task.WorktreePath, + task.Branch, + l.layout.IntegrationWorktree, + ) + if removed { + result.Removed = append(result.Removed, task.WorktreePath) + } + if err != nil { + cleanupErr = errors.Join(cleanupErr, err) + if !removed { + result.Retained = append(result.Retained, task.WorktreePath) + retainIntegration = true + } + continue + } + } + removed, retained, err := l.cleanupIntegration(ctx, retainIntegration) + if err != nil { + cleanupErr = errors.Join(cleanupErr, err) + } + if removed { + result.Removed = append(result.Removed, l.layout.IntegrationWorktree) + } + if retained { + result.Retained = append(result.Retained, l.layout.IntegrationWorktree) + } + sort.Strings(result.Removed) + sort.Strings(result.Retained) + return result, cleanupErr +} + +func (l *Lifecycle) cleanupIntegration(ctx context.Context, retain bool) (bool, bool, error) { + present, err := l.managedResourcePresent(ctx, l.layout.IntegrationWorktree) + if err != nil { + return false, true, err + } + if !present { + return false, false, nil + } + if retain { + return false, true, nil + } + merged, err := l.git.isAncestor( + ctx, + l.start.WorkspaceRoot, + l.layout.IntegrationBranch, + l.start.Branch, + ) + if err != nil { + return false, true, fmt.Errorf("task worktree: check finalized integration: %w", err) + } + if !merged { + return false, true, nil + } + removed, err := l.removeCleanWorktree( + ctx, + l.layout.IntegrationWorktree, + l.layout.IntegrationBranch, + l.start.WorkspaceRoot, + ) + if err != nil { + return removed, !removed, err + } + return removed, false, nil +} + +func (l *Lifecycle) commitOwnedCheckout( + ctx context.Context, + path string, + message string, +) (string, bool, error) { + dirty, err := isDirty(ctx, l.git, path) + if err != nil { + return "", false, err + } + if !dirty { + head, headErr := revParseHEAD(ctx, l.git, path) + return head, false, headErr + } + if strings.TrimSpace(message) == "" { + return "", false, errors.New("task worktree: commit message is required") + } + if _, err := l.git.output(ctx, path, "add", "--all"); err != nil { + return "", false, fmt.Errorf("task worktree: stage changes in %q: %w", path, err) + } + if _, err := l.git.output(ctx, path, "commit", "--no-gpg-sign", "-m", message); err != nil { + return "", false, fmt.Errorf("task worktree: commit changes in %q: %w", path, err) + } + head, err := revParseHEAD(ctx, l.git, path) + if err != nil { + return "", false, err + } + return head, true, nil +} + +func (l *Lifecycle) managedResourcePresent(ctx context.Context, path string) (bool, error) { + worktrees, err := l.git.worktrees(ctx, l.start.WorkspaceRoot) + if err != nil { + return false, err + } + if _, registered := worktrees[path]; registered { + return true, nil + } + _, err = os.Stat(path) + if err == nil { + return true, nil + } + if errors.Is(err, os.ErrNotExist) { + return false, nil + } + return false, fmt.Errorf("task worktree: stat managed path %q: %w", path, err) +} + +func (l *Lifecycle) validateOwnedResources(ctx context.Context) error { + worktrees, err := l.git.worktrees(ctx, l.start.WorkspaceRoot) + if err != nil { + return err + } + resources := []struct { + path string + branch string + }{{l.layout.IntegrationWorktree, l.layout.IntegrationBranch}} + for _, task := range l.layout.Tasks { + resources = append(resources, struct { + path string + branch string + }{task.WorktreePath, task.Branch}) + } + for _, resource := range resources { + if err := validateOwnedResource( + ctx, + l.git, + l.start.WorkspaceRoot, + worktrees, + resource.path, + resource.branch, + ); err != nil { + return err + } + } + return nil +} + +func validateOwnedResource( + ctx context.Context, + runner gitRunner, + repoRoot string, + worktrees map[string]string, + path string, + branch string, +) error { + state, err := inspectOwnedResource(ctx, runner, repoRoot, worktrees, path, branch) + if err != nil { + return err + } + return validateOwnedResourceState(state) +} + +type ownedResourceState struct { + path string + branch string + registeredBranch string + pathExists bool + registered bool + branchExists bool +} + +func inspectOwnedResource( + ctx context.Context, + runner gitRunner, + repoRoot string, + worktrees map[string]string, + path string, + branch string, +) (ownedResourceState, error) { + registeredBranch, registered := worktrees[path] + _, statErr := os.Stat(path) + pathExists := statErr == nil + if statErr != nil && !errors.Is(statErr, os.ErrNotExist) { + return ownedResourceState{}, fmt.Errorf("task worktree: stat managed path %q: %w", path, statErr) + } + branchExists, err := runner.branchExists(ctx, repoRoot, branch) + if err != nil { + return ownedResourceState{}, err + } + return ownedResourceState{ + path: path, + branch: branch, + registeredBranch: registeredBranch, + pathExists: pathExists, + registered: registered, + branchExists: branchExists, + }, nil +} + +func validateOwnedResourceState(state ownedResourceState) error { + switch { + case !state.pathExists && !state.registered && !state.branchExists: + return nil + case state.pathExists && state.registered && state.branchExists && state.registeredBranch == state.branch: + return nil + case state.pathExists && !state.registered: + return fmt.Errorf( + "task worktree: managed path %q exists but is not a registered Git worktree", + state.path, + ) + case state.registered && state.registeredBranch != state.branch: + return fmt.Errorf( + "task worktree: managed path %q uses branch %q, expected %q", + state.path, + state.registeredBranch, + state.branch, + ) + case state.branchExists && !state.registered: + return fmt.Errorf( + "task worktree: managed branch %q exists without its expected worktree", + state.branch, + ) + default: + return fmt.Errorf( + "task worktree: inconsistent managed resource %q on branch %q", + state.path, + state.branch, + ) + } +} + +func (l *Lifecycle) createOwnedWorktree( + ctx context.Context, + path string, + branch string, + base string, +) (CheckoutResult, error) { + worktrees, err := l.git.worktrees(ctx, l.start.WorkspaceRoot) + if err != nil { + return CheckoutResult{}, err + } + if err := validateOwnedResource(ctx, l.git, l.start.WorkspaceRoot, worktrees, path, branch); err != nil { + return CheckoutResult{}, err + } + if registeredBranch, exists := worktrees[path]; exists && registeredBranch == branch { + head, headErr := revParseHEAD(ctx, l.git, path) + if headErr != nil { + return CheckoutResult{}, headErr + } + return CheckoutResult{Branch: branch, WorktreePath: path, HEAD: head, Current: true}, nil + } + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return CheckoutResult{}, fmt.Errorf("task worktree: create parent for %q: %w", path, err) + } + if _, err := l.git.output(ctx, l.start.WorkspaceRoot, "worktree", "add", "-b", branch, path, base); err != nil { + return CheckoutResult{}, fmt.Errorf("task worktree: create %q: %w", path, err) + } + head, err := revParseHEAD(ctx, l.git, path) + if err != nil { + return CheckoutResult{}, err + } + return CheckoutResult{Branch: branch, WorktreePath: path, HEAD: head}, nil +} + +func (l *Lifecycle) requireOwnedCheckout(ctx context.Context, path, branch string) error { + worktrees, err := l.git.worktrees(ctx, l.start.WorkspaceRoot) + if err != nil { + return err + } + registeredBranch, ok := worktrees[path] + if !ok { + return fmt.Errorf("task worktree: expected managed worktree %q is not registered", path) + } + if registeredBranch != branch { + return fmt.Errorf( + "task worktree: managed worktree %q uses branch %q, expected %q", + path, + registeredBranch, + branch, + ) + } + actualBranch, err := l.git.output(ctx, path, "symbolic-ref", "--quiet", "--short", "HEAD") + if err != nil { + return fmt.Errorf("task worktree: resolve managed branch in %q: %w", path, err) + } + if strings.TrimSpace(actualBranch) != branch { + return fmt.Errorf( + "task worktree: managed checkout %q is on branch %q, expected %q", + path, + strings.TrimSpace(actualBranch), + branch, + ) + } + return nil +} + +func (l *Lifecycle) orderedTasks(taskIDs []string) ([]TaskLayout, error) { + seen := make(map[string]struct{}, len(taskIDs)) + result := make([]TaskLayout, 0, len(taskIDs)) + for _, taskID := range taskIDs { + task, err := l.task(taskID) + if err != nil { + return nil, err + } + if _, exists := seen[task.TaskID]; exists { + return nil, fmt.Errorf("task worktree: duplicate task ID %q", task.TaskID) + } + seen[task.TaskID] = struct{}{} + result = append(result, task) + } + sort.Slice(result, func(i, j int) bool { return result[i].TaskID < result[j].TaskID }) + return result, nil +} + +func (l *Lifecycle) task(taskID string) (TaskLayout, error) { + task, ok := l.tasks[strings.TrimSpace(taskID)] + if !ok { + return TaskLayout{}, fmt.Errorf("task worktree: unknown task ID %q", taskID) + } + return task, nil +} + +func (l *Lifecycle) taskBase(taskID string) (string, error) { + base := strings.TrimSpace(l.taskBases[strings.TrimSpace(taskID)]) + if base == "" { + return "", fmt.Errorf("task worktree: task %s has no recorded wave base", taskID) + } + return base, nil +} + +func (l *Lifecycle) requireTaskBase(ctx context.Context, task TaskLayout) error { + base, err := l.taskBase(task.TaskID) + if err != nil { + return err + } + head, err := revParseHEAD(ctx, l.git, task.WorktreePath) + if err != nil { + return err + } + valid, err := l.git.isAncestor(ctx, task.WorktreePath, base, head) + if err != nil { + return fmt.Errorf("task worktree: validate base ancestry for %s: %w", task.TaskID, err) + } + if !valid { + return fmt.Errorf( + "task worktree: task %s HEAD %s does not descend from recorded wave base %s", + task.TaskID, + head, + base, + ) + } + return nil +} + +func (l *Lifecycle) changedTaskPaths( + ctx context.Context, + task TaskLayout, + base string, +) ([]string, error) { + tracked, err := l.git.output( + ctx, + task.WorktreePath, + "diff", + "--no-renames", + "--name-only", + "-z", + base, + "--", + ) + if err != nil { + return nil, fmt.Errorf("task worktree: inspect changed paths for %s: %w", task.TaskID, err) + } + untracked, err := l.git.output( + ctx, + task.WorktreePath, + "ls-files", + "--others", + "--exclude-standard", + "-z", + ) + if err != nil { + return nil, fmt.Errorf("task worktree: inspect untracked paths for %s: %w", task.TaskID, err) + } + seen := make(map[string]struct{}) + for _, path := range append(splitNUL(tracked), splitNUL(untracked)...) { + seen[path] = struct{}{} + } + paths := make([]string, 0, len(seen)) + for path := range seen { + paths = append(paths, path) + } + sort.Strings(paths) + return paths, nil +} + +func (l *Lifecycle) conflictedPaths(ctx context.Context) ([]string, error) { + output, err := l.git.output( + ctx, + l.layout.IntegrationWorktree, + "diff", + "--name-only", + "--diff-filter=U", + "-z", + ) + if err != nil { + return nil, fmt.Errorf("task worktree: list merge conflicts: %w", err) + } + paths := splitNUL(output) + sort.Strings(paths) + return paths, nil +} + +func (l *Lifecycle) abortMergeIfActive(ctx context.Context) error { + active, err := l.git.refExists(ctx, l.layout.IntegrationWorktree, "MERGE_HEAD") + if err != nil { + return fmt.Errorf("task worktree: inspect failed merge: %w", err) + } + if !active { + return nil + } + if _, err := l.git.output(ctx, l.layout.IntegrationWorktree, "merge", "--abort"); err != nil { + return fmt.Errorf("task worktree: abort failed merge: %w", err) + } + return nil +} + +func (l *Lifecycle) removeCleanWorktree( + ctx context.Context, + path string, + branch string, + branchDeleteDir string, +) (bool, error) { + worktrees, err := l.git.worktrees(ctx, l.start.WorkspaceRoot) + if err != nil { + return false, err + } + registeredBranch, exists := worktrees[path] + if !exists { + if _, statErr := os.Stat(path); statErr == nil { + return false, fmt.Errorf("task worktree: retain unregistered path %q", path) + } else if !errors.Is(statErr, os.ErrNotExist) { + return false, fmt.Errorf("task worktree: stat %q: %w", path, statErr) + } + return false, nil + } + if registeredBranch != branch { + return false, fmt.Errorf("task worktree: retain %q on unexpected branch %q", path, registeredBranch) + } + dirty, err := isDirty(ctx, l.git, path) + if err != nil { + return false, err + } + if dirty { + return false, fmt.Errorf("task worktree: retain dirty worktree %q", path) + } + if _, err := l.git.output(ctx, l.start.WorkspaceRoot, "worktree", "remove", path); err != nil { + return false, fmt.Errorf("task worktree: remove %q: %w", path, err) + } + if _, err := l.git.output(ctx, branchDeleteDir, "branch", "-d", branch); err != nil { + return true, fmt.Errorf("task worktree: delete merged branch %q: %w", branch, err) + } + return true, nil +} + +func resolveWorktreesRoot(configured string) (string, error) { + if strings.TrimSpace(configured) != "" { + root, err := filepath.Abs(strings.TrimSpace(configured)) + if err != nil { + return "", fmt.Errorf("task worktree: resolve worktrees root: %w", err) + } + return canonicalProspectivePath(root) + } + homePaths, err := productizeconfig.ResolveHomePaths() + if err != nil { + return "", fmt.Errorf("task worktree: resolve Productize home: %w", err) + } + return canonicalProspectivePath(filepath.Join(homePaths.HomeDir, "worktrees")) +} + +// canonicalProspectivePath resolves symlinks in the deepest existing parent +// while preserving the not-yet-created suffix. Git reports registered +// worktrees using canonical paths, so plans must use the same representation. +func canonicalProspectivePath(path string) (string, error) { + abs, err := filepath.Abs(path) + if err != nil { + return "", fmt.Errorf("task worktree: resolve prospective path %q: %w", path, err) + } + candidate := filepath.Clean(abs) + suffix := make([]string, 0) + for { + resolved, resolveErr := filepath.EvalSymlinks(candidate) + if resolveErr == nil { + parts := append([]string{resolved}, suffix...) + return filepath.Join(parts...), nil + } + if !errors.Is(resolveErr, os.ErrNotExist) { + return "", fmt.Errorf("task worktree: resolve prospective path %q: %w", path, resolveErr) + } + parent := filepath.Dir(candidate) + if parent == candidate { + return "", fmt.Errorf("task worktree: no existing parent for %q", path) + } + suffix = append([]string{filepath.Base(candidate)}, suffix...) + candidate = parent + } +} + +func canonicalDirectory(path string) (string, error) { + trimmed := strings.TrimSpace(path) + if trimmed == "" { + return "", errors.New("directory is required") + } + abs, err := filepath.Abs(trimmed) + if err != nil { + return "", err + } + resolved, err := filepath.EvalSymlinks(abs) + if err != nil { + return "", err + } + info, err := os.Stat(resolved) + if err != nil { + return "", err + } + if !info.IsDir() { + return "", fmt.Errorf("%q is not a directory", resolved) + } + return filepath.Clean(resolved), nil +} + +func pathWithin(path, parent string) bool { + relative, err := filepath.Rel(parent, path) + if err != nil { + return false + } + return relative == "." || (relative != ".." && !strings.HasPrefix( + relative, + ".."+string(filepath.Separator), + )) +} + +func normalizeTaskIDs(taskIDs []string) ([]string, error) { + if len(taskIDs) == 0 { + return nil, errors.New("task worktree: at least one task ID is required") + } + seen := make(map[string]struct{}, len(taskIDs)) + result := make([]string, 0, len(taskIDs)) + for _, raw := range taskIDs { + taskID := strings.TrimSpace(raw) + if taskID == "" { + return nil, errors.New("task worktree: task ID is required") + } + if _, exists := seen[taskID]; exists { + return nil, fmt.Errorf("task worktree: duplicate task ID %q", taskID) + } + seen[taskID] = struct{}{} + result = append(result, taskID) + } + sort.Strings(result) + return result, nil +} + +func safeComponent(value string) string { + value = strings.TrimSpace(value) + var builder strings.Builder + lastDash := false + for _, r := range value { + switch { + case unicode.IsLetter(r), unicode.IsDigit(r), r == '_': + builder.WriteRune(unicode.ToLower(r)) + lastDash = false + case !lastDash: + builder.WriteByte('-') + lastDash = true + } + } + result := strings.Trim(builder.String(), "-_.") + if result == "" { + return "run" + } + return result +} + +func shortHash(value string) string { + digest := sha256.Sum256([]byte(value)) + return hex.EncodeToString(digest[:6]) +} + +func revParseHEAD(ctx context.Context, runner gitRunner, dir string) (string, error) { + head, err := runner.output(ctx, dir, "rev-parse", "--verify", "HEAD") + if err != nil { + return "", fmt.Errorf("task worktree: resolve HEAD in %q: %w", dir, err) + } + return strings.TrimSpace(head), nil +} + +func requireClean(ctx context.Context, runner gitRunner, dir string) error { + dirty, err := isDirty(ctx, runner, dir) + if err != nil { + return err + } + if dirty { + return fmt.Errorf("%w: %s", ErrDirtyWorkspace, dir) + } + return nil +} + +func isDirty(ctx context.Context, runner gitRunner, dir string) (bool, error) { + status, err := runner.output(ctx, dir, "status", "--porcelain=v1", "-z", "--untracked-files=all") + if err != nil { + return false, fmt.Errorf("task worktree: inspect status in %q: %w", dir, err) + } + return status != "", nil +} + +func splitNUL(value string) []string { + raw := strings.Split(value, "\x00") + result := make([]string, 0, len(raw)) + for _, item := range raw { + if item != "" { + result = append(result, filepath.ToSlash(item)) + } + } + return result +} diff --git a/internal/core/run/internal/taskworktree/lifecycle_test.go b/internal/core/run/internal/taskworktree/lifecycle_test.go new file mode 100644 index 00000000..20307780 --- /dev/null +++ b/internal/core/run/internal/taskworktree/lifecycle_test.go @@ -0,0 +1,641 @@ +package taskworktree + +import ( + "errors" + "os" + "os/exec" + "path/filepath" + "reflect" + "sort" + "strings" + "testing" +) + +func TestPreflightBuildsDeterministicPlanWithoutWrites(t *testing.T) { + repo := initRepository(t) + worktreesRoot := filepath.Join(t.TempDir(), "owned-worktrees") + + lifecycle, err := Preflight(t.Context(), Options{ + WorkspaceRoot: repo, + WorktreesRoot: worktreesRoot, + RunID: "Tasks Demo/Run", + TaskIDs: []string{"task_02", "task_01"}, + }) + if err != nil { + t.Fatalf("Preflight() error = %v", err) + } + + start := lifecycle.StartState() + canonicalRepo, err := canonicalDirectory(repo) + if err != nil { + t.Fatalf("canonicalDirectory(repo) error = %v", err) + } + if start.WorkspaceRoot != canonicalRepo || start.Branch != "main" || start.HEAD == "" { + t.Fatalf("StartState() = %#v", start) + } + layout := lifecycle.Layout() + canonicalWorktreesRoot, err := canonicalProspectivePath(worktreesRoot) + if err != nil { + t.Fatalf("canonicalProspectivePath(worktreesRoot) error = %v", err) + } + if !strings.HasPrefix(layout.RunRoot, canonicalWorktreesRoot+string(filepath.Separator)) { + t.Fatalf("RunRoot = %q, want beneath %q", layout.RunRoot, canonicalWorktreesRoot) + } + if got, want := layout.IntegrationBranch, "productize/tasks-demo-run-ba0cb60501ea/integration"; got != want { + t.Fatalf("IntegrationBranch = %q, want %q", got, want) + } + if got, want := taskIDs(layout.Tasks), []string{"task_01", "task_02"}; !reflect.DeepEqual(got, want) { + t.Fatalf("task order = %v, want %v", got, want) + } + if _, err := os.Stat(worktreesRoot); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("Preflight wrote worktree root: stat error = %v", err) + } + + second, err := Preflight(t.Context(), Options{ + WorkspaceRoot: repo, + WorktreesRoot: worktreesRoot, + RunID: "Tasks Demo/Run", + TaskIDs: []string{"task_01", "task_02"}, + }) + if err != nil { + t.Fatalf("second Preflight() error = %v", err) + } + if !reflect.DeepEqual(layout, second.Layout()) { + t.Fatalf("plans differ:\nfirst: %#v\nsecond: %#v", layout, second.Layout()) + } +} + +func TestPreflightRejectsUnsafeOriginalCheckout(t *testing.T) { + tests := []struct { + name string + prepare func(*testing.T, string) string + wantError error + }{ + { + name: "dirty", + prepare: func(t *testing.T, repo string) string { + t.Helper() + writeFile(t, filepath.Join(repo, "untracked.txt"), "dirty\n") + return repo + }, + wantError: ErrDirtyWorkspace, + }, + { + name: "detached head", + prepare: func(t *testing.T, repo string) string { + t.Helper() + runGit(t, repo, "switch", "--detach", "HEAD") + return repo + }, + wantError: ErrDetachedHEAD, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + repo := tc.prepare(t, initRepository(t)) + _, err := Preflight(t.Context(), Options{ + WorkspaceRoot: repo, + WorktreesRoot: filepath.Join(t.TempDir(), "worktrees"), + RunID: "run-1", + TaskIDs: []string{"task_01"}, + }) + if !errors.Is(err, tc.wantError) { + t.Fatalf("Preflight() error = %v, want errors.Is(%v)", err, tc.wantError) + } + }) + } +} + +func TestPreflightRejectsNestedDirectoryAndManagedPathCollision(t *testing.T) { + repo := initRepository(t) + nested := filepath.Join(repo, "nested") + if err := os.MkdirAll(nested, 0o755); err != nil { + t.Fatalf("MkdirAll(nested): %v", err) + } + baseOptions := Options{ + WorkspaceRoot: nested, + WorktreesRoot: filepath.Join(t.TempDir(), "worktrees"), + RunID: "run-1", + TaskIDs: []string{"task_01"}, + } + if _, err := Preflight(t.Context(), baseOptions); err == nil || !strings.Contains( + err.Error(), + "is not Git top level", + ) { + t.Fatalf("Preflight(nested) error = %v", err) + } + + baseOptions.WorkspaceRoot = repo + lifecycle, err := Preflight(t.Context(), baseOptions) + if err != nil { + t.Fatalf("Preflight() error = %v", err) + } + if err := os.MkdirAll(lifecycle.Layout().IntegrationWorktree, 0o755); err != nil { + t.Fatalf("MkdirAll(collision): %v", err) + } + if _, err := Preflight(t.Context(), baseOptions); err == nil || !strings.Contains( + err.Error(), + "not a registered Git worktree", + ) { + t.Fatalf("Preflight(collision) error = %v", err) + } +} + +func TestPreflightRejectsWorktreesRootInsideRepository(t *testing.T) { + repo := initRepository(t) + _, err := Preflight(t.Context(), Options{ + WorkspaceRoot: repo, + WorktreesRoot: filepath.Join(repo, ".productize", "worktrees"), + RunID: "run-1", + TaskIDs: []string{"task_01"}, + }) + if err == nil || !strings.Contains(err.Error(), "must be outside workspace") { + t.Fatalf("Preflight() error = %v", err) + } +} + +func TestLifecycleCreatesCommitsMergesFinalizesAndCleansUp(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_02", "task_01"}) + + integration, err := lifecycle.CreateIntegration(t.Context()) + if err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + if integration.HEAD != lifecycle.StartState().HEAD || integration.Current { + t.Fatalf("CreateIntegration() = %#v", integration) + } + resumed, err := lifecycle.CreateIntegration(t.Context()) + if err != nil { + t.Fatalf("CreateIntegration(resume) error = %v", err) + } + if !resumed.Current { + t.Fatalf("CreateIntegration(resume) Current = false") + } + + for _, taskID := range []string{"task_01", "task_02"} { + checkout, createErr := lifecycle.CreateTask(t.Context(), taskID) + if createErr != nil { + t.Fatalf("CreateTask(%s) error = %v", taskID, createErr) + } + writeFile(t, filepath.Join(checkout.WorktreePath, taskID+".txt"), taskID+"\n") + commit, commitErr := lifecycle.CommitTask(t.Context(), taskID, "feat: implement "+taskID) + if commitErr != nil { + t.Fatalf("CommitTask(%s) error = %v", taskID, commitErr) + } + if !commit.Changed || commit.Commit == lifecycle.StartState().HEAD { + t.Fatalf("CommitTask(%s) = %#v", taskID, commit) + } + } + + merged, err := lifecycle.MergeTasks(t.Context(), []string{"task_02", "task_01"}) + if err != nil { + t.Fatalf("MergeTasks() error = %v", err) + } + if got, want := mergeTaskIDs(merged), []string{"task_01", "task_02"}; !reflect.DeepEqual(got, want) { + t.Fatalf("merge order = %v, want %v", got, want) + } + for _, taskID := range []string{"task_01", "task_02"} { + assertFileContent(t, filepath.Join(lifecycle.Layout().IntegrationWorktree, taskID+".txt"), taskID+"\n") + } + + finalized, err := lifecycle.Finalize(t.Context()) + if err != nil { + t.Fatalf("Finalize() error = %v", err) + } + if finalized.PreviousHEAD != lifecycle.StartState().HEAD || finalized.FinalHEAD == finalized.PreviousHEAD { + t.Fatalf("Finalize() = %#v", finalized) + } + for _, taskID := range []string{"task_01", "task_02"} { + assertFileContent(t, filepath.Join(repo, taskID+".txt"), taskID+"\n") + } + current, err := lifecycle.Finalize(t.Context()) + if err != nil { + t.Fatalf("Finalize(current) error = %v", err) + } + if !current.Current || current.FinalHEAD != finalized.FinalHEAD { + t.Fatalf("Finalize(current) = %#v", current) + } + + cleanup, err := lifecycle.Cleanup(t.Context(), map[string]TaskOutcome{ + "task_01": TaskSucceeded, + "task_02": TaskSucceeded, + }) + if err != nil { + t.Fatalf("Cleanup() error = %v", err) + } + if len(cleanup.Retained) != 0 || len(cleanup.Removed) != 3 { + t.Fatalf("Cleanup() = %#v", cleanup) + } + for _, removed := range cleanup.Removed { + if _, err := os.Stat(removed); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("removed path %q still exists: %v", removed, err) + } + } +} + +func TestLifecycleEnforcesTaskLocalMemoryChanges(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01"}) + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + checkout, err := lifecycle.CreateTask(t.Context(), "task_01") + if err != nil { + t.Fatalf("CreateTask() error = %v", err) + } + memoryRoot := filepath.Join(".productize", "tasks", "demo", "memory") + absMemoryRoot := filepath.Join(checkout.WorktreePath, memoryRoot) + if err := os.MkdirAll(absMemoryRoot, 0o755); err != nil { + t.Fatalf("MkdirAll(memory): %v", err) + } + writeFile(t, filepath.Join(absMemoryRoot, "task_01.md"), "task-local\n") + if err := lifecycle.ValidateTaskMemoryIsolation(t.Context(), "task_01", memoryRoot); err != nil { + t.Fatalf("ValidateTaskMemoryIsolation(task local) error = %v", err) + } + + writeFile(t, filepath.Join(absMemoryRoot, "MEMORY.md"), "shared mutation\n") + err = lifecycle.ValidateTaskMemoryIsolation(t.Context(), "task_01", memoryRoot) + var isolationErr *TaskMemoryIsolationError + if !errors.As(err, &isolationErr) { + t.Fatalf("isolation error = %v, want *TaskMemoryIsolationError", err) + } + want := []string{".productize/tasks/demo/memory/MEMORY.md"} + if got := isolationErr.Paths; !reflect.DeepEqual(got, want) { + t.Fatalf("isolation paths = %v, want %v", got, want) + } +} + +func TestLifecycleRejectsRenamingSharedMemoryToTaskLocalMemory(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01"}) + integration, err := lifecycle.CreateIntegration(t.Context()) + if err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + memoryRoot := filepath.Join(".productize", "tasks", "demo", "memory") + sharedMemory := filepath.Join(integration.WorktreePath, memoryRoot, "MEMORY.md") + if err := os.MkdirAll(filepath.Dir(sharedMemory), 0o755); err != nil { + t.Fatalf("create shared memory directory: %v", err) + } + writeFile(t, sharedMemory, "shared\n") + if _, err := lifecycle.CommitIntegration(t.Context(), "initialize memory"); err != nil { + t.Fatalf("CommitIntegration() error = %v", err) + } + task, err := lifecycle.CreateTask(t.Context(), "task_01") + if err != nil { + t.Fatalf("CreateTask() error = %v", err) + } + taskMemory := filepath.Join(task.WorktreePath, memoryRoot, "task_01.md") + if err := os.Rename( + filepath.Join(task.WorktreePath, memoryRoot, "MEMORY.md"), + taskMemory, + ); err != nil { + t.Fatalf("rename shared memory: %v", err) + } + + err = lifecycle.ValidateTaskMemoryIsolation(t.Context(), "task_01", memoryRoot) + var isolationErr *TaskMemoryIsolationError + if !errors.As(err, &isolationErr) { + t.Fatalf("isolation error = %v, want *TaskMemoryIsolationError", err) + } + want := []string{".productize/tasks/demo/memory/MEMORY.md"} + if got := isolationErr.Paths; !reflect.DeepEqual(got, want) { + t.Fatalf("isolation paths = %v, want %v", got, want) + } +} + +func TestCommitTaskAcceptsAnAgentCreatedCommit(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01"}) + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + task, err := lifecycle.CreateTask(t.Context(), "task_01") + if err != nil { + t.Fatalf("CreateTask() error = %v", err) + } + writeFile(t, filepath.Join(task.WorktreePath, "agent.txt"), "committed by agent\n") + runGit(t, task.WorktreePath, "add", "--all") + runGit(t, task.WorktreePath, "commit", "--no-gpg-sign", "-m", "agent commit") + agentHEAD := strings.TrimSpace(runGit(t, task.WorktreePath, "rev-parse", "HEAD")) + + result, err := lifecycle.CommitTask(t.Context(), "task_01", "coordinator commit") + if err != nil { + t.Fatalf("CommitTask() error = %v", err) + } + if result.Changed || result.Commit != agentHEAD { + t.Fatalf("CommitTask() = %#v, want existing commit %s", result, agentHEAD) + } +} + +func TestCommitIntegrationCommitsCoordinatorChangesAndAcceptsCleanState(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01"}) + integration, err := lifecycle.CreateIntegration(t.Context()) + if err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + writeFile(t, filepath.Join(integration.WorktreePath, "shared-memory.md"), "wave one\n") + if err := lifecycle.RequireIntegrationClean(t.Context()); !errors.Is(err, ErrDirtyWorkspace) { + t.Fatalf("RequireIntegrationClean(dirty) error = %v, want ErrDirtyWorkspace", err) + } + + committed, err := lifecycle.CommitIntegration(t.Context(), "productize: rebuild shared memory") + if err != nil { + t.Fatalf("CommitIntegration() error = %v", err) + } + if !committed.Changed || committed.Commit == integration.HEAD { + t.Fatalf("CommitIntegration() = %#v", committed) + } + if err := lifecycle.RequireIntegrationClean(t.Context()); err != nil { + t.Fatalf("RequireIntegrationClean(clean) error = %v", err) + } + current, err := lifecycle.CommitIntegration(t.Context(), "unused message") + if err != nil { + t.Fatalf("CommitIntegration(clean) error = %v", err) + } + if current.Changed || current.Commit != committed.Commit { + t.Fatalf("CommitIntegration(clean) = %#v, want current commit %s", current, committed.Commit) + } + task, err := lifecycle.CreateTask(t.Context(), "task_01") + if err != nil { + t.Fatalf("CreateTask() error = %v", err) + } + if task.HEAD != committed.Commit { + t.Fatalf("task base HEAD = %s, want integration HEAD %s", task.HEAD, committed.Commit) + } +} + +func TestMergeConflictIsReportedAndAbortedCleanly(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01", "task_02"}) + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + for _, taskID := range []string{"task_01", "task_02"} { + checkout, err := lifecycle.CreateTask(t.Context(), taskID) + if err != nil { + t.Fatalf("CreateTask(%s) error = %v", taskID, err) + } + writeFile(t, filepath.Join(checkout.WorktreePath, "README.md"), taskID+"\n") + if _, err := lifecycle.CommitTask(t.Context(), taskID, "feat: "+taskID); err != nil { + t.Fatalf("CommitTask(%s) error = %v", taskID, err) + } + } + + merged, err := lifecycle.MergeTasks(t.Context(), []string{"task_02", "task_01"}) + if !errors.Is(err, ErrMergeConflict) { + t.Fatalf("MergeTasks() error = %v, want ErrMergeConflict", err) + } + if got, want := mergeTaskIDs(merged), []string{"task_01"}; !reflect.DeepEqual(got, want) { + t.Fatalf("merged tasks = %v, want %v", got, want) + } + var conflict *MergeConflictError + if !errors.As(err, &conflict) { + t.Fatalf("MergeTasks() error type = %T, want *MergeConflictError", err) + } + if got, want := conflict.Paths, []string{"README.md"}; !reflect.DeepEqual(got, want) { + t.Fatalf("conflict paths = %v, want %v", got, want) + } + if status := runGit(t, lifecycle.Layout().IntegrationWorktree, "status", "--porcelain=v1"); status != "" { + t.Fatalf("integration worktree remained dirty after abort: %q", status) + } + if _, statErr := os.Stat(lifecycle.Layout().Tasks[1].WorktreePath); statErr != nil { + t.Fatalf("conflicted task worktree was not retained: %v", statErr) + } +} + +func TestFinalizeRefusesChangedOriginalCheckout(t *testing.T) { + tests := []struct { + name string + change func(*testing.T, string) + }{ + { + name: "dirty", + change: func(t *testing.T, repo string) { + t.Helper() + writeFile(t, filepath.Join(repo, "user.txt"), "user change\n") + }, + }, + { + name: "new commit", + change: func(t *testing.T, repo string) { + t.Helper() + writeFile(t, filepath.Join(repo, "user.txt"), "user commit\n") + runGit(t, repo, "add", "--all") + runGit(t, repo, "commit", "--no-gpg-sign", "-m", "user commit") + }, + }, + { + name: "different branch", + change: func(t *testing.T, repo string) { + t.Helper() + runGit(t, repo, "switch", "-c", "user-branch") + }, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + repo := initRepository(t) + lifecycle := lifecycleWithIntegratedTask(t, repo) + integrationHEAD := strings.TrimSpace(runGit(t, lifecycle.Layout().IntegrationWorktree, "rev-parse", "HEAD")) + tc.change(t, repo) + + _, err := lifecycle.Finalize(t.Context()) + if !errors.Is(err, ErrOriginalChanged) { + t.Fatalf("Finalize() error = %v, want ErrOriginalChanged", err) + } + got := strings.TrimSpace(runGit( + t, + lifecycle.Layout().IntegrationWorktree, + "rev-parse", + "HEAD", + )) + if got != integrationHEAD { + t.Fatalf("integration HEAD changed: got %s, want %s", got, integrationHEAD) + } + }) + } +} + +func TestFinalizeRefusesDirtyIntegrationCheckout(t *testing.T) { + repo := initRepository(t) + lifecycle := lifecycleWithIntegratedTask(t, repo) + writeFile(t, filepath.Join(lifecycle.Layout().IntegrationWorktree, "verification-output.txt"), "dirty\n") + + _, err := lifecycle.Finalize(t.Context()) + if !errors.Is(err, ErrDirtyWorkspace) { + t.Fatalf("Finalize() error = %v, want ErrDirtyWorkspace", err) + } + if got := strings.TrimSpace(runGit(t, repo, "rev-parse", "HEAD")); got != lifecycle.StartState().HEAD { + t.Fatalf("original HEAD = %s, want unchanged %s", got, lifecycle.StartState().HEAD) + } +} + +func TestCleanupRetainsFailedConflictedAndDirtyWorktrees(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01", "task_02", "task_03"}) + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + for _, taskID := range []string{"task_01", "task_02", "task_03"} { + if _, err := lifecycle.CreateTask(t.Context(), taskID); err != nil { + t.Fatalf("CreateTask(%s) error = %v", taskID, err) + } + } + dirtyTask := lifecycle.Layout().Tasks[2] + writeFile(t, filepath.Join(dirtyTask.WorktreePath, "diagnostic.txt"), "retain me\n") + + result, err := lifecycle.Cleanup(t.Context(), map[string]TaskOutcome{ + "task_01": TaskFailed, + "task_02": TaskConflicted, + "task_03": TaskSucceeded, + }) + if err == nil || !strings.Contains(err.Error(), "retain dirty worktree") { + t.Fatalf("Cleanup() error = %v, want dirty retention error", err) + } + if len(result.Removed) != 0 || len(result.Retained) != 4 { + t.Fatalf("Cleanup() = %#v", result) + } + for _, retained := range result.Retained { + if _, statErr := os.Stat(retained); statErr != nil { + t.Fatalf("retained path %q unavailable: %v", retained, statErr) + } + } +} + +func TestCleanupDoesNotReportUncreatedPlannedWorktreesAsRetained(t *testing.T) { + repo := initRepository(t) + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01", "task_02"}) + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + created, err := lifecycle.CreateTask(t.Context(), "task_01") + if err != nil { + t.Fatalf("CreateTask(task_01) error = %v", err) + } + + result, err := lifecycle.Cleanup(t.Context(), map[string]TaskOutcome{ + "task_01": TaskFailed, + }) + if err != nil { + t.Fatalf("Cleanup() error = %v", err) + } + want := []string{created.WorktreePath, lifecycle.Layout().IntegrationWorktree} + for index := range want { + want[index], err = canonicalProspectivePath(want[index]) + if err != nil { + t.Fatalf("canonicalProspectivePath() error = %v", err) + } + } + sort.Strings(want) + if !reflect.DeepEqual(result.Retained, want) { + t.Fatalf("retained = %v, want %v", result.Retained, want) + } + if len(result.Removed) != 0 { + t.Fatalf("removed = %v, want none", result.Removed) + } +} + +func lifecycleWithIntegratedTask(t *testing.T, repo string) *Lifecycle { + t.Helper() + lifecycle := preflightTestLifecycle(t, repo, []string{"task_01"}) + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + task, err := lifecycle.CreateTask(t.Context(), "task_01") + if err != nil { + t.Fatalf("CreateTask() error = %v", err) + } + writeFile(t, filepath.Join(task.WorktreePath, "task.txt"), "task change\n") + if _, err := lifecycle.CommitTask(t.Context(), "task_01", "feat: task"); err != nil { + t.Fatalf("CommitTask() error = %v", err) + } + if _, err := lifecycle.MergeTasks(t.Context(), []string{"task_01"}); err != nil { + t.Fatalf("MergeTasks() error = %v", err) + } + return lifecycle +} + +func preflightTestLifecycle(t *testing.T, repo string, taskIDs []string) *Lifecycle { + t.Helper() + lifecycle, err := Preflight(t.Context(), Options{ + WorkspaceRoot: repo, + WorktreesRoot: filepath.Join(t.TempDir(), "worktrees"), + RunID: "run-1", + TaskIDs: taskIDs, + }) + if err != nil { + t.Fatalf("Preflight() error = %v", err) + } + return lifecycle +} + +func initRepository(t *testing.T) string { + t.Helper() + if _, err := exec.LookPath("git"); err != nil { + t.Skip("git binary not available") + } + repo := t.TempDir() + runGit(t, repo, "init", "-q", "-b", "main") + runGit(t, repo, "config", "user.email", "taskworktree@example.com") + runGit(t, repo, "config", "user.name", "Task Worktree Test") + runGit(t, repo, "config", "commit.gpgsign", "false") + writeFile(t, filepath.Join(repo, "README.md"), "initial\n") + runGit(t, repo, "add", "README.md") + runGit(t, repo, "commit", "--no-gpg-sign", "-m", "initial") + return repo +} + +func runGit(t *testing.T, dir string, args ...string) string { + t.Helper() + cmd := exec.CommandContext(t.Context(), "git", args...) + cmd.Dir = dir + cmd.Env = append(os.Environ(), + "GIT_AUTHOR_DATE=2026-01-01T00:00:00Z", + "GIT_COMMITTER_DATE=2026-01-01T00:00:00Z", + "GIT_TERMINAL_PROMPT=0", + ) + output, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("git %s: %v: %s", strings.Join(args, " "), err, output) + } + return string(output) +} + +func writeFile(t *testing.T, path, content string) { + t.Helper() + if err := os.WriteFile(path, []byte(content), 0o600); err != nil { + t.Fatalf("WriteFile(%q): %v", path, err) + } +} + +func assertFileContent(t *testing.T, path, want string) { + t.Helper() + content, err := os.ReadFile(path) + if err != nil { + t.Fatalf("ReadFile(%q): %v", path, err) + } + if string(content) != want { + t.Fatalf("content of %q = %q, want %q", path, content, want) + } +} + +func taskIDs(tasks []TaskLayout) []string { + result := make([]string, 0, len(tasks)) + for _, task := range tasks { + result = append(result, task.TaskID) + } + return result +} + +func mergeTaskIDs(tasks []MergeResult) []string { + result := make([]string, 0, len(tasks)) + for _, task := range tasks { + result = append(result, task.TaskID) + } + return result +} diff --git a/internal/core/run/task_worktrees.go b/internal/core/run/task_worktrees.go new file mode 100644 index 00000000..eca98257 --- /dev/null +++ b/internal/core/run/task_worktrees.go @@ -0,0 +1,52 @@ +package run + +import ( + "context" + + internal "github.com/itseffi/productize/internal/core/run/internal/taskworktree" +) + +// Task-worktree lifecycle aliases expose the focused implementation to the +// daemon without widening the nested internal package's import boundary. +type ( + Options = internal.Options + RepositoryState = internal.RepositoryState + TaskLayout = internal.TaskLayout + Layout = internal.Layout + Lifecycle = internal.Lifecycle + CheckoutResult = internal.CheckoutResult + CommitResult = internal.CommitResult + IntegrationCommitResult = internal.IntegrationCommitResult + MergeResult = internal.MergeResult + MergeConflictError = internal.MergeConflictError + TaskMemoryIsolationError = internal.TaskMemoryIsolationError + FinalizeResult = internal.FinalizeResult + TaskOutcome = internal.TaskOutcome + CleanupResult = internal.CleanupResult +) + +const ( + // TaskSucceeded allows cleanup of a clean, merged task worktree. + TaskSucceeded = internal.TaskSucceeded + // TaskFailed retains a task worktree for diagnosis. + TaskFailed = internal.TaskFailed + // TaskConflicted retains a task worktree for conflict resolution. + TaskConflicted = internal.TaskConflicted +) + +var ( + // ErrDirtyWorkspace indicates that a caller-owned checkout has local changes. + ErrDirtyWorkspace = internal.ErrDirtyWorkspace + // ErrDetachedHEAD indicates that the original checkout is not on a branch. + ErrDetachedHEAD = internal.ErrDetachedHEAD + // ErrOriginalChanged indicates that the original checkout changed after preflight. + ErrOriginalChanged = internal.ErrOriginalChanged + // ErrMergeConflict indicates a task merge conflict. + ErrMergeConflict = internal.ErrMergeConflict +) + +// Preflight validates the original checkout and creates a deterministic, +// write-free plan for Productize-owned task worktrees. +func Preflight(ctx context.Context, opts Options) (*Lifecycle, error) { + return internal.Preflight(ctx, opts) +} diff --git a/internal/core/subprocess/process.go b/internal/core/subprocess/process.go index 2daf5544..7a63d229 100644 --- a/internal/core/subprocess/process.go +++ b/internal/core/subprocess/process.go @@ -57,10 +57,7 @@ func Launch(ctx context.Context, cfg LaunchConfig) (*Process, error) { cmd.Env = append([]string(nil), cfg.Env...) cmd.Dir = strings.TrimSpace(cfg.WorkingDir) cmd.WaitDelay = cfg.WaitDelay - cmd.Cancel = func() error { - return forceTerminateProcess(cmd) - } - if err := configureCommand(cmd); err != nil { + if err := ConfigureCommand(cmd); err != nil { return nil, err } @@ -103,6 +100,18 @@ func Launch(ctx context.Context, cfg LaunchConfig) (*Process, error) { return process, nil } +// ConfigureCommand gives an exec command the same process-group cancellation +// semantics used by Launch. Call it before starting the command. +func ConfigureCommand(cmd *exec.Cmd) error { + if err := configureCommand(cmd); err != nil { + return err + } + cmd.Cancel = func() error { + return forceTerminateProcess(cmd) + } + return nil +} + // Stdin returns the subprocess stdin pipe. func (p *Process) Stdin() io.WriteCloser { if p == nil { diff --git a/internal/core/subprocess/process_unix_test.go b/internal/core/subprocess/process_unix_test.go index a40f0194..284af706 100644 --- a/internal/core/subprocess/process_unix_test.go +++ b/internal/core/subprocess/process_unix_test.go @@ -202,6 +202,9 @@ func TestProcessHelpers(t *testing.T) { if err := configureCommand(nil); err == nil { t.Fatal("expected nil command configuration error") } + if err := ConfigureCommand(nil); err == nil { + t.Fatal("expected nil public command configuration error") + } } func TestProcessCapturesStderrAndSignalsCompletion(t *testing.T) { diff --git a/internal/core/tasks/dependency_graph.go b/internal/core/tasks/dependency_graph.go new file mode 100644 index 00000000..d378fdbe --- /dev/null +++ b/internal/core/tasks/dependency_graph.go @@ -0,0 +1,424 @@ +package tasks + +import ( + "errors" + "fmt" + "path/filepath" + "slices" + "strings" + + "github.com/itseffi/productize/internal/core/model" +) + +// DependencyIssueCode identifies a stable task dependency validation failure. +type DependencyIssueCode string + +const ( + DependencyIssueInvalidTaskID DependencyIssueCode = "invalid_task_id" + DependencyIssueDuplicateTaskID DependencyIssueCode = "duplicate_task_id" + DependencyIssueMissingDependency DependencyIssueCode = "missing_dependency" + DependencyIssueSelfDependency DependencyIssueCode = "self_dependency" + DependencyIssueCycle DependencyIssueCode = "dependency_cycle" +) + +// DependencyNode is the scheduler-facing representation of one workflow task. +type DependencyNode struct { + ID string `json:"id"` + Status string `json:"status"` + Dependencies []string `json:"dependencies,omitempty"` +} + +// DependencyIssue describes one actionable dependency graph problem. +type DependencyIssue struct { + Code DependencyIssueCode `json:"code"` + TaskID string `json:"task_id,omitempty"` + Dependency string `json:"dependency,omitempty"` + Cycle []string `json:"cycle,omitempty"` +} + +// DependencyGraphError reports all dependency graph problems in stable order. +type DependencyGraphError struct { + Issues []DependencyIssue `json:"issues"` +} + +func (e *DependencyGraphError) Error() string { + if e == nil || len(e.Issues) == 0 { + return "task dependency graph is invalid" + } + + messages := make([]string, 0, len(e.Issues)) + for _, issue := range e.Issues { + messages = append(messages, formatDependencyIssue(issue)) + } + return "task dependency graph is invalid: " + strings.Join(messages, "; ") +} + +// DependencyGraph stores an immutable, validated workflow dependency graph. +type DependencyGraph struct { + nodes map[string]DependencyNode + order []string +} + +// BuildDependencyGraph normalizes and validates workflow dependency nodes. +func BuildDependencyGraph(input []DependencyNode) (*DependencyGraph, error) { + nodes, order, issues := normalizeDependencyNodes(input) + issues = append(issues, validateDependencyReferences(nodes, order)...) + issues = append(issues, findDependencyCycles(nodes, order)...) + sortDependencyIssues(issues) + if len(issues) > 0 { + return nil, &DependencyGraphError{Issues: issues} + } + + return &DependencyGraph{nodes: nodes, order: order}, nil +} + +// ReadDependencyGraph parses all workflow task files and builds their dependency graph. +func ReadDependencyGraph(tasksDir string) (*DependencyGraph, error) { + nodes := make([]DependencyNode, 0) + if err := walkTaskFiles(tasksDir, func(entry model.IssueEntry, task model.TaskEntry) error { + nodes = append(nodes, DependencyNode{ + ID: entry.CodeFile, + Status: task.Status, + Dependencies: task.Dependencies, + }) + return nil + }); err != nil { + return nil, fmt.Errorf("read task dependency graph: %w", err) + } + + graph, err := BuildDependencyGraph(nodes) + if err != nil { + return nil, fmt.Errorf("build task dependency graph: %w", err) + } + return graph, nil +} + +// Nodes returns every graph node in deterministic task order. +func (g *DependencyGraph) Nodes() []DependencyNode { + if g == nil { + return nil + } + + nodes := make([]DependencyNode, 0, len(g.order)) + for _, id := range g.order { + nodes = append(nodes, cloneDependencyNode(g.nodes[id])) + } + return nodes +} + +// Waves returns dependency-ready, deterministically ordered waves of unfinished tasks. +// Dependencies are re-evaluated after every wave, and each wave contains at most limit tasks. +func (g *DependencyGraph) Waves(limit int) ([][]DependencyNode, error) { + return g.WavesWithOptions(limit, false) +} + +// WavesWithOptions returns deterministic waves and optionally schedules tasks +// whose status is already completed. +func (g *DependencyGraph) WavesWithOptions( + limit int, + includeCompleted bool, +) ([][]DependencyNode, error) { + if g == nil { + return nil, errors.New("task dependency graph is required") + } + if limit < 1 { + return nil, fmt.Errorf("task concurrency must be greater than zero, got %d", limit) + } + + satisfied := make(map[string]struct{}, len(g.nodes)) + remaining := make(map[string]struct{}, len(g.nodes)) + for _, id := range g.order { + if !includeCompleted && IsTaskCompleted(model.TaskEntry{Status: g.nodes[id].Status}) { + satisfied[id] = struct{}{} + continue + } + remaining[id] = struct{}{} + } + + waves := make([][]DependencyNode, 0) + for len(remaining) > 0 { + ready := g.readyNodes(remaining, satisfied) + if len(ready) == 0 { + return nil, errors.New("task dependency graph has unfinished tasks with no satisfiable dependency path") + } + if len(ready) > limit { + ready = ready[:limit] + } + + wave := make([]DependencyNode, 0, len(ready)) + for _, id := range ready { + wave = append(wave, cloneDependencyNode(g.nodes[id])) + delete(remaining, id) + satisfied[id] = struct{}{} + } + waves = append(waves, wave) + } + return waves, nil +} + +func (g *DependencyGraph) readyNodes( + remaining map[string]struct{}, + satisfied map[string]struct{}, +) []string { + ready := make([]string, 0, len(remaining)) + for _, id := range g.order { + if _, ok := remaining[id]; !ok { + continue + } + if dependenciesSatisfied(g.nodes[id].Dependencies, satisfied) { + ready = append(ready, id) + } + } + return ready +} + +func dependenciesSatisfied(dependencies []string, satisfied map[string]struct{}) bool { + for _, dependency := range dependencies { + if _, ok := satisfied[dependency]; !ok { + return false + } + } + return true +} + +func normalizeDependencyNodes( + input []DependencyNode, +) (map[string]DependencyNode, []string, []DependencyIssue) { + nodes := make(map[string]DependencyNode, len(input)) + counts := make(map[string]int, len(input)) + issues := make([]DependencyIssue, 0) + for _, inputNode := range input { + id := normalizeDependencyID(inputNode.ID) + if id == "" { + issues = append(issues, DependencyIssue{Code: DependencyIssueInvalidTaskID}) + continue + } + + counts[id]++ + if counts[id] > 1 { + continue + } + nodes[id] = DependencyNode{ + ID: id, + Status: strings.TrimSpace(inputNode.Status), + Dependencies: normalizeGraphDependencies(inputNode.Dependencies), + } + } + + order := make([]string, 0, len(nodes)) + for id := range nodes { + order = append(order, id) + if counts[id] > 1 { + issues = append(issues, DependencyIssue{ + Code: DependencyIssueDuplicateTaskID, + TaskID: id, + }) + } + } + slices.SortFunc(order, compareDependencyIDs) + return nodes, order, issues +} + +func validateDependencyReferences( + nodes map[string]DependencyNode, + order []string, +) []DependencyIssue { + issues := make([]DependencyIssue, 0) + for _, id := range order { + for _, dependency := range nodes[id].Dependencies { + switch { + case dependency == id: + issues = append(issues, DependencyIssue{ + Code: DependencyIssueSelfDependency, + TaskID: id, + Dependency: dependency, + }) + case !dependencyNodeExists(nodes, dependency): + issues = append(issues, DependencyIssue{ + Code: DependencyIssueMissingDependency, + TaskID: id, + Dependency: dependency, + }) + } + } + } + return issues +} + +func dependencyNodeExists(nodes map[string]DependencyNode, id string) bool { + _, ok := nodes[id] + return ok +} + +func findDependencyCycles( + nodes map[string]DependencyNode, + order []string, +) []DependencyIssue { + state := dependencySCCState{ + nodes: nodes, + indices: make(map[string]int, len(nodes)), + lowLink: make(map[string]int, len(nodes)), + onStack: make(map[string]bool, len(nodes)), + index: 1, + } + for _, id := range order { + if state.indices[id] == 0 { + state.visit(id) + } + } + + issues := make([]DependencyIssue, 0, len(state.cycles)) + for _, cycle := range state.cycles { + issues = append(issues, DependencyIssue{ + Code: DependencyIssueCycle, + TaskID: cycle[0], + Cycle: cycle, + }) + } + return issues +} + +type dependencySCCState struct { + nodes map[string]DependencyNode + indices map[string]int + lowLink map[string]int + onStack map[string]bool + stack []string + cycles [][]string + index int +} + +func (s *dependencySCCState) visit(id string) { + s.indices[id] = s.index + s.lowLink[id] = s.index + s.index++ + s.stack = append(s.stack, id) + s.onStack[id] = true + + for _, dependency := range s.nodes[id].Dependencies { + if dependency == id || !dependencyNodeExists(s.nodes, dependency) { + continue + } + switch { + case s.indices[dependency] == 0: + s.visit(dependency) + s.lowLink[id] = min(s.lowLink[id], s.lowLink[dependency]) + case s.onStack[dependency]: + s.lowLink[id] = min(s.lowLink[id], s.indices[dependency]) + } + } + + if s.lowLink[id] != s.indices[id] { + return + } + + component := make([]string, 0) + for len(s.stack) > 0 { + last := len(s.stack) - 1 + member := s.stack[last] + s.stack = s.stack[:last] + s.onStack[member] = false + component = append(component, member) + if member == id { + break + } + } + if len(component) < 2 { + return + } + slices.SortFunc(component, compareDependencyIDs) + s.cycles = append(s.cycles, component) +} + +func normalizeGraphDependencies(dependencies []string) []string { + normalized := make([]string, 0, len(dependencies)) + seen := make(map[string]struct{}, len(dependencies)) + for _, raw := range dependencies { + dependency := normalizeDependencyID(raw) + if dependency == "" || strings.EqualFold(dependency, "none") { + continue + } + if _, ok := seen[dependency]; ok { + continue + } + seen[dependency] = struct{}{} + normalized = append(normalized, dependency) + } + if len(normalized) == 0 { + return nil + } + slices.SortFunc(normalized, compareDependencyIDs) + return normalized +} + +func normalizeDependencyID(raw string) string { + trimmed := strings.TrimSpace(raw) + return strings.TrimSuffix(trimmed, filepath.Ext(trimmed)) +} + +func compareDependencyIDs(a, b string) int { + numberA := ExtractTaskNumber(a + ".md") + numberB := ExtractTaskNumber(b + ".md") + if numberA > 0 && numberB > 0 && numberA != numberB { + return numberA - numberB + } + return strings.Compare(a, b) +} + +func sortDependencyIssues(issues []DependencyIssue) { + slices.SortStableFunc(issues, func(a, b DependencyIssue) int { + if byTask := compareDependencyIDs(a.TaskID, b.TaskID); byTask != 0 { + return byTask + } + if byCode := dependencyIssueRank(a.Code) - dependencyIssueRank(b.Code); byCode != 0 { + return byCode + } + return compareDependencyIDs(a.Dependency, b.Dependency) + }) +} + +func dependencyIssueRank(code DependencyIssueCode) int { + switch code { + case DependencyIssueInvalidTaskID: + return 0 + case DependencyIssueDuplicateTaskID: + return 1 + case DependencyIssueMissingDependency: + return 2 + case DependencyIssueSelfDependency: + return 3 + case DependencyIssueCycle: + return 4 + default: + return 5 + } +} + +func formatDependencyIssue(issue DependencyIssue) string { + switch issue.Code { + case DependencyIssueInvalidTaskID: + return "task ID cannot be empty" + case DependencyIssueDuplicateTaskID: + return fmt.Sprintf("duplicate task ID %q (task IDs must be unique)", issue.TaskID) + case DependencyIssueMissingDependency: + return fmt.Sprintf( + "task %q references missing dependency %q (add the task or remove the dependency)", + issue.TaskID, + issue.Dependency, + ) + case DependencyIssueSelfDependency: + return fmt.Sprintf("task %q depends on itself (remove the self-dependency)", issue.TaskID) + case DependencyIssueCycle: + return fmt.Sprintf( + "dependency cycle among tasks %s (remove at least one dependency edge)", + strings.Join(issue.Cycle, ", "), + ) + default: + return fmt.Sprintf("task %q has an invalid dependency", issue.TaskID) + } +} + +func cloneDependencyNode(node DependencyNode) DependencyNode { + node.Dependencies = slices.Clone(node.Dependencies) + return node +} diff --git a/internal/core/tasks/dependency_graph_test.go b/internal/core/tasks/dependency_graph_test.go new file mode 100644 index 00000000..18f3c67b --- /dev/null +++ b/internal/core/tasks/dependency_graph_test.go @@ -0,0 +1,325 @@ +package tasks + +import ( + "errors" + "os" + "path/filepath" + "reflect" + "strings" + "testing" +) + +func TestBuildDependencyGraphNormalizesNodesDeterministically(t *testing.T) { + t.Parallel() + + graph, err := BuildDependencyGraph([]DependencyNode{ + {ID: "task_10.md", Status: " pending ", Dependencies: []string{" task_02.md ", "task_02"}}, + {ID: "task_02", Status: "completed"}, + {ID: "task_01", Status: "pending"}, + }) + if err != nil { + t.Fatalf("build dependency graph: %v", err) + } + + want := []DependencyNode{ + {ID: "task_01", Status: "pending"}, + {ID: "task_02", Status: "completed"}, + {ID: "task_10", Status: "pending", Dependencies: []string{"task_02"}}, + } + if got := graph.Nodes(); !reflect.DeepEqual(got, want) { + t.Fatalf("Nodes() mismatch\nwant: %#v\ngot: %#v", want, got) + } + + got := graph.Nodes() + got[2].Dependencies[0] = "mutated" + if secondRead := graph.Nodes(); !reflect.DeepEqual(secondRead, want) { + t.Fatalf("Nodes() exposed mutable graph state: %#v", secondRead) + } +} + +func TestBuildDependencyGraphReportsStableValidationIssues(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + nodes []DependencyNode + wantIssues []DependencyIssue + wantText []string + }{ + { + name: "empty task ID", + nodes: []DependencyNode{{ID: " ", Status: "pending"}}, + wantIssues: []DependencyIssue{{Code: DependencyIssueInvalidTaskID}}, + wantText: []string{"task ID cannot be empty"}, + }, + { + name: "duplicate IDs after normalization", + nodes: []DependencyNode{ + {ID: "task_01", Status: "pending"}, + {ID: "task_01.md", Status: "completed"}, + }, + wantIssues: []DependencyIssue{{ + Code: DependencyIssueDuplicateTaskID, + TaskID: "task_01", + }}, + wantText: []string{`duplicate task ID "task_01"`, "task IDs must be unique"}, + }, + { + name: "missing and self dependencies", + nodes: []DependencyNode{ + {ID: "task_02", Status: "pending", Dependencies: []string{"task_99", "task_02"}}, + {ID: "task_01", Status: "pending"}, + }, + wantIssues: []DependencyIssue{ + { + Code: DependencyIssueMissingDependency, + TaskID: "task_02", + Dependency: "task_99", + }, + { + Code: DependencyIssueSelfDependency, + TaskID: "task_02", + Dependency: "task_02", + }, + }, + wantText: []string{ + `task "task_02" references missing dependency "task_99"`, + `task "task_02" depends on itself`, + }, + }, + { + name: "multiple cycles", + nodes: []DependencyNode{ + {ID: "task_04", Status: "pending", Dependencies: []string{"task_03"}}, + {ID: "task_02", Status: "pending", Dependencies: []string{"task_01"}}, + {ID: "task_03", Status: "pending", Dependencies: []string{"task_04"}}, + {ID: "task_01", Status: "pending", Dependencies: []string{"task_02"}}, + }, + wantIssues: []DependencyIssue{ + { + Code: DependencyIssueCycle, + TaskID: "task_01", + Cycle: []string{"task_01", "task_02"}, + }, + { + Code: DependencyIssueCycle, + TaskID: "task_03", + Cycle: []string{"task_03", "task_04"}, + }, + }, + wantText: []string{ + "dependency cycle among tasks task_01, task_02", + "dependency cycle among tasks task_03, task_04", + }, + }, + } + + for _, tt := range tests { + tt := tt + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + graph, err := BuildDependencyGraph(tt.nodes) + if err == nil { + t.Fatal("expected dependency graph validation error") + } + if graph != nil { + t.Fatalf("graph = %#v, want nil after validation failure", graph) + } + + var graphErr *DependencyGraphError + if !errors.As(err, &graphErr) { + t.Fatalf("error type = %T, want *DependencyGraphError: %v", err, err) + } + if !reflect.DeepEqual(graphErr.Issues, tt.wantIssues) { + t.Fatalf("issues mismatch\nwant: %#v\ngot: %#v", tt.wantIssues, graphErr.Issues) + } + for _, part := range tt.wantText { + if !strings.Contains(err.Error(), part) { + t.Fatalf("error %q does not contain %q", err, part) + } + } + }) + } +} + +func TestDependencyGraphWaves(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + nodes []DependencyNode + limit int + want [][]string + }{ + { + name: "orders independent and dependent tasks", + nodes: []DependencyNode{ + {ID: "task_04", Status: "pending", Dependencies: []string{"task_02", "task_03"}}, + {ID: "task_03", Status: "pending", Dependencies: []string{"task_01"}}, + {ID: "task_02", Status: "pending", Dependencies: []string{"task_01"}}, + {ID: "task_01", Status: "pending"}, + }, + limit: 4, + want: [][]string{{"task_01"}, {"task_02", "task_03"}, {"task_04"}}, + }, + { + name: "completed dependencies are already satisfied", + nodes: []DependencyNode{ + {ID: "task_03", Status: "pending", Dependencies: []string{"task_02"}}, + {ID: "task_02", Status: "pending", Dependencies: []string{"task_01"}}, + {ID: "task_01", Status: "completed"}, + }, + limit: 3, + want: [][]string{{"task_02"}, {"task_03"}}, + }, + { + name: "cap is applied and readiness is reevaluated after each wave", + nodes: []DependencyNode{ + {ID: "task_05", Status: "pending"}, + {ID: "task_04", Status: "pending"}, + {ID: "task_03", Status: "pending", Dependencies: []string{"task_02"}}, + {ID: "task_02", Status: "pending", Dependencies: []string{"task_01"}}, + {ID: "task_01", Status: "pending"}, + }, + limit: 2, + want: [][]string{{"task_01", "task_04"}, {"task_02", "task_05"}, {"task_03"}}, + }, + { + name: "cap one preserves deterministic ready order", + nodes: []DependencyNode{ + {ID: "task_04", Status: "pending"}, + {ID: "task_02", Status: "pending", Dependencies: []string{"task_01"}}, + {ID: "task_01", Status: "pending"}, + }, + limit: 1, + want: [][]string{{"task_01"}, {"task_02"}, {"task_04"}}, + }, + { + name: "all tasks completed", + nodes: []DependencyNode{ + {ID: "task_01", Status: "completed"}, + {ID: "task_02", Status: "completed", Dependencies: []string{"task_01"}}, + }, + limit: 2, + want: [][]string{}, + }, + } + + for _, tt := range tests { + tt := tt + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + graph, err := BuildDependencyGraph(tt.nodes) + if err != nil { + t.Fatalf("build dependency graph: %v", err) + } + waves, err := graph.Waves(tt.limit) + if err != nil { + t.Fatalf("plan waves: %v", err) + } + if got := waveIDs(waves); !reflect.DeepEqual(got, tt.want) { + t.Fatalf("wave IDs mismatch\nwant: %#v\ngot: %#v", tt.want, got) + } + }) + } +} + +func TestDependencyGraphWavesRejectsInvalidLimit(t *testing.T) { + t.Parallel() + + graph, err := BuildDependencyGraph([]DependencyNode{{ID: "task_01", Status: "pending"}}) + if err != nil { + t.Fatalf("build dependency graph: %v", err) + } + if _, err := graph.Waves(0); err == nil || !strings.Contains(err.Error(), "greater than zero") { + t.Fatalf("Waves(0) error = %v, want actionable concurrency error", err) + } +} + +func TestDependencyGraphWavesCanIncludeCompletedTasks(t *testing.T) { + t.Parallel() + + graph, err := BuildDependencyGraph([]DependencyNode{ + {ID: "task_02", Status: "completed", Dependencies: []string{"task_01"}}, + {ID: "task_01", Status: "completed"}, + }) + if err != nil { + t.Fatalf("build dependency graph: %v", err) + } + waves, err := graph.WavesWithOptions(2, true) + if err != nil { + t.Fatalf("plan waves including completed tasks: %v", err) + } + if got, want := waveIDs(waves), [][]string{{"task_01"}, {"task_02"}}; !reflect.DeepEqual(got, want) { + t.Fatalf("wave IDs mismatch\nwant: %#v\ngot: %#v", want, got) + } +} + +func TestReadDependencyGraphUsesTaskParserMetadata(t *testing.T) { + t.Parallel() + + tasksDir := t.TempDir() + writeDependencyTask(t, tasksDir, "task_02.md", "pending", []string{"task_01.md"}) + writeDependencyTask(t, tasksDir, "task_01.md", "completed", nil) + + graph, err := ReadDependencyGraph(tasksDir) + if err != nil { + t.Fatalf("read dependency graph: %v", err) + } + waves, err := graph.Waves(2) + if err != nil { + t.Fatalf("plan waves: %v", err) + } + if got, want := waveIDs(waves), [][]string{{"task_02"}}; !reflect.DeepEqual(got, want) { + t.Fatalf("wave IDs mismatch\nwant: %#v\ngot: %#v", want, got) + } +} + +func waveIDs(waves [][]DependencyNode) [][]string { + ids := make([][]string, 0, len(waves)) + for _, wave := range waves { + waveIDs := make([]string, 0, len(wave)) + for _, node := range wave { + waveIDs = append(waveIDs, node.ID) + } + ids = append(ids, waveIDs) + } + return ids +} + +func writeDependencyTask( + t *testing.T, + tasksDir string, + name string, + status string, + dependencies []string, +) { + t.Helper() + + dependencyLines := "dependencies: []" + if len(dependencies) > 0 { + lines := make([]string, 0, len(dependencies)+1) + lines = append(lines, "dependencies:") + for _, dependency := range dependencies { + lines = append(lines, " - "+dependency) + } + dependencyLines = strings.Join(lines, "\n") + } + content := strings.Join([]string{ + "---", + "status: " + status, + "title: Dependency fixture", + "type: backend", + "complexity: medium", + dependencyLines, + "---", + "", + "# Dependency fixture", + "", + }, "\n") + if err := os.WriteFile(filepath.Join(tasksDir, name), []byte(content), 0o600); err != nil { + t.Fatalf("write dependency task %s: %v", name, err) + } +} diff --git a/internal/core/workspace/config_merge.go b/internal/core/workspace/config_merge.go index 73ed1dec..8a8a0ea6 100644 --- a/internal/core/workspace/config_merge.go +++ b/internal/core/workspace/config_merge.go @@ -89,6 +89,7 @@ func buildEffectiveTaskRunConfig( workspace TaskRunConfig, ) TaskRunConfig { return TaskRunConfig{ + Concurrent: cloneOptionalValue(preferOverlay(global.Concurrent, workspace.Concurrent)), IncludeCompleted: cloneOptionalValue(preferOverlay(global.IncludeCompleted, workspace.IncludeCompleted)), OutputFormat: effectiveCommandOverride( globalDefaults.OutputFormat, @@ -97,6 +98,7 @@ func buildEffectiveTaskRunConfig( workspace.OutputFormat, ), TaskRuntimeRules: mergeTaskRunRuntimeRules(global.TaskRuntimeRules, workspace.TaskRuntimeRules), + VerifyCommand: cloneOptionalValue(preferOverlay(global.VerifyCommand, workspace.VerifyCommand)), } } diff --git a/internal/core/workspace/config_test.go b/internal/core/workspace/config_test.go index dddf09d0..adf79efa 100644 --- a/internal/core/workspace/config_test.go +++ b/internal/core/workspace/config_test.go @@ -3,6 +3,7 @@ package workspace import ( "context" "errors" + "fmt" "os" "path/filepath" "strings" @@ -263,8 +264,10 @@ max_retries = 0 retry_backoff_multiplier = 1.5 [tasks.run] +concurrent = 3 include_completed = false output_format = "json" +verify_command = "make verify" [fix_reviews] concurrent = 2 @@ -320,9 +323,15 @@ output_format = "json" if cfg.Tasks.Run.IncludeCompleted == nil || *cfg.Tasks.Run.IncludeCompleted { t.Fatalf("unexpected tasks.run.include_completed: %#v", cfg.Tasks.Run.IncludeCompleted) } + if cfg.Tasks.Run.Concurrent == nil || *cfg.Tasks.Run.Concurrent != 3 { + t.Fatalf("unexpected tasks.run.concurrent: %#v", cfg.Tasks.Run.Concurrent) + } if cfg.Tasks.Run.OutputFormat == nil || *cfg.Tasks.Run.OutputFormat != "json" { t.Fatalf("unexpected tasks.run.output_format: %#v", cfg.Tasks.Run.OutputFormat) } + if cfg.Tasks.Run.VerifyCommand == nil || *cfg.Tasks.Run.VerifyCommand != "make verify" { + t.Fatalf("unexpected tasks.run.verify_command: %#v", cfg.Tasks.Run.VerifyCommand) + } if cfg.FixReviews.Concurrent == nil || *cfg.FixReviews.Concurrent != 2 { t.Fatalf("unexpected fix_reviews.concurrent: %#v", cfg.FixReviews.Concurrent) } @@ -531,6 +540,56 @@ reasoning_effort = "xhigh" } } +func TestLoadConfigRejectsNonPositiveTaskRunConcurrency(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + concurrent int + }{ + {name: "zero", concurrent: 0}, + {name: "negative", concurrent: -1}, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeWorkspaceConfig(t, root, fmt.Sprintf(` +[tasks.run] +concurrent = %d +`, tt.concurrent)) + + _, _, err := LoadConfig(context.Background(), root) + if err == nil { + t.Fatal("expected invalid tasks.run.concurrent error") + } + if !strings.Contains(err.Error(), "tasks.run.concurrent must be greater than zero") { + t.Fatalf("unexpected error: %v", err) + } + }) + } +} + +func TestLoadConfigRejectsBlankTaskRunVerifyCommand(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeWorkspaceConfig(t, root, ` +[tasks.run] +verify_command = " " +`) + + _, _, err := LoadConfig(context.Background(), root) + if err == nil { + t.Fatal("expected invalid tasks.run.verify_command error") + } + if !strings.Contains(err.Error(), "tasks.run.verify_command cannot be blank") { + t.Fatalf("unexpected error: %v", err) + } +} + func TestLoadConfigMergesStartTaskRuntimeRulesByType(t *testing.T) { homeDir := isolateWorkspaceConfigHome(t) root := t.TempDir() @@ -1125,14 +1184,18 @@ model = "sonnet" access_mode = "default" [tasks.run] +concurrent = 2 include_completed = false +verify_command = "go test ./..." `) writeWorkspaceConfig(t, root, ` [defaults] model = "gpt-5.5" [tasks.run] +concurrent = 4 include_completed = true +verify_command = "make verify" `) cfg, path, err := LoadConfig(context.Background(), root) @@ -1151,6 +1214,12 @@ include_completed = true if cfg.Tasks.Run.IncludeCompleted == nil || !*cfg.Tasks.Run.IncludeCompleted { t.Fatalf("expected workspace tasks.run.include_completed override, got %#v", cfg.Tasks.Run.IncludeCompleted) } + if cfg.Tasks.Run.Concurrent == nil || *cfg.Tasks.Run.Concurrent != 4 { + t.Fatalf("expected workspace tasks.run.concurrent override, got %#v", cfg.Tasks.Run.Concurrent) + } + if cfg.Tasks.Run.VerifyCommand == nil || *cfg.Tasks.Run.VerifyCommand != "make verify" { + t.Fatalf("expected workspace tasks.run.verify_command override, got %#v", cfg.Tasks.Run.VerifyCommand) + } } func TestLoadConfigKeepsWorkspaceDefaultsAheadOfGlobalCommandOverrides(t *testing.T) { diff --git a/internal/core/workspace/config_types.go b/internal/core/workspace/config_types.go index 9a2f14d5..b2f72fb0 100644 --- a/internal/core/workspace/config_types.go +++ b/internal/core/workspace/config_types.go @@ -40,9 +40,11 @@ type RuntimeOverrides struct { type DefaultsConfig RuntimeOverrides type TaskRunConfig struct { + Concurrent *int `toml:"concurrent"` IncludeCompleted *bool `toml:"include_completed"` OutputFormat *string `toml:"output_format"` TaskRuntimeRules *[]model.TaskRuntimeRule `toml:"task_runtime_rules"` + VerifyCommand *string `toml:"verify_command"` } type TasksConfig struct { diff --git a/internal/core/workspace/config_validate.go b/internal/core/workspace/config_validate.go index 642794f8..df1448b6 100644 --- a/internal/core/workspace/config_validate.go +++ b/internal/core/workspace/config_validate.go @@ -137,12 +137,22 @@ func validateDefaults(scope string, cfg DefaultsConfig) error { } func validateTaskRun(scope string, cfg TaskRunConfig) error { + if cfg.Concurrent != nil && *cfg.Concurrent <= 0 { + return fmt.Errorf( + "%s must be greater than zero (got %d)", + configFieldName(scope, "tasks.run.concurrent"), + *cfg.Concurrent, + ) + } if err := validateOutputFormatValue( configFieldName(scope, "tasks.run.output_format"), cfg.OutputFormat, ); err != nil { return err } + if cfg.VerifyCommand != nil && strings.TrimSpace(*cfg.VerifyCommand) == "" { + return fmt.Errorf("%s cannot be blank", configFieldName(scope, "tasks.run.verify_command")) + } return validateTaskRunRuntimeRules(scope, cfg.TaskRuntimeRules) } diff --git a/internal/daemon/parallel_task_run.go b/internal/daemon/parallel_task_run.go new file mode 100644 index 00000000..6a101a59 --- /dev/null +++ b/internal/daemon/parallel_task_run.go @@ -0,0 +1,1174 @@ +package daemon + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "sort" + "strings" + "time" + + apicore "github.com/itseffi/productize/internal/api/core" + corepkg "github.com/itseffi/productize/internal/core" + "github.com/itseffi/productize/internal/core/memory" + "github.com/itseffi/productize/internal/core/model" + taskworktree "github.com/itseffi/productize/internal/core/run" + "github.com/itseffi/productize/internal/core/subprocess" + "github.com/itseffi/productize/internal/core/tasks" + "github.com/itseffi/productize/internal/store/globaldb" + eventspkg "github.com/itseffi/productize/pkg/productize/events" + "github.com/itseffi/productize/pkg/productize/events/kinds" +) + +const ( + parallelTaskManifestName = "parallel-tasks.json" + parallelChildDrainTimeout = 30 * time.Second +) + +type parallelTaskManifest struct { + SchemaVersion int `json:"schema_version"` + RunID string `json:"run_id"` + Status string `json:"status"` + Concurrent int `json:"concurrent"` + StartingBranch string `json:"starting_branch"` + StartingCommit string `json:"starting_commit"` + IntegrationBranch string `json:"integration_branch"` + IntegrationPath string `json:"integration_path"` + Waves [][]string `json:"waves"` + Tasks []parallelTaskManifestRecord `json:"tasks"` + Verifications []parallelVerificationRecord `json:"verifications,omitempty"` + Finalized bool `json:"finalized"` + FinalCommit string `json:"final_commit,omitempty"` + NextActions []string `json:"next_actions,omitempty"` +} + +type parallelTaskManifestRecord struct { + TaskID string `json:"task_id"` + Dependencies []string `json:"dependencies,omitempty"` + Status string `json:"status"` + RunID string `json:"run_id,omitempty"` + Branch string `json:"branch,omitempty"` + WorktreePath string `json:"worktree_path,omitempty"` + Commit string `json:"commit,omitempty"` + Error string `json:"error,omitempty"` +} + +type parallelVerificationRecord struct { + ID string `json:"id"` + Command string `json:"command"` + Status string `json:"status"` + ExitCode int `json:"exit_code"` + Commit string `json:"commit"` + StdoutPath string `json:"stdout_path"` + StderrPath string `json:"stderr_path"` + Error string `json:"error,omitempty"` +} + +type parallelTaskChild struct { + taskID string + runID string + active *activeRun +} + +type parallelTaskCoordinator struct { + manager *RunManager + active *activeRun + row *globaldb.Run + config *model.RuntimeConfig + lifecycle *taskworktree.Lifecycle + manifest parallelTaskManifest + waves [][]tasks.DependencyNode + completed map[string]struct{} + failed map[string]error + outcomes map[string]taskworktree.TaskOutcome + mergedAny bool +} + +func (m *RunManager) executeParallelTaskRun( + active *activeRun, + row globaldb.Run, + runtimeCfg *model.RuntimeConfig, +) { + fallback := m.runParallelTaskCoordinator(active, &row, runtimeCfg) + m.finishRun(active, row, fallback) +} + +func (m *RunManager) runParallelTaskCoordinator( + active *activeRun, + row *globaldb.Run, + runtimeCfg *model.RuntimeConfig, +) terminalState { + coordinator, terminal := m.prepareParallelTaskCoordinator(active, row, runtimeCfg) + if terminal != nil { + return *terminal + } + return coordinator.run() +} + +func (m *RunManager) prepareParallelTaskCoordinator( + active *activeRun, + row *globaldb.Run, + runtimeCfg *model.RuntimeConfig, +) (*parallelTaskCoordinator, *terminalState) { + artifacts := active.scope.RunArtifacts() + if err := context.Cause(active.ctx); err != nil { + terminal := cancelledTerminalState(err) + return nil, &terminal + } + if len(runtimeCfg.AddDirs) > 0 { + terminal := failedTerminalState( + artifacts, + errors.New( + "parallel task runs do not support add_dirs because shared external directories are not worktree-isolated", + ), + ) + return nil, &terminal + } + if err := startScopeRuntime(active.ctx, active.scope); err != nil { + terminal := fallbackTerminalState(artifacts, err, active.cancelWasRequested()) + return nil, &terminal + } + graph, err := tasks.ReadDependencyGraph(runtimeCfg.TasksDir) + if err != nil { + terminal := failedTerminalState(artifacts, err) + return nil, &terminal + } + waves, err := graph.WavesWithOptions(runtimeCfg.Concurrent, runtimeCfg.IncludeCompleted) + if err != nil { + terminal := failedTerminalState(artifacts, err) + return nil, &terminal + } + nodes := graph.Nodes() + pendingTaskIDs := scheduledParallelTaskIDs(nodes, runtimeCfg.IncludeCompleted) + lifecycle, err := taskworktree.Preflight(active.ctx, taskworktree.Options{ + WorkspaceRoot: runtimeCfg.WorkspaceRoot, + RunID: runtimeCfg.RunID, + TaskIDs: pendingTaskIDs, + }) + if err != nil { + terminal := failedTerminalState(artifacts, err) + return nil, &terminal + } + if err := m.markParallelRunStarted(active, row, runtimeCfg, len(pendingTaskIDs)); err != nil { + terminal := failedTerminalState(artifacts, err) + return nil, &terminal + } + coordinator := ¶llelTaskCoordinator{ + manager: m, + active: active, + row: row, + config: runtimeCfg, + lifecycle: lifecycle, + manifest: newParallelTaskManifest(runtimeCfg, lifecycle, nodes, waves), + waves: waves, + completed: completedParallelTasks(nodes, runtimeCfg.IncludeCompleted), + failed: make(map[string]error), + outcomes: make(map[string]taskworktree.TaskOutcome), + } + if err := coordinator.persistManifest(); err != nil { + terminal := failedTerminalState(artifacts, err) + return nil, &terminal + } + return coordinator, nil +} + +func (m *RunManager) markParallelRunStarted( + active *activeRun, + row *globaldb.Run, + runtimeCfg *model.RuntimeConfig, + jobsTotal int, +) error { + row.Status = runStatusRunning + updated, err := m.globalDB.UpdateRun(detachContext(active.ctx), *row) + if err != nil { + return err + } + *row = updated + m.publishRunWorkspaceEvent( + active.ctx, + *row, + active.workflowSlug, + apicore.WorkspaceEventKindRunStatusChanged, + ) + return emitParallelRunStarted(active, runtimeCfg, jobsTotal) +} + +func emitParallelRunStarted( + active *activeRun, + runtimeCfg *model.RuntimeConfig, + jobsTotal int, +) error { + if active.scope == nil || active.scope.RunJournal() == nil { + return nil + } + return submitSyntheticEvent( + active.ctx, + active.scope.RunJournal(), + active.runID, + eventspkg.EventKindRunStarted, + kinds.RunStartedPayload{ + Mode: string(runtimeCfg.Mode), + Name: runtimeCfg.Name, + WorkspaceRoot: runtimeCfg.WorkspaceRoot, + IDE: runtimeCfg.IDE, + Model: runtimeCfg.Model, + ReasoningEffort: runtimeCfg.ReasoningEffort, + AccessMode: runtimeCfg.AccessMode, + ArtifactsDir: active.scope.RunArtifacts().RunDir, + JobsTotal: jobsTotal, + }, + ) +} + +func (c *parallelTaskCoordinator) run() terminalState { + artifacts := c.active.scope.RunArtifacts() + if c.config.DryRun { + return completedTerminalState(artifacts, "parallel task plan generated") + } + if _, err := c.lifecycle.CreateIntegration(c.active.ctx); err != nil { + return failedTerminalState(artifacts, err) + } + c.manifest.Status = "running" + if err := c.persistManifest(); err != nil { + return failedTerminalState(artifacts, err) + } + if err := initializeParallelWorkflowMemory(c.active.ctx, c.lifecycle, c.config); err != nil { + return c.failureTerminal( + err, + "Inspect the retained integration worktree and fix workflow-memory initialization failures.", + ) + } + if err := c.executeWaves(); err != nil { + return c.terminalForExecutionError(err) + } + if !c.mergedAny && len(c.failed) > 0 { + return c.failedTasksTerminal() + } + if err := refreshParallelProjectKnowledge(c.active.ctx, c.lifecycle); err != nil { + return c.failureTerminal( + err, + "Inspect the retained integration worktree and fix project-knowledge refresh failures.", + ) + } + if err := c.verifyIntegration("verify-final"); err != nil { + return c.failureTerminal( + err, + "Inspect the retained integration worktree and fix final verification failures.", + ) + } + if len(c.failed) > 0 { + return c.failedTasksTerminal() + } + return c.finalize() +} + +func (c *parallelTaskCoordinator) executeWaves() error { + for waveIndex, wave := range c.waves { + if err := context.Cause(c.active.ctx); err != nil { + return err + } + if err := c.executeWave(waveIndex, wave); err != nil { + return err + } + } + return nil +} + +func (c *parallelTaskCoordinator) executeWave( + waveIndex int, + wave []tasks.DependencyNode, +) error { + runnable := c.runnableTasks(wave) + if len(runnable) == 0 { + return c.persistManifest() + } + children, err := c.startWaveChildren(runnable) + if err != nil { + return errors.Join(err, cancelAndWaitParallelChildren(c.active.ctx, children)) + } + childRows, err := c.manager.waitParallelTaskChildren(c.active.ctx, children) + if err != nil { + return err + } + c.completeWaveChildren(children, childRows) + merged, err := c.mergeWaveTasks(runnable) + if err != nil { + return err + } + if !merged { + return c.persistManifest() + } + if err := rebuildParallelWorkflowMemory(c.active.ctx, c.lifecycle, c.config, waveIndex); err != nil { + return err + } + if err := c.verifyIntegration(fmt.Sprintf("verify-wave-%03d", waveIndex+1)); err != nil { + return fmt.Errorf("verify integrated task wave %d: %w", waveIndex+1, err) + } + return c.persistManifest() +} + +func (c *parallelTaskCoordinator) runnableTasks(wave []tasks.DependencyNode) []tasks.DependencyNode { + runnable := make([]tasks.DependencyNode, 0, len(wave)) + for _, node := range wave { + dependency := unsatisfiedParallelDependency(node, c.completed, c.failed) + if dependency == "" { + runnable = append(runnable, node) + continue + } + err := fmt.Errorf("blocked by dependency %s", dependency) + c.failed[node.ID] = err + c.outcomes[node.ID] = taskworktree.TaskFailed + c.manifest.updateTask(node.ID, "blocked", "", "", "", "", err) + } + return runnable +} + +func (c *parallelTaskCoordinator) startWaveChildren( + runnable []tasks.DependencyNode, +) ([]parallelTaskChild, error) { + children := make([]parallelTaskChild, 0, len(runnable)) + for _, node := range runnable { + if err := context.Cause(c.active.ctx); err != nil { + return children, err + } + checkout, err := c.lifecycle.CreateTask(c.active.ctx, node.ID) + if err != nil { + c.recordTaskFailure(node.ID, "failed", "", taskworktree.TaskFailed, err) + continue + } + child, err := c.manager.startParallelTaskChild(c.active, *c.row, c.config, node.ID, checkout) + if err != nil { + c.recordTaskFailure(node.ID, "failed", "", taskworktree.TaskFailed, err) + c.manifest.updateTask(node.ID, "failed", "", checkout.Branch, checkout.WorktreePath, "", err) + continue + } + children = append(children, child) + c.manifest.updateTask( + node.ID, + "running", + child.runID, + checkout.Branch, + checkout.WorktreePath, + "", + nil, + ) + } + return children, c.persistManifest() +} + +func (c *parallelTaskCoordinator) completeWaveChildren( + children []parallelTaskChild, + rows map[string]globaldb.Run, +) { + for _, child := range children { + c.completeWaveChild(child, rows[child.taskID]) + } +} + +func (c *parallelTaskCoordinator) completeWaveChild(child parallelTaskChild, row globaldb.Run) { + if row.Status != runStatusCompleted { + err := fmt.Errorf( + "task child run %s ended with status %s: %s", + child.runID, + row.Status, + row.ErrorText, + ) + c.recordTaskFailure(child.taskID, "failed", child.runID, taskworktree.TaskFailed, err) + return + } + layout, err := c.lifecycle.TaskWorktree(child.taskID) + if err != nil { + c.recordTaskFailure(child.taskID, "failed", child.runID, taskworktree.TaskFailed, err) + return + } + childTasksDir := model.TaskDirectoryForWorkspace(layout.WorktreePath, c.config.Name) + completed, err := parallelTaskCompleted(childTasksDir, child.taskID) + if err != nil || !completed { + if err == nil { + err = fmt.Errorf("task %s remained pending after its child run", child.taskID) + } + c.recordTaskFailure(child.taskID, "failed", child.runID, taskworktree.TaskFailed, err) + return + } + memoryRoot, err := filepath.Rel(layout.WorktreePath, memory.Directory(childTasksDir)) + if err != nil { + c.recordTaskFailure(child.taskID, "failed", child.runID, taskworktree.TaskFailed, err) + return + } + if err := c.lifecycle.ValidateTaskMemoryIsolation(c.active.ctx, child.taskID, memoryRoot); err != nil { + c.recordTaskFailure(child.taskID, "failed", child.runID, taskworktree.TaskFailed, err) + return + } + commit, err := c.lifecycle.CommitTask(c.active.ctx, child.taskID, "productize: complete "+child.taskID) + if err != nil { + c.recordTaskFailure(child.taskID, "failed", child.runID, taskworktree.TaskFailed, err) + return + } + c.manifest.updateTask( + child.taskID, + runStatusCompleted, + child.runID, + layout.Branch, + layout.WorktreePath, + commit.Commit, + nil, + ) +} + +func (c *parallelTaskCoordinator) mergeWaveTasks(runnable []tasks.DependencyNode) (bool, error) { + merged := false + for _, node := range runnable { + if err := context.Cause(c.active.ctx); err != nil { + return merged, err + } + if _, taskFailed := c.failed[node.ID]; taskFailed { + continue + } + results, err := c.lifecycle.MergeTasks(c.active.ctx, []string{node.ID}) + if err != nil { + c.recordTaskFailure(node.ID, "conflicted", "", taskworktree.TaskConflicted, err) + continue + } + c.completed[node.ID] = struct{}{} + c.outcomes[node.ID] = taskworktree.TaskSucceeded + merged = true + c.mergedAny = true + if len(results) == 1 { + c.manifest.updateTask(node.ID, "merged", "", "", "", results[0].TaskCommit, nil) + } + } + return merged, nil +} + +func (c *parallelTaskCoordinator) recordTaskFailure( + taskID string, + status string, + runID string, + outcome taskworktree.TaskOutcome, + err error, +) { + c.failed[taskID] = err + c.outcomes[taskID] = outcome + c.manifest.updateTask(taskID, status, runID, "", "", "", err) +} + +func (c *parallelTaskCoordinator) verifyIntegration(label string) error { + record, err := runParallelIntegrationVerification( + c.active, + c.config, + c.lifecycle, + label, + ) + c.manifest.Verifications = append(c.manifest.Verifications, record) + return err +} + +func (c *parallelTaskCoordinator) terminalForExecutionError(err error) terminalState { + if context.Cause(c.active.ctx) != nil || errors.Is(err, context.Canceled) { + c.manifest.Status = "canceled" + c.manifest.NextActions = []string{"Inspect retained task worktrees before restarting the workflow."} + if persistErr := c.persistManifest(); persistErr != nil { + err = errors.Join(err, persistErr) + } + return cancelledTerminalState(err) + } + return c.failureTerminal(err, "Inspect the retained integration worktree and task worktrees before retrying.") +} + +func (c *parallelTaskCoordinator) failedTasksTerminal() terminalState { + taskErr := parallelTaskFailures(c.failed) + c.manifest.Status = "failed" + c.manifest.NextActions = []string{ + "Inspect retained failed task worktrees and rerun the workflow; completed tasks will be skipped.", + } + if c.mergedAny { + finalized, finalizeErr := c.lifecycle.Finalize(c.active.ctx) + switch { + case finalizeErr == nil: + c.manifest.Finalized = true + c.manifest.FinalCommit = finalized.FinalHEAD + case errors.Is(finalizeErr, taskworktree.ErrOriginalChanged): + c.manifest.NextActions = append( + c.manifest.NextActions, + fmt.Sprintf("git merge --ff-only %s", c.manifest.IntegrationBranch), + ) + default: + taskErr = errors.Join(taskErr, finalizeErr) + } + } + _, cleanupErr := c.lifecycle.Cleanup(c.active.ctx, c.outcomes) + if c.manifest.Finalized { + if _, syncErr := c.manager.syncActiveWorkflow(c.active.ctx, c.active); syncErr != nil { + cleanupErr = errors.Join(cleanupErr, syncErr) + } + } + return failedTerminalState( + c.active.scope.RunArtifacts(), + errors.Join(taskErr, cleanupErr, c.persistManifest()), + ) +} + +func (c *parallelTaskCoordinator) failureTerminal(err error, nextAction string) terminalState { + if context.Cause(c.active.ctx) != nil || errors.Is(err, context.Canceled) { + return c.terminalForExecutionError(err) + } + c.manifest.Status = "failed" + c.manifest.NextActions = []string{nextAction} + _, cleanupErr := c.lifecycle.Cleanup(c.active.ctx, c.outcomes) + persistErr := c.persistManifest() + return failedTerminalState( + c.active.scope.RunArtifacts(), + errors.Join(err, cleanupErr, persistErr), + ) +} + +func (c *parallelTaskCoordinator) finalize() terminalState { + finalized, err := c.lifecycle.Finalize(c.active.ctx) + switch { + case err == nil: + c.manifest.Status = runStatusCompleted + c.manifest.Finalized = true + c.manifest.FinalCommit = finalized.FinalHEAD + case errors.Is(err, taskworktree.ErrOriginalChanged): + c.manifest.Status = "needs_merge" + c.manifest.NextActions = []string{ + fmt.Sprintf("git merge --ff-only %s", c.manifest.IntegrationBranch), + } + default: + return c.failureTerminal( + err, + "Inspect the retained integration worktree before retrying finalization.", + ) + } + return c.finalizeReadModels() +} + +func (c *parallelTaskCoordinator) finalizeReadModels() terminalState { + _, cleanupErr := c.lifecycle.Cleanup(c.active.ctx, c.outcomes) + if c.manifest.Finalized { + if _, syncErr := c.manager.syncActiveWorkflow(c.active.ctx, c.active); syncErr != nil { + cleanupErr = errors.Join(cleanupErr, syncErr) + } + } + cleanupErr = errors.Join(cleanupErr, c.persistManifest()) + if cleanupErr != nil { + return failedTerminalState(c.active.scope.RunArtifacts(), cleanupErr) + } + if c.manifest.Finalized { + return completedTerminalState( + c.active.scope.RunArtifacts(), + "parallel task workflow completed and fast-forwarded", + ) + } + return completedTerminalState( + c.active.scope.RunArtifacts(), + "parallel tasks verified but the original checkout changed; "+c.manifest.NextActions[0], + ) +} + +func (c *parallelTaskCoordinator) persistManifest() error { + artifacts := c.active.scope.RunArtifacts() + if err := writeParallelTaskManifest(artifacts, c.manifest); err != nil { + return err + } + if c.active.scope.RunJournal() == nil { + return nil + } + return submitSyntheticEvent( + c.active.ctx, + c.active.scope.RunJournal(), + c.active.runID, + eventspkg.EventKindTaskSchedulerUpdated, + parallelSchedulerPayload(artifacts, c.manifest), + ) +} + +func parallelSchedulerPayload( + artifacts model.RunArtifacts, + manifest parallelTaskManifest, +) kinds.TaskSchedulerUpdatedPayload { + payload := kinds.TaskSchedulerUpdatedPayload{ + ManifestPath: filepath.Join(artifacts.RunDir, parallelTaskManifestName), + Status: manifest.Status, + Concurrent: manifest.Concurrent, + StartingBranch: manifest.StartingBranch, + StartingCommit: manifest.StartingCommit, + IntegrationBranch: manifest.IntegrationBranch, + IntegrationPath: manifest.IntegrationPath, + Waves: cloneParallelWaves(manifest.Waves), + Tasks: make([]kinds.TaskSchedulerTaskState, 0, len(manifest.Tasks)), + Verifications: make([]kinds.TaskSchedulerVerification, 0, len(manifest.Verifications)), + Finalized: manifest.Finalized, + FinalCommit: manifest.FinalCommit, + NextActions: append([]string(nil), manifest.NextActions...), + } + for index := range manifest.Tasks { + task := &manifest.Tasks[index] + payload.Tasks = append(payload.Tasks, kinds.TaskSchedulerTaskState{ + TaskID: task.TaskID, + Dependencies: append([]string(nil), task.Dependencies...), + Status: task.Status, + RunID: task.RunID, + Branch: task.Branch, + WorktreePath: task.WorktreePath, + Commit: task.Commit, + Error: task.Error, + }) + } + for _, verification := range manifest.Verifications { + payload.Verifications = append(payload.Verifications, kinds.TaskSchedulerVerification{ + ID: verification.ID, + Command: verification.Command, + Status: verification.Status, + ExitCode: verification.ExitCode, + Commit: verification.Commit, + StdoutPath: verification.StdoutPath, + StderrPath: verification.StderrPath, + Error: verification.Error, + }) + } + sort.Strings(payload.NextActions) + return payload +} + +func cloneParallelWaves(waves [][]string) [][]string { + cloned := make([][]string, len(waves)) + for index := range waves { + cloned[index] = append([]string(nil), waves[index]...) + } + return cloned +} + +func (m *RunManager) startParallelTaskChild( + parent *activeRun, + parentRow globaldb.Run, + baseCfg *model.RuntimeConfig, + taskID string, + checkout taskworktree.CheckoutResult, +) (parallelTaskChild, error) { + workspace, err := m.globalDB.Get(parent.ctx, parentRow.WorkspaceID) + if err != nil { + return parallelTaskChild{}, fmt.Errorf("load parent workspace for %s: %w", taskID, err) + } + childCfg := baseCfg.Clone() + childCfg.WorkspaceRoot = checkout.WorktreePath + childCfg.TasksDir = model.TaskDirectoryForWorkspace(checkout.WorktreePath, baseCfg.Name) + childCfg.Concurrent = 1 + childCfg.BatchSize = 1 + childCfg.TaskIDs = []string{taskID} + childCfg.TaskMemoryLocalOnly = true + childCfg.RunID = parallelChildRunID(parent.runID, taskID) + childCfg.ParentRunID = parent.runID + childCfg.DryRun = false + childCfg.DaemonOwned = true + + run, err := m.startRun(parent.ctx, startRunSpec{ + workspace: workspace, + workflowID: parent.workflowID, + workflowSlug: parent.workflowSlug, + mode: runModeTask, + presentationMode: "detach", + parentRunID: parent.runID, + runtimeCfg: childCfg, + }) + if err != nil { + return parallelTaskChild{}, err + } + return parallelTaskChild{ + taskID: taskID, + runID: run.RunID, + active: m.getActive(run.RunID), + }, nil +} + +func (m *RunManager) waitParallelTaskChildren( + ctx context.Context, + children []parallelTaskChild, +) (map[string]globaldb.Run, error) { + rows := make(map[string]globaldb.Run, len(children)) + for _, child := range children { + if child.active != nil { + select { + case <-child.active.done: + case <-ctx.Done(): + drainErr := cancelAndWaitParallelChildren(ctx, children) + return nil, errors.Join(context.Cause(ctx), drainErr) + } + } + row, err := m.globalDB.GetRun(detachContext(ctx), child.runID) + if err != nil { + return nil, fmt.Errorf("load child run %s: %w", child.runID, err) + } + rows[child.taskID] = row + } + return rows, nil +} + +func cancelParallelChildren(children []parallelTaskChild) { + for _, child := range children { + if child.active == nil { + continue + } + child.active.markCancelRequested() + child.active.cancel() + } +} + +func cancelAndWaitParallelChildren(ctx context.Context, children []parallelTaskChild) error { + cancelParallelChildren(children) + drainCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), parallelChildDrainTimeout) + defer cancel() + for _, child := range children { + if child.active == nil { + continue + } + select { + case <-child.active.done: + case <-drainCtx.Done(): + return fmt.Errorf("wait for child run %s to stop: %w", child.runID, drainCtx.Err()) + } + } + return nil +} + +func runParallelIntegrationVerification( + parent *activeRun, + baseCfg *model.RuntimeConfig, + lifecycle *taskworktree.Lifecycle, + label string, +) (parallelVerificationRecord, error) { + record := parallelVerificationRecord{ID: strings.TrimSpace(label), Status: "failed", ExitCode: -1} + if err := lifecycle.RequireIntegrationClean(parent.ctx); err != nil { + record.Error = err.Error() + return record, err + } + beforeHEAD, err := lifecycle.IntegrationHEAD(parent.ctx) + if err != nil { + record.Error = err.Error() + return record, err + } + record.Commit = beforeHEAD + command, err := resolveParallelVerificationCommand( + lifecycle.Layout().IntegrationWorktree, + baseCfg.VerificationCommand, + ) + if err != nil { + record.Error = err.Error() + return record, err + } + record.Command = command + stdoutPath, stderrPath, exitCode, err := executeParallelVerificationCommand( + parent.ctx, + parent.scope.RunArtifacts(), + lifecycle.Layout().IntegrationWorktree, + label, + command, + ) + record.StdoutPath = stdoutPath + record.StderrPath = stderrPath + record.ExitCode = exitCode + if err != nil { + record.Error = err.Error() + return record, err + } + if err := lifecycle.RequireIntegrationClean(parent.ctx); err != nil { + err = fmt.Errorf("integration verification modified the checkout: %w", err) + record.Error = err.Error() + return record, err + } + afterHEAD, err := lifecycle.IntegrationHEAD(parent.ctx) + if err != nil { + record.Error = err.Error() + return record, err + } + if afterHEAD != beforeHEAD { + err = fmt.Errorf( + "integration verification changed HEAD from %s to %s", + beforeHEAD, + afterHEAD, + ) + record.Error = err.Error() + return record, err + } + record.Status = "passed" + return record, nil +} + +func executeParallelVerificationCommand( + ctx context.Context, + artifacts model.RunArtifacts, + workspaceRoot string, + label string, + command string, +) (string, string, int, error) { + safeLabel := strings.NewReplacer("/", "-", "\\", "-").Replace(strings.TrimSpace(label)) + stdoutPath := filepath.Join(artifacts.RunDir, safeLabel+".out.log") + stderrPath := filepath.Join(artifacts.RunDir, safeLabel+".err.log") + stdout, err := os.OpenFile(stdoutPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return stdoutPath, stderrPath, -1, fmt.Errorf("create verification stdout log: %w", err) + } + stderr, err := os.OpenFile(stderrPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + _ = stdout.Close() + return stdoutPath, stderrPath, -1, fmt.Errorf("create verification stderr log: %w", err) + } + cmd := exec.CommandContext(ctx, "/bin/sh", "-c", command) + cmd.Dir = workspaceRoot + cmd.Stdout = stdout + cmd.Stderr = stderr + if err := subprocess.ConfigureCommand(cmd); err != nil { + closeErr := errors.Join(stdout.Close(), stderr.Close()) + return stdoutPath, stderrPath, -1, errors.Join( + fmt.Errorf("configure parallel verification command: %w", err), + closeErr, + ) + } + runErr := cmd.Run() + closeErr := errors.Join(stdout.Close(), stderr.Close()) + if runErr == nil && closeErr == nil { + return stdoutPath, stderrPath, 0, nil + } + exitCode := -1 + var exitErr *exec.ExitError + if errors.As(runErr, &exitErr) { + exitCode = exitErr.ExitCode() + } + var commandErr error + if runErr != nil { + commandErr = fmt.Errorf("parallel verification command %q failed: %w", command, runErr) + } + return stdoutPath, stderrPath, exitCode, errors.Join(commandErr, closeErr) +} + +func resolveParallelVerificationCommand(workspaceRoot string, configured string) (string, error) { + if command := strings.TrimSpace(configured); command != "" { + return command, nil + } + makefilePath := filepath.Join(workspaceRoot, "Makefile") + if content, err := os.ReadFile(makefilePath); err == nil && makefileHasVerifyTarget(string(content)) { + return "make verify", nil + } else if err != nil && !errors.Is(err, os.ErrNotExist) { + return "", fmt.Errorf("inspect Makefile verification target: %w", err) + } + if command, found, err := resolveNodeVerificationCommand(workspaceRoot); err != nil { + return "", err + } else if found { + return command, nil + } + if _, err := os.Stat(filepath.Join(workspaceRoot, "go.mod")); err == nil { + return "go test ./...", nil + } else if !errors.Is(err, os.ErrNotExist) { + return "", fmt.Errorf("inspect Go verification target: %w", err) + } + if _, err := os.Stat(filepath.Join(workspaceRoot, "Cargo.toml")); err == nil { + return "cargo test --workspace", nil + } else if !errors.Is(err, os.ErrNotExist) { + return "", fmt.Errorf("inspect Rust verification target: %w", err) + } + return "", errors.New( + "parallel task verification command is required; pass --verify-command or set tasks.run.verify_command", + ) +} + +func resolveNodeVerificationCommand(workspaceRoot string) (string, bool, error) { + content, err := os.ReadFile(filepath.Join(workspaceRoot, "package.json")) + if errors.Is(err, os.ErrNotExist) { + return "", false, nil + } + if err != nil { + return "", false, fmt.Errorf("read package.json verification target: %w", err) + } + var manifest struct { + Scripts map[string]string `json:"scripts"` + } + if err := json.Unmarshal(content, &manifest); err != nil { + return "", false, fmt.Errorf("parse package.json verification target: %w", err) + } + script := "" + for _, candidate := range []string{"verify", "test"} { + if strings.TrimSpace(manifest.Scripts[candidate]) != "" { + script = candidate + break + } + } + if script == "" { + return "", false, nil + } + manager := "npm" + for _, candidate := range []struct { + path string + manager string + }{ + {path: "pnpm-lock.yaml", manager: "pnpm"}, + {path: "yarn.lock", manager: "yarn"}, + {path: "bun.lock", manager: "bun"}, + {path: "bun.lockb", manager: "bun"}, + } { + if _, err := os.Stat(filepath.Join(workspaceRoot, candidate.path)); err == nil { + manager = candidate.manager + break + } else if !errors.Is(err, os.ErrNotExist) { + return "", false, fmt.Errorf("inspect Node lockfile %s: %w", candidate.path, err) + } + } + return manager + " run " + script, true, nil +} + +func makefileHasVerifyTarget(content string) bool { + for _, line := range strings.Split(content, "\n") { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "verify:") || strings.HasPrefix(trimmed, "verify :") { + return true + } + } + return false +} + +func rebuildParallelWorkflowMemory( + ctx context.Context, + lifecycle *taskworktree.Lifecycle, + runtimeCfg *model.RuntimeConfig, + waveIndex int, +) error { + integrationTasksDir := model.TaskDirectoryForWorkspace( + lifecycle.Layout().IntegrationWorktree, + runtimeCfg.Name, + ) + if _, err := memory.RebuildWorkflowIndex(integrationTasksDir); err != nil { + return fmt.Errorf("rebuild shared workflow memory after wave %d: %w", waveIndex+1, err) + } + if _, err := lifecycle.CommitIntegration( + ctx, + fmt.Sprintf("productize: integrate task memory wave %d", waveIndex+1), + ); err != nil { + return fmt.Errorf("commit shared workflow memory after wave %d: %w", waveIndex+1, err) + } + return nil +} + +func initializeParallelWorkflowMemory( + ctx context.Context, + lifecycle *taskworktree.Lifecycle, + runtimeCfg *model.RuntimeConfig, +) error { + integrationTasksDir := model.TaskDirectoryForWorkspace( + lifecycle.Layout().IntegrationWorktree, + runtimeCfg.Name, + ) + if _, err := memory.RebuildWorkflowIndex(integrationTasksDir); err != nil { + return fmt.Errorf("initialize shared workflow memory: %w", err) + } + if _, err := lifecycle.CommitIntegration(ctx, "productize: initialize workflow memory"); err != nil { + return fmt.Errorf("commit initialized workflow memory: %w", err) + } + return nil +} + +func refreshParallelProjectKnowledge( + ctx context.Context, + lifecycle *taskworktree.Lifecycle, +) error { + if _, err := corepkg.RefreshProjectKnowledge(ctx, lifecycle.Layout().IntegrationWorktree); err != nil { + return fmt.Errorf("refresh project knowledge in integration worktree: %w", err) + } + if _, err := lifecycle.CommitIntegration(ctx, "productize: refresh project knowledge"); err != nil { + return fmt.Errorf("commit refreshed project knowledge: %w", err) + } + return lifecycle.RequireIntegrationClean(ctx) +} + +func scheduledParallelTaskIDs(nodes []tasks.DependencyNode, includeCompleted bool) []string { + ids := make([]string, 0, len(nodes)) + for _, node := range nodes { + if !includeCompleted && tasks.IsTaskCompleted(model.TaskEntry{Status: node.Status}) { + continue + } + ids = append(ids, node.ID) + } + return ids +} + +func completedParallelTasks(nodes []tasks.DependencyNode, includeCompleted bool) map[string]struct{} { + completed := make(map[string]struct{}, len(nodes)) + for _, node := range nodes { + if !includeCompleted && tasks.IsTaskCompleted(model.TaskEntry{Status: node.Status}) { + completed[node.ID] = struct{}{} + } + } + return completed +} + +func unsatisfiedParallelDependency( + node tasks.DependencyNode, + completed map[string]struct{}, + failed map[string]error, +) string { + for _, dependency := range node.Dependencies { + if _, ok := failed[dependency]; ok { + return dependency + } + if _, ok := completed[dependency]; !ok { + return dependency + } + } + return "" +} + +func parallelTaskCompleted(tasksDir, taskID string) (bool, error) { + entries, err := tasks.ReadTaskEntries(tasksDir, true) + if err != nil { + return false, err + } + for _, entry := range entries { + if entry.CodeFile != strings.TrimSuffix(taskID, filepath.Ext(taskID)) { + continue + } + task, err := tasks.ParseTaskFile(entry.Content) + if err != nil { + return false, err + } + return tasks.IsTaskCompleted(task), nil + } + return false, fmt.Errorf("task %s not found in %s", taskID, tasksDir) +} + +func newParallelTaskManifest( + cfg *model.RuntimeConfig, + lifecycle *taskworktree.Lifecycle, + nodes []tasks.DependencyNode, + waves [][]tasks.DependencyNode, +) parallelTaskManifest { + start := lifecycle.StartState() + layout := lifecycle.Layout() + manifest := parallelTaskManifest{ + SchemaVersion: 1, + RunID: cfg.RunID, + Status: "planned", + Concurrent: cfg.Concurrent, + StartingBranch: start.Branch, + StartingCommit: start.HEAD, + IntegrationBranch: layout.IntegrationBranch, + IntegrationPath: layout.IntegrationWorktree, + Waves: make([][]string, 0, len(waves)), + Tasks: make([]parallelTaskManifestRecord, 0, len(nodes)), + } + for _, wave := range waves { + ids := make([]string, 0, len(wave)) + for _, node := range wave { + ids = append(ids, node.ID) + } + manifest.Waves = append(manifest.Waves, ids) + } + for _, node := range nodes { + status := runStatusPending + if tasks.IsTaskCompleted(model.TaskEntry{Status: node.Status}) && !cfg.IncludeCompleted { + status = runStatusCompleted + } + record := parallelTaskManifestRecord{ + TaskID: node.ID, + Dependencies: append([]string(nil), node.Dependencies...), + Status: status, + } + if taskLayout, err := lifecycle.TaskWorktree(node.ID); err == nil { + record.Branch = taskLayout.Branch + record.WorktreePath = taskLayout.WorktreePath + } + manifest.Tasks = append(manifest.Tasks, record) + } + return manifest +} + +func (m *parallelTaskManifest) updateTask( + taskID string, + status string, + runID string, + branch string, + worktreePath string, + commit string, + err error, +) { + for idx := range m.Tasks { + record := &m.Tasks[idx] + if record.TaskID != taskID { + continue + } + record.Status = status + if runID != "" { + record.RunID = runID + } + if branch != "" { + record.Branch = branch + } + if worktreePath != "" { + record.WorktreePath = worktreePath + } + if commit != "" { + record.Commit = commit + } + if err != nil { + record.Error = err.Error() + } else { + record.Error = "" + } + return + } +} + +func writeParallelTaskManifest(artifacts model.RunArtifacts, manifest parallelTaskManifest) error { + manifest.NextActions = append([]string(nil), manifest.NextActions...) + sort.Strings(manifest.NextActions) + payload, err := json.MarshalIndent(manifest, "", " ") + if err != nil { + return fmt.Errorf("encode parallel task manifest: %w", err) + } + payload = append(payload, '\n') + path := filepath.Join(artifacts.RunDir, parallelTaskManifestName) + tmp, err := os.CreateTemp(artifacts.RunDir, parallelTaskManifestName+".tmp-*") + if err != nil { + return fmt.Errorf("create parallel task manifest: %w", err) + } + tmpPath := tmp.Name() + cleanup := func() { + _ = tmp.Close() + _ = os.Remove(tmpPath) + } + if _, err := tmp.Write(payload); err != nil { + cleanup() + return fmt.Errorf("write parallel task manifest: %w", err) + } + if err := tmp.Close(); err != nil { + _ = os.Remove(tmpPath) + return fmt.Errorf("close parallel task manifest: %w", err) + } + if err := os.Chmod(tmpPath, 0o600); err != nil { + _ = os.Remove(tmpPath) + return fmt.Errorf("chmod parallel task manifest: %w", err) + } + if err := os.Rename(tmpPath, path); err != nil { + _ = os.Remove(tmpPath) + return fmt.Errorf("replace parallel task manifest: %w", err) + } + return nil +} + +func parallelTaskFailures(failures map[string]error) error { + ids := make([]string, 0, len(failures)) + for taskID := range failures { + ids = append(ids, taskID) + } + sort.Strings(ids) + parts := make([]string, 0, len(ids)) + for _, taskID := range ids { + parts = append(parts, fmt.Sprintf("%s: %v", taskID, failures[taskID])) + } + return fmt.Errorf("parallel task workflow failed: %s", strings.Join(parts, "; ")) +} + +func parallelChildRunID(parentRunID, taskID string) string { + return strings.TrimSpace(parentRunID) + "-" + strings.TrimSpace(taskID) +} diff --git a/internal/daemon/parallel_task_run_test.go b/internal/daemon/parallel_task_run_test.go new file mode 100644 index 00000000..ba1d4dea --- /dev/null +++ b/internal/daemon/parallel_task_run_test.go @@ -0,0 +1,485 @@ +package daemon + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" + + apicore "github.com/itseffi/productize/internal/api/core" + "github.com/itseffi/productize/internal/core/model" + taskworktree "github.com/itseffi/productize/internal/core/run" + "github.com/itseffi/productize/internal/store/globaldb" + eventspkg "github.com/itseffi/productize/pkg/productize/events" +) + +func TestRunManagerParallelTasksUsesWaveBarrierAndFinalizes(t *testing.T) { + started := make(chan string, 2) + release := make(chan struct{}) + var running atomic.Int32 + var maximum atomic.Int32 + env := newRunManagerTestEnv(t, runManagerTestDeps{ + prepare: func( + _ context.Context, + cfg *model.RuntimeConfig, + scope model.RunScope, + ) (*model.SolvePreparation, error) { + return &model.SolvePreparation{ + Jobs: []model.Job{{CodeFiles: append([]string(nil), cfg.TaskIDs...)}}, + RunArtifacts: scope.RunArtifacts(), + }, nil + }, + execute: func(ctx context.Context, _ *model.SolvePreparation, cfg *model.RuntimeConfig) error { + if _, err := os.Stat(filepath.Join(cfg.TasksDir, "memory", "MEMORY.md")); err != nil { + return fmt.Errorf("shared workflow memory was not initialized before child execution: %w", err) + } + current := running.Add(1) + defer running.Add(-1) + for { + prior := maximum.Load() + if current <= prior || maximum.CompareAndSwap(prior, current) { + break + } + } + taskID := cfg.TaskIDs[0] + started <- taskID + select { + case <-release: + case <-ctx.Done(): + return context.Cause(ctx) + } + taskPath := filepath.Join(cfg.TasksDir, taskID+".md") + content, err := os.ReadFile(taskPath) + if err != nil { + return err + } + completed := strings.Replace(string(content), "status: pending", "status: completed", 1) + if err := os.WriteFile(taskPath, []byte(completed), 0o600); err != nil { + return err + } + return os.WriteFile(filepath.Join(cfg.WorkspaceRoot, taskID+".txt"), []byte(taskID+"\n"), 0o600) + }, + }) + + initializeParallelDaemonWorkspace(t, env) + + run := env.startTaskRun( + t, + "parallel-parent", + rawJSON(t, `{"run_id":"parallel-parent","concurrent":2,"verification_command":"true"}`), + ) + waitCtx, cancel := context.WithTimeout(t.Context(), 10*time.Second) + defer cancel() + seen := make(map[string]struct{}, 2) + for len(seen) < 2 { + select { + case taskID := <-started: + seen[taskID] = struct{}{} + case <-waitCtx.Done(): + row, rowErr := env.globalDB.GetRun(t.Context(), run.RunID) + status := runGitOutput(t, env.workspaceRoot, "status", "--porcelain") + t.Fatalf( + "wait for concurrent task starts: %v; run=%#v; load=%v; git status=%q", + waitCtx.Err(), + row, + rowErr, + status, + ) + } + } + close(release) + row := waitForRun(t, env.globalDB, run.RunID, func(row globaldb.Run) bool { + return row.Status == runStatusCompleted || row.Status == runStatusFailed + }) + if row.Status != runStatusCompleted { + t.Fatalf("parallel run status = %q, error = %q", row.Status, row.ErrorText) + } + if maximum.Load() != 2 { + t.Fatalf("maximum concurrent task executions = %d, want 2", maximum.Load()) + } + for _, taskID := range []string{"task_01", "task_02"} { + if _, err := os.Stat(filepath.Join(env.workspaceRoot, taskID+".txt")); err != nil { + t.Fatalf("finalized task output %s: %v", taskID, err) + } + } + artifacts, err := model.ResolveHomeRunArtifacts(run.RunID) + if err != nil { + t.Fatalf("ResolveHomeRunArtifacts() error = %v", err) + } + payload, err := os.ReadFile(filepath.Join(artifacts.RunDir, parallelTaskManifestName)) + if err != nil { + t.Fatalf("read parallel manifest: %v", err) + } + var manifest parallelTaskManifest + if err := json.Unmarshal(payload, &manifest); err != nil { + t.Fatalf("decode parallel manifest: %v", err) + } + if !manifest.Finalized || len(manifest.Verifications) != 2 { + t.Fatalf("parallel manifest = %#v", manifest) + } + events, err := env.manager.Events(t.Context(), run.RunID, apicore.RunEventPageQuery{}) + if err != nil { + t.Fatalf("list run events: %v", err) + } + foundCheckpoint := false + for _, event := range events.Events { + if event.Kind == eventspkg.EventKindTaskSchedulerUpdated { + foundCheckpoint = true + break + } + } + if !foundCheckpoint { + t.Fatal("parallel run events did not contain task.scheduler_updated") + } +} + +func TestRunManagerParallelRerunSkipsPreviouslyIntegratedTasks(t *testing.T) { + var failSecond atomic.Bool + failSecond.Store(true) + var firstCalls atomic.Int32 + var secondCalls atomic.Int32 + env := newRunManagerTestEnv(t, runManagerTestDeps{ + prepare: func( + _ context.Context, + cfg *model.RuntimeConfig, + scope model.RunScope, + ) (*model.SolvePreparation, error) { + return &model.SolvePreparation{ + Jobs: []model.Job{{CodeFiles: append([]string(nil), cfg.TaskIDs...)}}, + RunArtifacts: scope.RunArtifacts(), + }, nil + }, + execute: func(_ context.Context, _ *model.SolvePreparation, cfg *model.RuntimeConfig) error { + taskID := cfg.TaskIDs[0] + if taskID == "task_01" { + firstCalls.Add(1) + } else { + secondCalls.Add(1) + if failSecond.Load() { + return errors.New("task_02 failed intentionally") + } + } + taskPath := filepath.Join(cfg.TasksDir, taskID+".md") + content, err := os.ReadFile(taskPath) + if err != nil { + return err + } + completed := strings.Replace(string(content), "status: pending", "status: completed", 1) + if err := os.WriteFile(taskPath, []byte(completed), 0o600); err != nil { + return err + } + return os.WriteFile(filepath.Join(cfg.WorkspaceRoot, taskID+".txt"), []byte(taskID+"\n"), 0o600) + }, + }) + initializeParallelDaemonWorkspace(t, env) + + first := env.startTaskRun( + t, + "parallel-partial", + rawJSON(t, `{"run_id":"parallel-partial","concurrent":2,"verification_command":"true"}`), + ) + firstRow := waitForRun(t, env.globalDB, first.RunID, func(row globaldb.Run) bool { + return row.Status == runStatusCompleted || row.Status == runStatusFailed + }) + if firstRow.Status != runStatusFailed { + t.Fatalf("first run status = %q, want failed", firstRow.Status) + } + if _, err := os.Stat(filepath.Join(env.workspaceRoot, "task_01.txt")); err != nil { + t.Fatalf("successful partial task was not finalized: %v", err) + } + if _, err := os.Stat(filepath.Join(env.workspaceRoot, "task_02.txt")); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("failed task output exists or stat failed unexpectedly: %v", err) + } + if status := runGitOutput(t, env.workspaceRoot, "status", "--porcelain"); status != "" { + t.Fatalf("partial finalization left the original checkout dirty: %q", status) + } + + failSecond.Store(false) + second := env.startTaskRun( + t, + "parallel-rerun", + rawJSON(t, `{"run_id":"parallel-rerun","concurrent":2,"verification_command":"true"}`), + ) + secondRow := waitForRun(t, env.globalDB, second.RunID, func(row globaldb.Run) bool { + return row.Status == runStatusCompleted || row.Status == runStatusFailed + }) + if secondRow.Status != runStatusCompleted { + t.Fatalf("rerun status = %q, error = %q", secondRow.Status, secondRow.ErrorText) + } + if firstCalls.Load() != 1 || secondCalls.Load() != 2 { + t.Fatalf( + "task call counts = task_01:%d task_02:%d, want 1 and 2", + firstCalls.Load(), + secondCalls.Load(), + ) + } +} + +func TestRunManagerParallelFailureWithoutMergeDoesNotAdvanceOriginal(t *testing.T) { + env := newRunManagerTestEnv(t, runManagerTestDeps{ + prepare: func( + _ context.Context, + cfg *model.RuntimeConfig, + scope model.RunScope, + ) (*model.SolvePreparation, error) { + return &model.SolvePreparation{ + Jobs: []model.Job{{CodeFiles: append([]string(nil), cfg.TaskIDs...)}}, + RunArtifacts: scope.RunArtifacts(), + }, nil + }, + execute: func(context.Context, *model.SolvePreparation, *model.RuntimeConfig) error { + return errors.New("task failed intentionally") + }, + }) + initializeParallelDaemonWorkspace(t, env) + startingHEAD := runGitOutput(t, env.workspaceRoot, "rev-parse", "HEAD") + + run := env.startTaskRun( + t, + "parallel-no-progress", + rawJSON(t, `{"run_id":"parallel-no-progress","concurrent":2,"verification_command":"true"}`), + ) + row := waitForRun(t, env.globalDB, run.RunID, func(row globaldb.Run) bool { + return row.Status == runStatusCompleted || row.Status == runStatusFailed + }) + if row.Status != runStatusFailed { + t.Fatalf("parallel run status = %q, want failed", row.Status) + } + if currentHEAD := runGitOutput(t, env.workspaceRoot, "rev-parse", "HEAD"); currentHEAD != startingHEAD { + t.Fatalf("original HEAD changed from %s to %s without a merged task", startingHEAD, currentHEAD) + } + sharedMemoryPath := filepath.Join( + env.workspaceRoot, + ".productize", + "tasks", + env.workflowSlug, + "memory", + "MEMORY.md", + ) + if _, err := os.Stat(sharedMemoryPath); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("bootstrap-only shared memory reached original checkout: %v", err) + } +} + +func TestRunParallelIntegrationVerificationEnforcesCommandAndImmutableHEAD(t *testing.T) { + tests := []struct { + name string + command string + wantExitCode int + wantError string + }{ + {name: "success", command: "printf verified", wantExitCode: 0}, + {name: "failed command", command: "printf failed >&2; exit 7", wantExitCode: 7, wantError: "failed"}, + {name: "dirty checkout", command: "printf dirty > verifier.tmp", wantExitCode: 0, wantError: "modified"}, + { + name: "committed mutation", + command: "printf mutation > verifier.tmp; git add verifier.tmp; git commit -m verifier", + wantExitCode: 0, + wantError: "changed HEAD", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + active, lifecycle := parallelVerificationFixture(t) + record, err := runParallelIntegrationVerification( + active, + &model.RuntimeConfig{VerificationCommand: tt.command}, + lifecycle, + "verify-wave-001", + ) + if tt.wantError == "" { + if err != nil { + t.Fatalf("runParallelIntegrationVerification() error = %v", err) + } + if record.Status != "passed" { + t.Fatalf("verification status = %q, want passed", record.Status) + } + } else if err == nil || !strings.Contains(err.Error(), tt.wantError) { + t.Fatalf("verification error = %v, want substring %q", err, tt.wantError) + } + if record.ExitCode != tt.wantExitCode { + t.Fatalf("exit code = %d, want %d", record.ExitCode, tt.wantExitCode) + } + for _, path := range []string{record.StdoutPath, record.StderrPath} { + if _, statErr := os.Stat(path); statErr != nil { + t.Fatalf("verification log %q: %v", path, statErr) + } + } + }) + } +} + +func TestResolveParallelVerificationCommand(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + configured string + files map[string]string + want string + wantError bool + }{ + {name: "configured", configured: "npm run check", want: "npm run check"}, + {name: "make verify", files: map[string]string{"Makefile": "verify:\n\ttrue\n"}, want: "make verify"}, + { + name: "pnpm verify script", + files: map[string]string{ + "package.json": `{"scripts":{"verify":"vitest run"}}`, + "pnpm-lock.yaml": "lockfileVersion: '9.0'\n", + }, + want: "pnpm run verify", + }, + {name: "go module", files: map[string]string{"go.mod": "module example.com/test\n"}, want: "go test ./..."}, + { + name: "rust workspace", + files: map[string]string{"Cargo.toml": "[workspace]\n"}, + want: "cargo test --workspace", + }, + {name: "missing", wantError: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + root := t.TempDir() + for name, content := range tt.files { + if err := os.WriteFile(filepath.Join(root, name), []byte(content), 0o600); err != nil { + t.Fatalf("write %s: %v", name, err) + } + } + got, err := resolveParallelVerificationCommand(root, tt.configured) + if tt.wantError { + if err == nil { + t.Fatal("expected missing verification command error") + } + return + } + if err != nil || got != tt.want { + t.Fatalf("resolve command = %q, %v; want %q", got, err, tt.want) + } + }) + } +} + +func TestPrepareParallelTaskCoordinatorRejectsSharedAdditionalDirectories(t *testing.T) { + t.Parallel() + + artifacts := model.RunArtifacts{RunID: "parallel-add-dir", RunDir: t.TempDir()} + active := &activeRun{ + runID: "parallel-add-dir", + ctx: t.Context(), + scope: &model.BaseRunScope{Artifacts: artifacts}, + } + manager := &RunManager{} + row := globaldb.Run{RunID: active.runID} + cfg := &model.RuntimeConfig{ + Concurrent: 2, + TasksDir: t.TempDir(), + AddDirs: []string{"../shared"}, + } + if err := os.WriteFile( + filepath.Join(cfg.TasksDir, "task_01.md"), + []byte(parallelDaemonTask("task_01", nil)), + 0o600, + ); err != nil { + t.Fatalf("write task: %v", err) + } + + _, terminal := manager.prepareParallelTaskCoordinator(active, &row, cfg) + if terminal == nil || !strings.Contains(terminal.errorText, "do not support add_dirs") { + t.Fatalf("parallel add-dir terminal = %#v", terminal) + } +} + +func parallelVerificationFixture(t *testing.T) (*activeRun, *taskworktree.Lifecycle) { + t.Helper() + repo := t.TempDir() + runGitOutput(t, repo, "init", "-q", "-b", "main") + runGitOutput(t, repo, "config", "user.email", "parallel@example.com") + runGitOutput(t, repo, "config", "user.name", "Parallel Test") + runGitOutput(t, repo, "config", "commit.gpgsign", "false") + if err := os.WriteFile(filepath.Join(repo, "README.md"), []byte("initial\n"), 0o600); err != nil { + t.Fatalf("write README: %v", err) + } + runGitOutput(t, repo, "add", "README.md") + runGitOutput(t, repo, "commit", "--no-gpg-sign", "-m", "initial") + lifecycle, err := taskworktree.Preflight(t.Context(), taskworktree.Options{ + WorkspaceRoot: repo, + WorktreesRoot: filepath.Join(t.TempDir(), "worktrees"), + RunID: "parallel-test", + TaskIDs: []string{"task_01"}, + }) + if err != nil { + t.Fatalf("Preflight() error = %v", err) + } + if _, err := lifecycle.CreateIntegration(t.Context()); err != nil { + t.Fatalf("CreateIntegration() error = %v", err) + } + artifacts := model.RunArtifacts{RunID: "parallel-test", RunDir: t.TempDir()} + active := &activeRun{ + runID: "parallel-test", + ctx: t.Context(), + scope: &model.BaseRunScope{Artifacts: artifacts}, + } + return active, lifecycle +} + +func TestMakefileHasVerifyTarget(t *testing.T) { + t.Parallel() + if !makefileHasVerifyTarget(".PHONY: verify\nverify: test\n") { + t.Fatal("expected verify target") + } + if makefileHasVerifyTarget("verify-docs:\n") { + t.Fatal("did not expect verify-docs target to match") + } +} + +func TestParallelTaskFailuresSortsTaskIDs(t *testing.T) { + t.Parallel() + err := parallelTaskFailures(map[string]error{ + "task_10": errors.New("ten"), + "task_02": errors.New("two"), + }) + if got := err.Error(); strings.Index(got, "task_02") > strings.Index(got, "task_10") { + t.Fatalf("failure order is not stable: %s", got) + } +} + +func parallelDaemonTask(taskID string, dependencies []string) string { + lines := []string{ + "---", + "status: pending", + "title: " + taskID, + "type: backend", + "complexity: low", + "dependencies:", + } + if len(dependencies) == 0 { + lines[len(lines)-1] = "dependencies: []" + } else { + for _, dependency := range dependencies { + lines = append(lines, " - "+dependency) + } + } + lines = append(lines, "---", "", "# "+taskID, "") + return strings.Join(lines, "\n") +} + +func initializeParallelDaemonWorkspace(t *testing.T, env *runManagerTestEnv) { + t.Helper() + for _, taskID := range []string{"task_01", "task_02"} { + env.writeWorkflowFile(t, env.workflowSlug, taskID+".md", parallelDaemonTask(taskID, nil)) + } + runGitOutput(t, env.workspaceRoot, "init", "-q", "-b", "main") + runGitOutput(t, env.workspaceRoot, "config", "user.email", "parallel-daemon@example.com") + runGitOutput(t, env.workspaceRoot, "config", "user.name", "Parallel Daemon Test") + runGitOutput(t, env.workspaceRoot, "config", "commit.gpgsign", "false") + runGitOutput(t, env.workspaceRoot, "add", ".productize") + runGitOutput(t, env.workspaceRoot, "commit", "--no-gpg-sign", "-m", "tasks") +} diff --git a/internal/daemon/query_service.go b/internal/daemon/query_service.go index 3fa2cc45..6fdbf99e 100644 --- a/internal/daemon/query_service.go +++ b/internal/daemon/query_service.go @@ -998,7 +998,7 @@ func normalizeLaneStatus(status string) string { if trimmed == "" { return runStatusPending } - if trimmed == "canceled" { + if trimmed == runStatusCancelled { return runStatusCancelled } return trimmed @@ -1014,7 +1014,7 @@ func laneTitle(status string) string { return "Retrying" case runStatusCompleted, "done", "finished": return "Completed" - case "failed": + case runStatusFailed: return "Failed" case runStatusCancelled: return "Canceled" diff --git a/internal/daemon/run_manager.go b/internal/daemon/run_manager.go index 80cac4e6..921a66d4 100644 --- a/internal/daemon/run_manager.go +++ b/internal/daemon/run_manager.go @@ -156,6 +156,7 @@ type runtimeOverrideInput struct { RetryBackoffMultiplier *float64 `json:"retry_backoff_multiplier"` Concurrent *int `json:"concurrent"` BatchSize *int `json:"batch_size"` + VerificationCommand *string `json:"verification_command"` Verbose *bool `json:"verbose"` Persist *bool `json:"persist"` IncludeCompleted *bool `json:"include_completed"` @@ -1144,9 +1145,11 @@ func (m *RunManager) startRun(ctx context.Context, spec startRunSpec) (apicore.R } }() active := newActiveRun(runCtx, cancel, row, spec, scope) - if err := m.syncWorkflowBeforeRun(runCtx, active); err != nil { - active.cancel() - return apicore.Run{}, m.failStartRun(ctx, row, active.currentCloseTimeout(), scope, createdRun, err) + if !skipsPreRunWorkflowSync(runtimeCfg) { + if err := m.syncWorkflowBeforeRun(runCtx, active); err != nil { + active.cancel() + return apicore.Run{}, m.failStartRun(ctx, row, active.currentCloseTimeout(), scope, createdRun, err) + } } if err := m.startWatcher(active); err != nil { active.cancel() @@ -1462,6 +1465,11 @@ func (m *RunManager) runAsync(active *activeRun, row globaldb.Run, runtimeCfg *m } func (m *RunManager) executeWorkflowRun(active *activeRun, row globaldb.Run, runtimeCfg *model.RuntimeConfig) { + if isParallelTaskRuntime(runtimeCfg) { + m.executeParallelTaskRun(active, row, runtimeCfg) + return + } + scope := active.scope var ( executionErr error @@ -1523,6 +1531,14 @@ func (m *RunManager) executeWorkflowRun(active *activeRun, row globaldb.Run, run m.finishRun(active, row, fallback) } +func isParallelTaskRuntime(cfg *model.RuntimeConfig) bool { + return cfg != nil && cfg.Mode == model.ExecutionModePRDTasks && cfg.Concurrent > 1 +} + +func skipsPreRunWorkflowSync(cfg *model.RuntimeConfig) bool { + return isParallelTaskRuntime(cfg) || (cfg != nil && cfg.TaskMemoryLocalOnly) +} + func (m *RunManager) executeExecRun(active *activeRun, row globaldb.Run, runtimeCfg *model.RuntimeConfig) { scope := active.scope var fallback terminalState @@ -2489,6 +2505,12 @@ func applyTaskProjectConfig(cfg *model.RuntimeConfig, projectCfg workspacecfg.Ta return } applyOptionalOutputFormat(cfg, projectCfg.OutputFormat) + if projectCfg.Concurrent != nil { + cfg.Concurrent = *projectCfg.Concurrent + } + if projectCfg.VerifyCommand != nil { + cfg.VerificationCommand = *projectCfg.VerifyCommand + } if projectCfg.IncludeCompleted != nil { cfg.IncludeCompleted = *projectCfg.IncludeCompleted } @@ -2631,6 +2653,9 @@ func applyRuntimeOverrideWorkflowScalars(cfg *model.RuntimeConfig, overrides run if overrides.BatchSize != nil { cfg.BatchSize = *overrides.BatchSize } + if overrides.VerificationCommand != nil { + cfg.VerificationCommand = *overrides.VerificationCommand + } if overrides.IncludeCompleted != nil { cfg.IncludeCompleted = *overrides.IncludeCompleted } diff --git a/internal/daemon/run_manager_test.go b/internal/daemon/run_manager_test.go index 39c634c8..b1074c4e 100644 --- a/internal/daemon/run_manager_test.go +++ b/internal/daemon/run_manager_test.go @@ -1736,10 +1736,15 @@ func TestRunManagerHelperOverridesAndUtilities(t *testing.T) { t.Fatalf("applyRuntimeOverridesFromProject() error = %v", err) } applyTaskProjectConfig(cfg, workspacecfg.TaskRunConfig{ + Concurrent: intPtr(2), IncludeCompleted: boolPtr(true), OutputFormat: stringPtr(string(model.OutputFormatRawJSON)), TaskRuntimeRules: &rules, + VerifyCommand: stringPtr("make verify"), }) + if cfg.Concurrent != 2 || cfg.VerificationCommand != "make verify" { + t.Fatalf("task run project config was not applied: %#v", cfg) + } applyReviewProjectConfig(cfg, workspacecfg.FixReviewsConfig{ Concurrent: intPtr(4), BatchSize: intPtr(2), @@ -1776,6 +1781,7 @@ func TestRunManagerHelperOverridesAndUtilities(t *testing.T) { RetryBackoffMultiplier: floatPtr(3.0), Concurrent: intPtr(6), BatchSize: intPtr(7), + VerificationCommand: stringPtr("go test ./..."), Verbose: boolPtr(true), Persist: boolPtr(true), IncludeCompleted: boolPtr(false), @@ -1805,6 +1811,9 @@ func TestRunManagerHelperOverridesAndUtilities(t *testing.T) { if cfg.Concurrent != 8 || cfg.BatchSize != 9 || !cfg.IncludeResolved { t.Fatalf("review batching application failed: %#v", cfg) } + if cfg.VerificationCommand != "go test ./..." { + t.Fatalf("runtime verification command = %q, want go test ./...", cfg.VerificationCommand) + } if cfg.OutputFormat != model.OutputFormatText || cfg.Timeout != 4*time.Minute { t.Fatalf("runtime output/timeout = %q / %v, want text / 4m", cfg.OutputFormat, cfg.Timeout) } @@ -2115,6 +2124,30 @@ func TestRunManagerHelperEdgeCases(t *testing.T) { } } +func TestSkipsPreRunWorkflowSyncForParallelParentAndChild(t *testing.T) { + t.Parallel() + + if !skipsPreRunWorkflowSync(&model.RuntimeConfig{ + Mode: model.ExecutionModePRDTasks, + Concurrent: 2, + }) { + t.Fatal("parallel parent must skip pre-run workflow sync") + } + if !skipsPreRunWorkflowSync(&model.RuntimeConfig{ + Mode: model.ExecutionModePRDTasks, + Concurrent: 1, + TaskMemoryLocalOnly: true, + }) { + t.Fatal("isolated parallel child must skip pre-run workflow sync") + } + if skipsPreRunWorkflowSync(&model.RuntimeConfig{ + Mode: model.ExecutionModePRDTasks, + Concurrent: 1, + }) { + t.Fatal("sequential task run must retain pre-run workflow sync") + } +} + func TestRunManagerTaskRunWatcherSyncsTaskEditsAndStopsOnCancel(t *testing.T) { env := newRunManagerTestEnv(t, runManagerTestDeps{ watcherDebounce: 40 * time.Millisecond, diff --git a/openapi/productize-daemon.json b/openapi/productize-daemon.json index b847afe9..de05799c 100644 --- a/openapi/productize-daemon.json +++ b/openapi/productize-daemon.json @@ -1546,6 +1546,16 @@ }, "runtime_overrides": { "additionalProperties": true, + "properties": { + "concurrent": { + "minimum": 1, + "type": "integer" + }, + "verification_command": { + "minLength": 1, + "type": "string" + } + }, "type": "object" }, "workspace": { diff --git a/pkg/productize/events/docs_test.go b/pkg/productize/events/docs_test.go index 7749d6d1..107cbe1f 100644 --- a/pkg/productize/events/docs_test.go +++ b/pkg/productize/events/docs_test.go @@ -41,6 +41,7 @@ func TestEventsDocumentationEnumeratesAllPublicKinds(t *testing.T) { EventKindTaskFileSkipped, EventKindTaskMetadataRefreshed, EventKindTaskMemoryUpdated, + EventKindTaskSchedulerUpdated, EventKindArtifactUpdated, EventKindExtensionLoaded, EventKindExtensionReady, diff --git a/pkg/productize/events/event.go b/pkg/productize/events/event.go index 25259b45..420f0634 100644 --- a/pkg/productize/events/event.go +++ b/pkg/productize/events/event.go @@ -63,6 +63,7 @@ const ( EventKindTaskFileSkipped EventKind = "task.file_skipped" EventKindTaskMetadataRefreshed EventKind = "task.metadata_refreshed" EventKindTaskMemoryUpdated EventKind = "task.memory_updated" + EventKindTaskSchedulerUpdated EventKind = "task.scheduler_updated" // Artifact and extension events. EventKindArtifactUpdated EventKind = "artifact.updated" diff --git a/pkg/productize/events/kinds/task.go b/pkg/productize/events/kinds/task.go index 291b23f7..784be97e 100644 --- a/pkg/productize/events/kinds/task.go +++ b/pkg/productize/events/kinds/task.go @@ -41,3 +41,45 @@ type TaskMetadataRefreshedPayload struct { Completed int `json:"completed,omitempty"` Pending int `json:"pending,omitempty"` } + +// TaskSchedulerTaskState describes one task in a parallel scheduler checkpoint. +type TaskSchedulerTaskState struct { + TaskID string `json:"task_id"` + Dependencies []string `json:"dependencies,omitempty"` + Status string `json:"status"` + RunID string `json:"run_id,omitempty"` + Branch string `json:"branch,omitempty"` + WorktreePath string `json:"worktree_path,omitempty"` + Commit string `json:"commit,omitempty"` + Error string `json:"error,omitempty"` +} + +// TaskSchedulerVerification describes one enforced integration verification. +type TaskSchedulerVerification struct { + ID string `json:"id"` + Command string `json:"command"` + Status string `json:"status"` + ExitCode int `json:"exit_code"` + Commit string `json:"commit"` + StdoutPath string `json:"stdout_path"` + StderrPath string `json:"stderr_path"` + Error string `json:"error,omitempty"` +} + +// TaskSchedulerUpdatedPayload is a complete durable checkpoint for one +// dependency-aware parallel task scheduler run. +type TaskSchedulerUpdatedPayload struct { + ManifestPath string `json:"manifest_path"` + Status string `json:"status"` + Concurrent int `json:"concurrent"` + StartingBranch string `json:"starting_branch"` + StartingCommit string `json:"starting_commit"` + IntegrationBranch string `json:"integration_branch"` + IntegrationPath string `json:"integration_path"` + Waves [][]string `json:"waves"` + Tasks []TaskSchedulerTaskState `json:"tasks"` + Verifications []TaskSchedulerVerification `json:"verifications,omitempty"` + Finalized bool `json:"finalized"` + FinalCommit string `json:"final_commit,omitempty"` + NextActions []string `json:"next_actions,omitempty"` +} diff --git a/sdk/extension-sdk-ts/src/types.ts b/sdk/extension-sdk-ts/src/types.ts index a57428c5..649670b5 100644 --- a/sdk/extension-sdk-ts/src/types.ts +++ b/sdk/extension-sdk-ts/src/types.ts @@ -395,8 +395,10 @@ export interface WorkflowMemoryContext { directory?: string; workflow_path?: string; task_path?: string; + dependency_paths?: string[]; workflow_needs_compaction?: boolean; task_needs_compaction?: boolean; + task_local_only?: boolean; } /** Mirrors the prompt build input snapshot exposed to prompt hooks. */ @@ -491,6 +493,9 @@ export interface RuntimeConfig { auto_commit?: boolean; concurrent?: number; batch_size?: number; + verification_command?: string; + task_ids?: string[]; + task_memory_local_only?: boolean; ide?: string; model?: string; add_dirs?: string[]; @@ -997,6 +1002,7 @@ export interface RunConfig { auto_commit?: boolean; concurrent?: number; batch_size?: number; + verification_command?: string; ide?: string; model?: string; add_dirs?: string[]; diff --git a/sdk/extension/host_api.go b/sdk/extension/host_api.go index c1f80ca6..90b39e0c 100644 --- a/sdk/extension/host_api.go +++ b/sdk/extension/host_api.go @@ -131,6 +131,7 @@ type RunConfig struct { AutoCommit bool `json:"auto_commit,omitempty"` Concurrent int `json:"concurrent,omitempty"` BatchSize int `json:"batch_size,omitempty"` + VerificationCommand string `json:"verification_command,omitempty"` IDE string `json:"ide,omitempty"` Model string `json:"model,omitempty"` AddDirs []string `json:"add_dirs,omitempty"` diff --git a/sdk/extension/types.go b/sdk/extension/types.go index 5d9b7d5c..a2e75b08 100644 --- a/sdk/extension/types.go +++ b/sdk/extension/types.go @@ -279,11 +279,13 @@ type IssueEntry struct { // WorkflowMemoryContext describes the current workflow memory documents. type WorkflowMemoryContext struct { - Directory string `json:"directory,omitempty"` - WorkflowPath string `json:"workflow_path,omitempty"` - TaskPath string `json:"task_path,omitempty"` - WorkflowNeedsCompaction bool `json:"workflow_needs_compaction,omitempty"` - TaskNeedsCompaction bool `json:"task_needs_compaction,omitempty"` + Directory string `json:"directory,omitempty"` + WorkflowPath string `json:"workflow_path,omitempty"` + TaskPath string `json:"task_path,omitempty"` + DependencyPaths []string `json:"dependency_paths,omitempty"` + WorkflowNeedsCompaction bool `json:"workflow_needs_compaction,omitempty"` + TaskNeedsCompaction bool `json:"task_needs_compaction,omitempty"` + TaskLocalOnly bool `json:"task_local_only,omitempty"` } // BatchParams mirrors the prompt build input snapshot exposed to prompt hooks. @@ -520,6 +522,7 @@ type RuntimeConfig struct { AutoCommit bool Concurrent int BatchSize int + VerificationCommand string IDE string Model string AddDirs []string @@ -529,6 +532,8 @@ type RuntimeConfig struct { AgentName string ExplicitRuntime ExplicitRuntimeFlags TaskRuntimeRules []TaskRuntimeRule + TaskIDs []string + TaskMemoryLocalOnly bool Mode ExecutionMode OutputFormat OutputFormat Verbose bool diff --git a/skills/execute-task/SKILL.md b/skills/execute-task/SKILL.md index dfadd111..eb6d3a2f 100644 --- a/skills/execute-task/SKILL.md +++ b/skills/execute-task/SKILL.md @@ -27,6 +27,8 @@ Execute one PRD task from exploration through tracking updates. - Read ADRs from the `adrs/` subdirectory of the PRD directory to understand the architectural decision context for this task. - After reading all sources, check for conflicts between the task specification, techspec, and ADRs. If any requirements contradict each other, stop and report the conflict instead of guessing — do not proceed to step 2. - If the caller provides workflow memory paths, use the installed `workflow-memory` skill before editing code. + - If the caller marks the task as isolated or task-local-only for parallel + execution, read shared workflow memory but write only current task memory. - Reconcile the current workspace state before new edits. 2. Build the execution checklist. diff --git a/skills/productize-runtime/SKILL.md b/skills/productize-runtime/SKILL.md index d572c567..ced37069 100644 --- a/skills/productize-runtime/SKILL.md +++ b/skills/productize-runtime/SKILL.md @@ -29,7 +29,7 @@ The standard development pipeline follows these phases in order. Each phase prod 4. **Requirements** -- `/create-prd` creates a business-focused Product Requirements Document at `.productize/tasks//_prd.md` with ADRs. 5. **Technical Design** -- `/create-techspec` translates the PRD into a technical specification at `.productize/tasks//_techspec.md` with ADRs. 6. **Task Decomposition** -- `/create-tasks` breaks down the PRD and TechSpec into independently implementable task files (`task_01.md`, `task_02.md`, etc.) and a master list at `_tasks.md`. -7. **Execution** -- `productize tasks run --ide ` dispatches task files sequentially to the configured AI agent for implementation. +7. **Execution** -- `productize tasks run --ide ` dispatches task files to the configured AI agent. Execution is sequential by default; `--concurrent N` enables bounded parallel execution of dependency-ready tasks. 8. **Review** -- `/review-round` (manual AI review) or `productize reviews fetch --provider coderabbit --pr ` (external provider) produces review issue files under `reviews-NNN/`. 9. **Remediation** -- `productize reviews fix ` processes review issues, triages, fixes, and verifies each one. 10. **Archive** -- `productize archive --name ` moves fully completed workflows to `.productize/tasks/_archived/`, then refreshes project knowledge. @@ -76,7 +76,7 @@ For a detailed step-by-step walkthrough of each phase, read `references/workflow | **Workflow Execution** | | | | `productize daemon` | Manage the home-scoped daemon lifecycle | `start`, `status`, `stop` | | `productize workspaces` | Inspect and manage daemon workspace registrations | `list`, `show`, `register`, `unregister`, `resolve` | -| `productize tasks run` | Execute PRD task files through the daemon | `--name`, `--attach`, `--stream`, `--detach`, `--task-runtime` | +| `productize tasks run` | Execute PRD task files through the daemon | `--name`, `--concurrent`, `--verify-command`, `--format`, `--attach`, `--stream`, `--detach`, `--task-runtime` | | `productize exec` | Execute an ad hoc prompt | `--agent`, `--format`, `--prompt-file`, `--persist`, `--run-id` | | `productize runs` | Attach, watch, and purge daemon-managed runs | `attach`, `watch`, `purge` | | **Review** | | | @@ -222,7 +222,9 @@ add_dirs = ["../shared-lib"] types = ["frontend", "backend", "docs", "test", "infra", "refactor", "chore", "bugfix"] [tasks.run] +concurrent = 1 include_completed = false +verify_command = "make verify" [fix_reviews] concurrent = 2 diff --git a/skills/productize-runtime/references/cli-reference.md b/skills/productize-runtime/references/cli-reference.md index 64230409..59b68466 100644 --- a/skills/productize-runtime/references/cli-reference.md +++ b/skills/productize-runtime/references/cli-reference.md @@ -122,26 +122,43 @@ Update the Productize CLI to the latest release. No flags. ### `productize tasks run` -Execute PRD task files sequentially from a workflow directory through the shared daemon. +Execute PRD task files from a workflow directory through the shared daemon. +The default concurrency of one preserves sequential execution. | Flag | Type | Default | Description | | --- | --- | --- | --- | | `--name` | string | | Task workflow name (resolves to `.productize/tasks/`) | +| `--concurrent` | int | 1 | Maximum dependency-ready tasks to execute concurrently; must be positive | | `--include-completed` | bool | false | Include tasks already marked as completed | +| `--format` | string | text | Output format: text, json, raw-json | | `--skip-validation` | bool | false | Skip task metadata preflight check | | `--force` | bool | false | Continue after validation fails in non-interactive mode | | `--attach` | string | auto | Attach mode: auto, stream, detach | | `--stream` | bool | false | Force textual stream attach mode | | `--detach` | bool | false | Start the run without attaching a client | | `--task-runtime` | string[] | | Per-task runtime override rules | +| `--verify-command` | string | | Shell command used to verify integrated parallel task changes | | + common flags | | | `--ide`, `--model`, `--reasoning-effort`, `--add-dir`, `--auto-commit`, `--dry-run` | +Text mode uses the human-readable workflow watcher. JSON streams lean JSONL +events, and raw JSON streams canonical event envelopes. For local parallel +dry-run, both machine-readable formats emit one schema-versioned plan object. + ``` productize tasks run multi-repo --ide claude +productize tasks run multi-repo --concurrent 3 --verify-command "make verify" productize tasks run --name multi-repo --ide codex --auto-commit productize tasks run multi-repo --stream ``` +Task batch size remains fixed at one. `--detach` affects client presentation +only; it does not change scheduling or isolation. Parallel execution requires a +verification command unless Productize discovers a Make `verify` target, Node +`verify`/`test` script, Go module, or Rust workspace. Verification is enforced +after every merged wave and before finalization. Parallel dry-run is local and +read-only and does not start the daemon or create Git resources. Parallel task +runs reject `--add-dir` because external directories are not worktree-isolated. + ### `productize exec [prompt]` Execute a single ad hoc prompt through the ACP runtime. Provide prompt as argument, via `--prompt-file`, or stdin. diff --git a/skills/productize-runtime/references/config-reference.md b/skills/productize-runtime/references/config-reference.md index 62fa70cd..26456421 100644 --- a/skills/productize-runtime/references/config-reference.md +++ b/skills/productize-runtime/references/config-reference.md @@ -32,8 +32,16 @@ Options specific to `productize tasks run`. | Field | Type | Description | | --- | --- | --- | +| `concurrent` | int | Maximum dependency-ready tasks to execute concurrently; default `1`, must be positive | | `include_completed` | bool | Include tasks already marked as completed | +| `output_format` | string | Output format: `text`, `json`, `raw-json` | | `task_runtime_rules` | `array` | Type-scoped runtime overrides applied after `[defaults]` for `productize tasks run` | +| `verify_command` | string | Optional shell command for integrated parallel-run verification | + +Task batch size is fixed at one. Parallel execution uses `verify_command` when +Productize cannot discover a Make `verify` target, Node `verify`/`test` script, +Go module, or Rust workspace. The command is executed directly after every +merged wave and before finalization. #### `[[tasks.run.task_runtime_rules]]` @@ -241,7 +249,9 @@ retry_backoff_multiplier = 1.5 types = ["frontend", "backend", "docs", "test", "infra", "refactor", "chore", "bugfix"] [tasks.run] +concurrent = 1 include_completed = false +verify_command = "make verify" [fix_reviews] concurrent = 2 diff --git a/skills/productize-runtime/references/workflow-guide.md b/skills/productize-runtime/references/workflow-guide.md index 0437a57b..b01ad769 100644 --- a/skills/productize-runtime/references/workflow-guide.md +++ b/skills/productize-runtime/references/workflow-guide.md @@ -107,11 +107,20 @@ Install flow: `productize ext install --yes itseffi/productize --remote github - 6. Workflow memory is maintained across tasks via `workflow-memory`. **Key flags:** +- `--concurrent N` -- execute up to `N` dependency-ready tasks concurrently; defaults to one. - `--auto-commit` -- create a local commit after each task completes cleanly. - `--dry-run` -- generate prompts without running the IDE tool. - `--include-completed` -- re-process tasks already marked as completed. +- `--verify-command "make verify"` -- set the integrated-run verification command when no standard target is discoverable. -**Attach mode:** In interactive terminals, `tasks run` streams textual run observation by default; use `--stream`, `--detach`, or `--attach` to control whether the command follows the run or returns immediately. +**Attach mode:** In interactive terminals, `tasks run` streams textual run observation by default; use `--stream`, `--detach`, or `--attach` to control whether the command follows the run or returns immediately. Attach mode is independent of concurrency. + +**Parallel safety:** Concurrency greater than one requires a clean Git-root +checkout and uses one Productize-owned worktree per ready task. Successful task +branches merge in dependency order, shared memory is rebuilt between waves, and +verification is enforced after each wave and before the final fast-forward. +Failed worktrees are retained, while verified completed tasks are finalized so +reruns skip them. Parallel dry-run performs read-only local planning. ## Phase 6: Review diff --git a/skills/workflow-memory/SKILL.md b/skills/workflow-memory/SKILL.md index 698e9635..b1e7985b 100644 --- a/skills/workflow-memory/SKILL.md +++ b/skills/workflow-memory/SKILL.md @@ -16,6 +16,11 @@ Maintain the workflow memory files provided by the caller. ## Workflow +If the caller marks the task as isolated or task-local-only for parallel +execution, read the shared workflow memory as required context but never modify +it. Write all new facts only to the current task memory file. The Productize +coordinator owns shared-memory reconstruction after task branches are merged. + 1. Load the memory state before editing code. - Read the shared workflow memory file and the current task memory file before making any code change. - Treat these files as mandatory context for the run, not optional notes. @@ -38,6 +43,9 @@ Maintain the workflow memory files provided by the caller. - Do not duplicate facts that are obvious from the repository, git diff, task file, or PRD documents. - Do not read unrelated task memory files unless the shared workflow memory or the caller explicitly points to them. - Keep shared memory durable and cross-task. Keep task memory local and operational. +- In isolated parallel execution, never write shared workflow memory even when + a durable fact would normally qualify for promotion; record it in task memory + for coordinator promotion after integration. ## Promotion Decision Test