Skip to content

feat(workflows): add dynamic until loops#36

Merged
jgmontoya merged 3 commits into
masterfrom
feat/workflow-until
Jul 10, 2026
Merged

feat(workflows): add dynamic until loops#36
jgmontoya merged 3 commits into
masterfrom
feat/workflow-until

Conversation

@jgmontoya

@jgmontoya jgmontoya commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • New Features
    • Added workflow “until” conditions to stop early based on run success or prompt verdict satisfaction.
    • Run progress and final summaries now show when satisfaction occurred and stop once met (including group-level stopping).
    • Review-and-fix workflow now supports iterative refinement until no issues remain.
  • Bug Fixes
    • Until/step outputs now preserve stderr when present.
  • Tests / Documentation
    • Expanded end-to-end and unit coverage for until (verdict parsing, artifacts, metadata) and updated workflow documentation/examples.

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds workflow until support across schema, parsing, execution, CLI output, docs, and tests, while also updating command resolution to use the current PATH and bumping the project to version 0.13.0.

Changes

Until condition feature

Layer / File(s) Summary
Domain schema for until
src/domain/workflow.ts
Adds UntilStep and extends Workflow, GroupStep, and RunMetadata with until and satisfiedAt.
Workflow discovery parsing
src/providers/workflow-discovery.ts
Parses and validates workflow, group, leaf, and reference-step until definitions and propagates them through resolution.
Discovery validation tests
test/unit/providers/workflow-discovery.test.ts
Covers valid and invalid until parsing across workflow, group, leaf, and workflow-reference cases.
Until execution in workflow runner
src/services/workflow-runner.ts
Adds until verdict parsing, iteration-level evaluation, group early-stop handling, merged stderr-aware output, and satisfiedAt persistence.
Runner and CLI tests
src/commands/run.ts, test/unit/commands/run.test.ts, test/unit/services/workflow-runner.test.ts
Updates run-status logging and adds unit coverage for until progress, verdict parsing, and stop conditions.
Workflow example and e2e coverage
defaults/system/workflows/review-and-fix.yaml, test/e2e/lib/workflow-until.sh, test/e2e/claudecode.sh, test/e2e/codex.sh, test/e2e/opencode.sh, test/e2e/pi.sh
Updates the built-in review workflow and adds shared/provider-specific e2e checks for until stopping behavior.

Executable resolution

Layer / File(s) Summary
Path-based command resolution
src/platform/paths.ts, src/providers/agent-runner.ts, src/services/provider-detection.ts
Resolves provider executables against the current PATH before execution and during detection.
Command assertion updates
test/unit/services/setup-session.test.ts
Adjusts setup-session tests to assert the resolved executable name rather than the raw command string.

Release version update

Layer / File(s) Summary
Versioned release metadata
CHANGELOG.md, defaults/config.json, package.json, test/unit/services/init-service.test.ts, test/unit/domain/version.test.ts
Bumps the project version to 0.13.0 and updates changelog and version expectations.
Workflow docs
README.md
Adds documentation for dynamic until stopping and updates the review-and-fix workflow example.
Version smoke tests
test/unit/hooks/format-reminder.test.ts
Adjusts platform-skipped reminder assertions already affected by the versioned release changes.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CreateRunCommand
  participant WorkflowRunner
  participant ExecuteSteps
  participant EvaluateUntil
  participant RunJson

  CreateRunCommand->>WorkflowRunner: onUntilStart / onUntilResult callbacks
  WorkflowRunner->>ExecuteSteps: executeSteps(workflow.until)
  loop each iteration
    ExecuteSteps->>ExecuteSteps: run leaf and group steps
    ExecuteSteps->>EvaluateUntil: evaluate until condition
    EvaluateUntil-->>ExecuteSteps: satisfied or continue
  end
  ExecuteSteps->>RunJson: persist satisfiedAt
  WorkflowRunner-->>CreateRunCommand: printSummary
Loading

Possibly related PRs

  • jgmontoya/shaka#13: Extends the same workflow runner and run command paths with the new until-loop control flow.
  • jgmontoya/shaka#18: Builds on the same loop-based workflow execution and satisfiedAt metadata used by this until feature.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding dynamic until loops to workflows.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/workflow-until

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/providers/workflow-discovery.ts`:
- Around line 376-379: The `!typeKey` guard in `getUntil` is unreachable because
`typeKeys` has already been validated to contain exactly one entry earlier in
the same flow. Remove the dead `if (!typeKey)` branch and keep the existing
`typeKeys[0]` selection and return path, using the `workflow-discovery` logic
around `typeKeys` and `typeKey` as the place to update.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: eb2b31bc-ec78-4035-87a9-303daf431dc8

📥 Commits

Reviewing files that changed from the base of the PR and between aa7aaec and a28d4eb.

📒 Files selected for processing (13)
  • defaults/system/workflows/review-and-fix.yaml
  • src/commands/run.ts
  • src/domain/workflow.ts
  • src/providers/workflow-discovery.ts
  • src/services/workflow-runner.ts
  • test/e2e/claudecode.sh
  • test/e2e/codex.sh
  • test/e2e/lib/workflow-until.sh
  • test/e2e/opencode.sh
  • test/e2e/pi.sh
  • test/unit/commands/run.test.ts
  • test/unit/providers/workflow-discovery.test.ts
  • test/unit/services/workflow-runner.test.ts

Comment thread src/providers/workflow-discovery.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 14: Tighten the wording in the changelog sentence by replacing the phrase
“by accident” with a more concise alternative such as “unintentionally” or
“accidentally” in the note about `until: { prompt: ... }`, keeping the rest of
the explanation intact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 95ba64f9-f953-41cf-8e4d-f408eac26d30

📥 Commits

Reviewing files that changed from the base of the PR and between a28d4eb and a800189.

📒 Files selected for processing (12)
  • CHANGELOG.md
  • README.md
  • defaults/config.json
  • package.json
  • src/providers/agent-runner.ts
  • src/providers/workflow-discovery.ts
  • src/services/provider-detection.ts
  • test/unit/domain/version.test.ts
  • test/unit/hooks/format-reminder.test.ts
  • test/unit/services/init-service.test.ts
  • test/unit/services/setup-session.test.ts
  • test/unit/services/workflow-runner.test.ts

Comment thread CHANGELOG.md
@jgmontoya
jgmontoya force-pushed the feat/workflow-until branch from a800189 to 17d5307 Compare July 8, 2026 19:36

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/providers/agent-runner.ts`:
- Around line 11-19: The PATH lookup logic is duplicated between agent execution
and provider detection, so extract it into a shared helper and reuse it in both
places. Move the existing whichOnCurrentPath behavior from provider-detection.ts
into a common utility such as resolveOnCurrentPath or a shared which module,
then have agent-runner.ts use that helper instead of calling Bun.which inline so
both code paths stay in sync.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: f84c6349-e909-46fa-aa20-ecb1ece9815e

📥 Commits

Reviewing files that changed from the base of the PR and between a800189 and 17d5307.

📒 Files selected for processing (9)
  • src/providers/agent-runner.ts
  • src/providers/workflow-discovery.ts
  • src/services/provider-detection.ts
  • test/unit/hooks/format-reminder.test.ts
  • test/unit/platform/process-runner.test.ts
  • test/unit/services/autoresearch.test.ts
  • test/unit/services/setup-session.test.ts
  • test/unit/services/uninstall-service.test.ts
  • test/unit/services/workflow-runner.test.ts

Comment thread src/providers/agent-runner.ts Outdated
@jgmontoya
jgmontoya force-pushed the feat/workflow-until branch from 17d5307 to fc92625 Compare July 9, 2026 21:49

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/unit/services/workflow-runner.test.ts (1)

593-614: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Skip POSIX-only until tests on Windows.

Three until integration tests use POSIX shell commands (test -f, test "..." = "...", printf) but are not wrapped in test.skipIf(process.platform === "win32"), unlike the adjacent tests at lines 511, 694, 725, 756, and 1031. On Windows (cmd.exe), these commands don't exist, causing the until condition to fail with a non-zero exit code — the workflow never satisfies, and assertions on satisfiedAt / callback events will fail.

🔧 Proposed fix: wrap POSIX-only tests with skipIf
- test("until condition resolves current iteration step templates", async () => {
+ test.skipIf(process.platform === "win32")(
+   "until condition resolves current iteration step templates",
+   async () => {
    // ... test body unchanged ...
- });
+ },
+ );
- test("workflow loop 1 with until evaluates once and records satisfaction", async () => {
+ test.skipIf(process.platform === "win32")(
+   "workflow loop 1 with until evaluates once and records satisfaction",
+   async () => {
    // ... test body unchanged ...
- });
+ },
+ );
- test("until progress callbacks report start and result", async () => {
+ test.skipIf(process.platform === "win32")(
+   "until progress callbacks report start and result",
+   async () => {
    // ... test body unchanged ...
- });
+ },
+ );

Also applies to: 641-663, 665-692

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/unit/services/workflow-runner.test.ts` around lines 593 - 614, The
POSIX-only `until` test in `workflow-runner.test.ts` uses shell commands that
fail on Windows, so wrap this `until` case in `test.skipIf(process.platform ===
"win32")` like the adjacent platform-gated tests. Apply the same guard to the
other affected `until` tests in the file that use `run` commands such as `test
-f`, `test "..." = "..."`, and `printf`, so `runWorkflow` assertions like
`metadata.satisfiedAt` only run where the commands are supported.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@test/unit/services/workflow-runner.test.ts`:
- Around line 593-614: The POSIX-only `until` test in `workflow-runner.test.ts`
uses shell commands that fail on Windows, so wrap this `until` case in
`test.skipIf(process.platform === "win32")` like the adjacent platform-gated
tests. Apply the same guard to the other affected `until` tests in the file that
use `run` commands such as `test -f`, `test "..." = "..."`, and `printf`, so
`runWorkflow` assertions like `metadata.satisfiedAt` only run where the commands
are supported.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 10fbfffc-e37d-4a7d-8361-955608822b27

📥 Commits

Reviewing files that changed from the base of the PR and between 17d5307 and fc92625.

📒 Files selected for processing (11)
  • src/platform/paths.ts
  • src/providers/agent-runner.ts
  • src/providers/workflow-discovery.ts
  • src/services/provider-detection.ts
  • test/unit/hooks/format-reminder.test.ts
  • test/unit/platform/paths.test.ts
  • test/unit/platform/process-runner.test.ts
  • test/unit/services/autoresearch.test.ts
  • test/unit/services/setup-session.test.ts
  • test/unit/services/uninstall-service.test.ts
  • test/unit/services/workflow-runner.test.ts

@jgmontoya
jgmontoya merged commit 68a3d21 into master Jul 10, 2026
3 checks passed
@jgmontoya
jgmontoya deleted the feat/workflow-until branch July 10, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant