Skip to content

Feat/interactive wizard#228

Open
iamyhe wants to merge 5 commits into
TestSprite:mainfrom
iamyhe:feat/interactive-wizard
Open

Feat/interactive wizard#228
iamyhe wants to merge 5 commits into
TestSprite:mainfrom
iamyhe:feat/interactive-wizard

Conversation

@iamyhe

@iamyhe iamyhe commented Jul 9, 2026

Copy link
Copy Markdown

What does this PR do?

This PR enhances the CLI user experience by introducing an interactive wizard. If a user runs a test command without specifying the required --plan-from path, instead of throwing a hard error, the CLI now gracefully prompts the user to enter the path using Node's native readline.

🖥️ CLI Experience (Before vs. After)

❌ Before:

$ testsprite test run
Error: Missing required argument: plan-from

✅ After:

$ testsprite test run
Enter path to plan JSON file [default: plan.json]: _

Key Highlights:

  • Built using native modules (Zero external dependencies added).
  • Checks for TTY and CI environments.
  • Gated on --dry-run and --output json to ensure automated pipelines remain unaffected.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behavior)

Summary by CodeRabbit

  • New Features

    • Added an interactive prompt for the test plan path when it’s missing in supported terminal sessions.
    • Test creation and run results in JSON mode now include dashboard links where available.
  • Bug Fixes

    • Improved messaging and guidance across test wait, conflict recovery, failure, artifact, rerun, flaky, and help flows.
    • Better handling of failed-only dry runs and clearer exit-code/help text.
  • Chores

    • Update-check errors now provide extra debug details when running with verbose or debug output.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Changes to src/commands/test.ts add an interactive plan-path prompt (via new src/lib/prompt.ts helper promptForPlanPath), enrich batch create/run JSON results with dashboardUrl, fix a bundle-file prediction predicate, and reword numerous advisory/help messages. src/lib/update-check.ts adds argv-based debug logging for cache failures.

Changes

Test CLI UX changes

Layer / File(s) Summary
Interactive plan-path prompting
src/lib/prompt.ts, src/commands/test.ts
New exported promptForPlanPath prompts for a plan path when TTY/non-CI and no --plan-from/dry-run/JSON output; wired into runCreateFromPlan, with updated ignored-flags advisory text.
Dashboard URL enrichment
src/commands/test.ts
runCreateBatch and runBatchRun JSON results are enriched with dashboardUrl resolved from projectId/testId when not dry-running.
Failed-only bundle file prediction fix
src/commands/test.ts
Restructures the inline predicate in plannedBundleFiles that selects steps around a failed step index for --failed-only dry-run previews.
Validation and advisory message wording
src/commands/test.ts
Rewords error/warning/advisory strings across create, delete-batch, steps, result, run, wait, rerun, and flaky command flows.
CLI help text formatting
src/commands/test.ts
Reformats help text and exit-code documentation for create, scaffold, result, delete-batch, run, wait, rerun, flaky, code put, and plan put commands.

Update-check debug logging

Layer / File(s) Summary
Debug-gated cache logging
src/lib/update-check.ts
Adds optional argv to UpdateCheckDeps, new isDebugLoggingEnabled helper, debug-gated stderr logging in cache read/write catch blocks, and reformats module docs and the update advisory string.

Estimated code review effort: 2 (Simple) | ~15 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant runCreateFromPlan
  participant promptForPlanPath
  participant promptText

  User->>runCreateFromPlan: test create (no --plan-from, non-JSON, non-dry-run)
  runCreateFromPlan->>promptForPlanPath: request plan path
  promptForPlanPath->>promptForPlanPath: check TTY status and CI env var
  promptForPlanPath->>promptText: prompt user for input
  promptText-->>promptForPlanPath: raw answer
  promptForPlanPath-->>runCreateFromPlan: trimmed path or fallback ("plan.json")
  runCreateFromPlan->>runCreateFromPlan: requireNonEmpty('plan-from', ...)
Loading

Possibly related PRs

Suggested reviewers: ruili-testsprite, zeshi-du

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title is related to the PR, but "interactive wizard" is too vague to clearly describe the main change. Use a more specific title like "Add interactive prompt for missing test plan path" to reflect the primary change.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (4)
src/lib/update-check.ts (2)

137-140: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Duplicated debug-log-on-catch pattern.

The isDebugLoggingEnabled(...) → resolved.stderr(...) block is repeated verbatim (only the prefix string differs) in both readUpdateCheckCache and writeUpdateCheckCache. Consider extracting a small helper (e.g., logDebug(resolved, label, err)) to avoid the duplication as more call sites are added.

♻️ Suggested helper
+function logDebugError(resolved: ResolvedUpdateCheckDeps, label: string, err: unknown): void {
+  if (isDebugLoggingEnabled(resolved.argv)) {
+    resolved.stderr(`[debug] ${label}: ${err instanceof Error ? err.message : String(err)}`);
+  }
+}

Then in each catch block:

-  } catch (err) {
-    if (isDebugLoggingEnabled(resolved.argv)) {
-      resolved.stderr(`[debug] readUpdateCheckCache: ${err instanceof Error ? err.message : String(err)}`);
-    }
+  } catch (err) {
+    logDebugError(resolved, 'readUpdateCheckCache', err);

Also applies to: 155-158

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/update-check.ts` around lines 137 - 140, The same debug-on-catch
logging pattern is duplicated in readUpdateCheckCache and writeUpdateCheckCache.
Extract a small helper near these functions, such as logDebug(resolved, label,
err), that checks isDebugLoggingEnabled(resolved.argv) and writes the formatted
message to resolved.stderr. Then replace both catch blocks to call the helper
with the appropriate label so the prefix stays specific while the shared logic
is centralized.

96-96: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Missing doc comment on new argv field.

Every other field in UpdateCheckDeps has a one-line JSDoc comment describing its purpose (e.g., Line 92 /** Sink for the single advisory line. */); argv lacks one, which is a minor inconsistency for an otherwise well-documented public interface.

📝 Suggested addition
+  /** Process arguments, used to detect --debug/--verbose for diagnostic logging. */
   argv?: string[];
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/update-check.ts` at line 96, Add a one-line JSDoc comment for the new
argv field in UpdateCheckDeps to match the documentation style used by the other
fields in update-check.ts. Locate the interface containing the existing
documented members like sink and other deps, and add a concise description of
what argv represents so the public API stays consistently documented.
src/commands/test.ts (1)

1986-1992: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

New interactive branch isn't wired through deps for testability.

Every other side-effecting call in this file (stderr, sleep, client) goes through the deps injection pattern, but promptForPlanPath() here hits process.stdin/process.stderr directly with no way for a caller/test to supply the PromptStreams it already accepts. As written, this branch has no path to deterministic unit-test coverage (short of mocking process.stdin.isTTY globally), and the duplicate-flags test cited in context uses output: 'json' specifically to avoid it.

🔧 Suggested direction
-  if (!opts.planFrom && !opts.dryRun && opts.output !== 'json') {
-    const interactivePath = await promptForPlanPath();
+  if (!opts.planFrom && !opts.dryRun && opts.output !== 'json') {
+    const interactivePath = await promptForPlanPath(undefined, undefined, {
+      input: deps.stdin,
+      output: deps.stderrStream,
+    });

(requires adding optional stdin/stderrStream fields to TestDeps)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/test.ts` around lines 1986 - 1992, The new interactive plan-path
branch in test command handling bypasses the existing deps injection pattern,
making it hard to test deterministically. Update the test flow around the
`promptForPlanPath()` call to source `stdin`/`stderrStream` from `TestDeps`
(adding those optional fields if needed) and pass them into `promptForPlanPath`
so this side effect is fully injectable like the other dependencies in
`src/commands/test.ts`.
src/lib/prompt.ts (1)

169-187: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

CI-detection only matches literal 'true'/'1'.

process.env.CI !== 'true' && process.env.CI !== '1' will treat any other truthy CI value (e.g. 'True', 'yes', or a CI system that just sets CI=<pipeline-id>) as "interactive," letting the prompt block on stdin in an unattended job. Since this is explicitly meant to guard against exactly that per the path instructions ("enable only for real TTY runs (not CI...)"), consider a case-insensitive/truthy check (Boolean(process.env.CI)) instead of an exact-match allowlist.

🔧 Proposed fix
   const isInteractive = (input as { isTTY?: boolean }).isTTY === true &&
     (output as { isTTY?: boolean }).isTTY === true &&
-    process.env.CI !== 'true' &&
-    process.env.CI !== '1';
+    !process.env.CI;
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/prompt.ts` around lines 169 - 187, The CI gate in promptForPlanPath
only blocks when process.env.CI is exactly 'true' or '1', so other CI values can
still be treated as interactive. Update the isInteractive check in
promptForPlanPath to use a truthy CI test instead of exact string comparisons,
while keeping the existing TTY checks and fallback behavior unchanged.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/commands/test.ts`:
- Around line 1986-1992: The new interactive plan-path branch in test command
handling bypasses the existing deps injection pattern, making it hard to test
deterministically. Update the test flow around the `promptForPlanPath()` call to
source `stdin`/`stderrStream` from `TestDeps` (adding those optional fields if
needed) and pass them into `promptForPlanPath` so this side effect is fully
injectable like the other dependencies in `src/commands/test.ts`.

In `@src/lib/prompt.ts`:
- Around line 169-187: The CI gate in promptForPlanPath only blocks when
process.env.CI is exactly 'true' or '1', so other CI values can still be treated
as interactive. Update the isInteractive check in promptForPlanPath to use a
truthy CI test instead of exact string comparisons, while keeping the existing
TTY checks and fallback behavior unchanged.

In `@src/lib/update-check.ts`:
- Around line 137-140: The same debug-on-catch logging pattern is duplicated in
readUpdateCheckCache and writeUpdateCheckCache. Extract a small helper near
these functions, such as logDebug(resolved, label, err), that checks
isDebugLoggingEnabled(resolved.argv) and writes the formatted message to
resolved.stderr. Then replace both catch blocks to call the helper with the
appropriate label so the prefix stays specific while the shared logic is
centralized.
- Line 96: Add a one-line JSDoc comment for the new argv field in
UpdateCheckDeps to match the documentation style used by the other fields in
update-check.ts. Locate the interface containing the existing documented members
like sink and other deps, and add a concise description of what argv represents
so the public API stays consistently documented.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: dcf9e9ca-95e1-4fc5-a6e2-c14e66340a5c

📥 Commits

Reviewing files that changed from the base of the PR and between 3305dfa and 1433c6c.

📒 Files selected for processing (3)
  • src/commands/test.ts
  • src/lib/prompt.ts
  • src/lib/update-check.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant