pkg:create-deepcrawl:feat add jwt onboarding and dry-run preview

[lumpinif]

Summary by CodeRabbit

• New Features

  • CLI: improved deploy flow with a success card, dry‑run preview, post‑deploy "Try your API now?" quick test (auto‑mints 15‑minute JWT when needed), shows Worker URL and ready‑to‑copy curl; writes JWT_SECRET to local env files.

• Website

  • New homepage section with animated terminal demo for "npm create deepcrawl"; docs pages gain Copy Markdown and Open Docs actions.

• Documentation

  • Added detailed self‑hosting guide and quick‑test walkthroughs.

• Tests

  • Added extensive unit tests covering deploy parsing, JWT creation/validation, env file handling, quick‑test flow, and CLI output.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
deepcrawl	Ready	Preview, Comment	Mar 12, 2026 5:44pm

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Adds a new "npm create deepcrawl" self-hosting guide and CLI post-deploy quick-test flow; implements HS256 JWT minting, env-file upsert and dual-file local JWT persistence, quick-test runner (with curl preview), deployment parsing, CLI card rendering, UI components, and many unit tests.

Changes

Cohort / File(s)	Summary
Docs / Site apps/app/content/docs/reference/self-hosting/create-deepcrawl.mdx, apps/app/content/docs/reference/self-hosting/index.mdx, apps/app/content/docs/reference/self-hosting/meta.json, packages/create-deepcrawl/docs/*, apps/app/(docs)/docs/.../page.tsx	New create-deepcrawl docs and index note about persisted JWT_SECRET; docs links and header UI updated.
Homepage / Components apps/app/components/www/sections/create-deepcrawl-cli.tsx, apps/app/components/www/create-deepcrawl-terminal.tsx, apps/app/components/www/installer.tsx, apps/app/app/(www)/page.tsx	New homepage section, animated terminal, installer component, and integration into homepage.
JWT & Env Management packages/create-deepcrawl/src/lib/jwt.ts, packages/create-deepcrawl/src/lib/jwt-token.ts, packages/create-deepcrawl/src/lib/env-file.ts, packages/create-deepcrawl/src/steps/write-v0-local-jwt-secret.ts, packages/create-deepcrawl/src/steps/prepare-v0-local-project.ts	HS256 minting utility, secret strength validator, env-file upsert implementation, writer that persists JWT_SECRET to both dev and production dev var files, and local project preparation logic.
Quick Test & Deploy Flow packages/create-deepcrawl/src/steps/quick-test-v0-worker.ts, packages/create-deepcrawl/src/steps/deploy-v0-worker.ts, packages/create-deepcrawl/src/run.ts	Quick-test runner (builds token, fetch, preview, curl), deploy output parsing returning worker metadata, integrated quick-test UX (dry-run and live).
CLI UI / Cards packages/create-deepcrawl/src/ui/deployment-success.ts, packages/create-deepcrawl/src/ui/quick-test.ts, packages/create-deepcrawl/src/ui/render-card.ts, packages/create-deepcrawl/src/ui/card-format.ts, packages/create-deepcrawl/src/ui/terminal-style.ts, packages/create-deepcrawl/src/ui/prompt-answers.ts	Card rendering utilities, secret box formatting, prompt/quick-test UI, and tracking of generated-jwt flag in prompts.
Exec & Helpers packages/create-deepcrawl/src/lib/exec.ts, packages/create-deepcrawl/src/lib/terminal-text.ts, packages/create-deepcrawl/src/lib/exec.ts	RunCommand streaming callbacks (onStdout/onStderr) and terminal text sanitization for CLI output.
Tests packages/create-deepcrawl/src/__tests__/*	Many new/updated tests covering jwt-token, env upsert, local secret writes, deploy parsing, quick-test behavior, exec streaming, and UI/card outputs.
Config / Build / Styles apps/app/next.config.mjs, apps/app/tsconfig.json, apps/app/package.json, packages/ui/src/styles/globals.css, pnpm-workspace.yaml, package.json, packages/ui/package.json	Next.js rewrites and turbopack root change, path alias for globals.css, dependency/catalog updates (Tailwind-related entries), and package version bump for create-deepcrawl.
Site Utilities apps/app/components/docs/copy-markdown-button.tsx, apps/app/components/docs/open-docs-menu.tsx	New components to copy page Markdown and open docs in external AI/chat services.

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant CLI
  participant FS as FileSystem
  participant JWT as JWT_Generator
  participant Worker

  User->>CLI: run "npm create deepcrawl" / accept "Try your API now?"
  CLI->>FS: prepare local project (patch wrangler, upsert env files)
  FS-->>CLI: return local JWT secret file paths
  CLI->>CLI: prompt quick-test input (kind, targetUrl)
  alt authMode == jwt
    CLI->>JWT: mintHs256Jwt(secret, iss?, aud?)
    JWT-->>CLI: temporary Bearer token (15m)
    CLI->>Worker: fetch workerUrl + path with Authorization header
  else
    CLI->>Worker: fetch workerUrl + path without Authorization
  end
  Worker-->>CLI: response (status, body)
  CLI->>CLI: format/truncate preview, build curl command
  CLI-->>User: render success/failure card with preview and curl

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• feat/create-deepcrawl #22: Prior create-deepcrawl scaffolding and docs that this PR extends with JWT, env handling, and quick-test UX.

Poem

🐰 I hopped in with a joyful squeak,
Minted a token for fifteen minutes meek,
Poked the worker with a tiny curl,
Saw a tidy preview—flags unfurl,
Nibbled a carrot, then did a twirl.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 1.15% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main feature additions: JWT onboarding and dry-run preview functionality for the create-deepcrawl package.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ref/create-deepcrawl-312

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Adds an improved post-deployment onboarding experience for create-deepcrawl, including JWT-focused guidance, a “success” summary card, and an interactive quick-test flow (plus a dry-run preview path).

Changes:

• Add success/preview CLI “cards” and a post-deploy quick test (with optional JWT auto-minting).
• Persist JWT_SECRET into local Wrangler env files (.dev.vars + .dev.vars.production) and optionally show generated secrets once.
• Improve deployment parsing to extract Worker name/URL/version, and expand docs to cover the new onboarding flow.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/create-deepcrawl/src/ui/terminal-style.ts	Adds additional terminal styling helpers (yellow/bold/highlight).
packages/create-deepcrawl/src/ui/render-card.ts	Introduces CLI box/card renderer that accounts for ANSI styling.
packages/create-deepcrawl/src/ui/card-format.ts	Adds wrapping/indent helpers and secret “box” rendering.
packages/create-deepcrawl/src/ui/quick-test.ts	Implements interactive prompt flow + formatting for quick test results.
packages/create-deepcrawl/src/ui/prompt-answers.ts	Tracks whether JWT secret was generated; defaults “Deploy now?” to yes.
packages/create-deepcrawl/src/ui/deployment-success.ts	Builds the post-deploy success card (docs link, JWT guidance, file locations).
packages/create-deepcrawl/src/lib/jwt-token.ts	Adds HS256 JWT minting for quick tests.
packages/create-deepcrawl/src/lib/env-file.ts	Adds env-file upsert helper used to persist JWT secrets locally.
packages/create-deepcrawl/src/steps/write-v0-local-jwt-secret.ts	Writes JWT_SECRET to both local and production .dev.vars* files.
packages/create-deepcrawl/src/steps/quick-test-v0-worker.ts	Runs real quick tests + builds simulated preview results; mints JWT when needed.
packages/create-deepcrawl/src/steps/patch-v0-wrangler.ts	Aligns root vars with env.production.vars from an allowlist.
packages/create-deepcrawl/src/steps/deploy-v0-worker.ts	Captures deploy output for parsing and returns deployment metadata.
packages/create-deepcrawl/src/run.ts	Orchestrates preview-mode flow + post-deploy success + optional quick test.
packages/create-deepcrawl/src/tests/*	Adds coverage for JWT minting, env file upserts, deploy output parsing, and quick tests.
packages/create-deepcrawl/docs/v0/*.md	Updates internal CLI flow docs to include secret persistence + quick test.
packages/create-deepcrawl/docs/README.md	Updates package docs to reflect the new onboarding/quick test behavior.
apps/app/content/docs/reference/self-hosting/meta.json	Adds the new public doc page to navigation.
apps/app/content/docs/reference/self-hosting/index.mdx	Points users to the new “create-deepcrawl” deployment path.
apps/app/content/docs/reference/self-hosting/create-deepcrawl.mdx	Adds a new public doc page for npm create deepcrawl.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 381e89f3a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78bff6c38c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99f8926f6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".