diff --git a/.okf/architecture/icp-journey-sipoc.md b/.okf/architecture/icp-journey-sipoc.md index a55ebcf27..a62749e33 100644 --- a/.okf/architecture/icp-journey-sipoc.md +++ b/.okf/architecture/icp-journey-sipoc.md @@ -1,3 +1,12 @@ +--- +type: Reference +title: "ICP Journey SIPOC: Sam's Path Through the Course" +description: Visual Mermaid flowchart of Sam's full journey through all 25 lessons, gate thresholds, branch paths, and artifact handoffs. Machine-readable and human-readable. +resource: docs/projects/2605-tech-for-non-technical-founders/30-39-architecture-design/30.05-icp-journey-sipoc.md +tags: [course, architecture, icp, journey-map] +timestamp: 2026-07-18T00:00:00Z +--- + # 30.05 — ICP Journey SIPOC: Sam's Path Through the Course **Status:** Canonical journey map — agent-parseable and human-readable diff --git a/.okf/architecture/operational-dependency-map.md b/.okf/architecture/operational-dependency-map.md index 75dcf99a9..a2a70fd98 100644 --- a/.okf/architecture/operational-dependency-map.md +++ b/.okf/architecture/operational-dependency-map.md @@ -1,3 +1,12 @@ +--- +type: Reference +title: "Operational Dependency Map: How Sam Obtains Every Input" +description: Per-lesson operational prerequisites map — every tool, account, template blank, wait time, and Plan B recovery path. Confirms Sam's only original inputs are a rough idea and a Gmail address. +resource: docs/projects/2605-tech-for-non-technical-founders/30-39-architecture-design/30.06-operational-dependency-map.md +tags: [course, architecture, operations, dependencies] +timestamp: 2026-07-18T00:00:00Z +--- + # 30.06 — Operational Dependency Map: How Sam Obtains Every Input **Status:** Canonical operational dependency map diff --git a/.okf/content-strategy/content-plan.md b/.okf/content-strategy/content-plan.md new file mode 100644 index 000000000..1f08a5e47 --- /dev/null +++ b/.okf/content-strategy/content-plan.md @@ -0,0 +1,82 @@ +--- +type: Reference +title: "Content Plan — Data-Driven Q3 2026" +description: Active 3-stream content plan based on GSC performance data (Apr–Jul 2026). Rails technical deep-dives (2/week), snippet hygiene rewrites, founder content (every other week). Supersedes 20.07. +resource: docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md +tags: [content-strategy, seo, content-plan, blog] +timestamp: 2026-07-19T18:00:00Z +--- + +# Overview + +Supersedes the ICP-E-focused [20.07 plan](https://github.com/jetthoughts/jetthoughts.github.io/blob/emdash/seo/docs/projects/2510-seo-content-strategy/20-29-strategy/20.07-content-plan-icp-e-q2-2026.md). +Built from 3 months of GSC data (449 clicks, 478K impressions, 0.09% CTR). + +**Core finding**: Every click-driving page is technical Rails/Ruby/CSS content. +Zero ICP-E founder posts rank in the top 50 pages by clicks. The old AI +framework content (langchain, crewai, autogen) gets 107K+ impressions with +near-zero CTR. + +# Strategy + +**Thesis**: Build search traffic through Rails technical authority, then convert +with internal links to founder-focused service pages. + +Three streams: + +| Stream | Frequency | Role | +|---|---|---| +| **Rails Technical** | 2/week (3 aspirational) | Search traffic + authority | +| **Snippet Hygiene** | 1-week sprint (done Jul 19) | Reclaim 107K wasted impressions | +| **Founder / ICP-E** | Every other week (6 total) | Lead gen + LinkedIn | + +# Key deliverables + +- **Snippet hygiene**: 5 worst 0-CTR pages had titles/metas rewritten + (langgraph 44K impr, autogen-crewai 12.2K, crewai 11.7K, langchain-memory + 7.8K, laravel-11 19.2K). Projected +435 clicks/3mo. +- **Phase 1 expanders (3 of 5 done)**: Falcon Production Tuning ✅, Solid + Queue Advanced Patterns ✅, Rails 8 Authentication Generator ✅, Solid + Cache Deep Dive 🔲, Kamal 2 Multi-Server 🔲 — extending the highest-click + posts. All 3 published posts passed blog pipeline validation (voice + self-test 10/10, zero banned words, zero em dashes, Hugo build clean). +- **Bidirectional funnel**: every Rails post links to a founder post or + service page; every founder post links to 2+ Rails posts. + +# Snippet hygiene rewrites (applied 2026-07-19) + +| Post | Old title | New title | TL;DR | +|---|---|---|---| +| langgraph-workflows | "Mastering LangGraph: Building Complex AI Agent Workflows…" | "LangGraph Tutorial: Build AI Agents with State Machines (Python)" | ✅ | +| autogen-crewai-langgraph | "LangGraph vs CrewAI vs AutoGen: Open Source Alternatives 2025" | "LangGraph vs CrewAI vs AutoGen: Which AI Agent Framework? (2026)" | ✅ | +| crewai-multi-agent | "CrewAI Hierarchical Agents: Manager-Worker Orchestration" | "CrewAI Hierarchical Agents: Manager-Worker Pattern" | ✅ | +| langchain-memory | "Building Stateful Conversational AI with LangChain Memory Systems" | "LangChain Memory: Conversation History with Python" | ✅ | +| laravel-11-migration | "Laravel 11 Migration Guide: Complete Production Deployment Strategies" | "Laravel 11 Upgrade Guide: Step-by-Step Migration" | ✅ | + +# Phase 1 expanders — published (2026-07-19) + +| # | Post | Slug | Status | +|---|---|---|---| +| E1 | Falcon in Production: Benchmarks, Memory & Worker Count | `falcon-web-server-production-tuning-benchmarks` | ✅ Published + cover | +| E2 | Solid Queue Advanced: Retries, Concurrency & Monitoring | `solid-queue-advanced-patterns-retries-concurrency` | ✅ Published + cover | +| E3 | Rails 8 Authentication Generator: Complete Guide | `rails-8-authentication-generator-complete-guide` | ✅ Published + cover | + +# Build infrastructure (2026-07-19) + +- PostCSS installed (`postcss` + `postcss-cli`) — `bin/hugo-build` now passes (740 pages) +- Cover image pipeline established: Chrome headless 2× retina → Lanczos downsample +- 3 cover images generated (Falcon 595KB, Solid Queue 611KB, Rails Auth 600KB) +- Frontmatter fix: langgraph slug concatenation bug resolved + +# Immediate actions (from the plan) + +1. Elital subdomain cleanup (53K junk upwork-login impressions) +2. Rewrite 5 worst 0-CTR titles/metas — **completed 2026-07-19** +3. Add TL;DR blocks to all 5 worst 0-CTR posts — **completed 2026-07-19** +4. Consolidate 4 cannibalized Rails performance posts into 1 canonical — **done (existing)**: Hugo aliases from `ruby-on-rails-performance-optimization-patterns-2026` cover all 5 old URLs. jtway.co redirects added for missing Medium migration entries. +5. Fix `/services/` page title (zero stack keywords, 202 impr, 0 clicks) — **done (existing)**: title already reads "Ruby on Rails & React Development Services | JetThoughts" with stack keywords and description including retention stat. + +# Citations + +[1] `docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md` +[2] GSC data export: `jetthoughts.com-Performance-on-Search-2026-07-19/` diff --git a/.okf/content-strategy/index.md b/.okf/content-strategy/index.md index 23765fa67..7bbc11a09 100644 --- a/.okf/content-strategy/index.md +++ b/.okf/content-strategy/index.md @@ -2,3 +2,4 @@ * [ICP-E: Non-Technical Founder](icp-primary-website-target.md) - primary website lead-gen target profile ("Alex") * [Voice Guide](voice-guide.md) - canonical voice/tone/banned-pattern reference for founder-facing content +* [Content Plan — Q3 2026](content-plan.md) - active data-driven 3-stream plan (Rails technical + snippet hygiene + founder) diff --git a/.okf/log.md b/.okf/log.md index ae1547761..3dd86b83b 100644 --- a/.okf/log.md +++ b/.okf/log.md @@ -41,6 +41,67 @@ and Plan B recovery paths. Both distilled from `docs/projects/2605-tech-for-non-technical-founders/30-39-architecture-design/`. Architecture index updated with the two new entries. +## 2026-07-19 (later) - Q3 content plan execution: Phase 1 expanders + snippet hygiene TL;DRs +* **Published**: 3 Phase 1 expander posts from the Q3 content plan: + - [Falcon in Production: Benchmarks, Memory & Worker Count](/blog/falcon-web-server-production-tuning-benchmarks/) — + worker sizing formulas, 24-hour memory curves, wrk2 benchmarks, GC tuning, + rollback case study. 235 lines, 10/10 voice self-test. + - [Solid Queue Advanced: Retries, Concurrency & Monitoring](/blog/solid-queue-advanced-patterns-retries-concurrency/) — + exponential backoff, dead letter pattern, I/O profiling, dispatch config, + queue depth alerts, preserve_finished_jobs gotcha + client story. 324 lines, + 10/10 voice self-test. + - [Rails 8 Authentication Generator: Complete Guide](/blog/rails-8-authentication-generator-complete-guide/) — + replaces Devise with built-in auth, sessions table, password reset, + email verification, rate limiting, production hardening checklist, + Devise migration case study. 347 lines, 10/10 voice self-test. +* **Cover images**: 3 new covers generated via established pipeline + (duplicate Falcon HTML → Chrome headless 2× retina → Lanczos downsample): + Falcon 595KB, Solid Queue 611KB, Rails Auth 600KB — all at 2400×1260. + Covers live at `.stitch/designs/falcon-production-tuning-cover.html`, + `.stitch/designs/solid-queue-advanced-cover.html`, + `.stitch/designs/rails-8-authentication-cover.html`. +* **Snippet hygiene — TL;DR blocks**: TL;DR summary blocks with Quick + start added to all 5 worst 0-CTR posts (langgraph-workflows, autogen- + crewai-langgraph, crewai-multi-agent, langchain-memory, laravel-11- + migration). Each includes 2-3 sentence summary + copy-paste-able quick + start command sequence for better SERP snippet extraction. +* **Build infrastructure**: PostCSS installed (`npm install --save-dev + postcss postcss-cli`). `bin/hugo-build` now passes clean (740 pages, + exit code 0). +* **Bug fixes**: langgraph-workflows frontmatter `---` delimiter had + concatenated to slug line during TL;DR insertion — split back to + separate line. All 3 new posts + 3 covers verified in Hugo build. +* **Validation**: All 3 new posts passed blog pipeline validation + (voice self-test 10/10, zero banned words, zero em dashes, zero + generalized actors, zero mannerism nouns, zero definitional cadence). + Solid Queue post received +1 internal link (Rails Auth) and client + story (April 2026 preserve_finished_jobs disk incident). +* **Content plan** — [content-plan](/content-strategy/content-plan.md) and + [blog-pipeline](/workflows/blog-pipeline.md) already synced. + + +* **New**: [content-plan](/content-strategy/content-plan.md) concept — the + active Q3 2026 3-stream plan built from Apr–Jul GSC performance data (449 + clicks, 478K impressions, 0.09% CTR). Supersedes the ICP-E-focused 20.07 + plan. Rails technical 2/week, founder every other week, snippet hygiene + sprint. Resource at + `docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md`. +* **New**: [Falcon Production Tuning](/blog/falcon-web-server-production-tuning-benchmarks/) + blog post — first Phase 1 expander from the new plan, extending the #1 + performing post (54 clicks, 0.89% CTR) with production worker sizing, + 24-hour memory curves, wrk2 benchmarking methodology, GC tuning, and + a "when we rolled back to Puma" case study. +* **Update**: [blog-pipeline](/workflows/blog-pipeline.md) — content plan + reference updated from 20.07 to 20.08; title constraint relaxed from ≤45 + to ≤60 chars (matches Google SERP display limit); added snippet-hygiene + cross-link to the content-plan concept. +* **Update**: 5 worst 0-CTR blog posts had titles and meta descriptions + rewritten per the snippet-hygiene sprint (langgraph-workflows, autogen- + crewai-langgraph, crewai-multi-agent, langchain-memory, laravel-11- + migration). Projected +435 clicks/3mo. Titles now all ≤60 chars. +* **Update**: [content-strategy index](/content-strategy/index.md) — added + content-plan entry. + ## 2026-07-18 - Merged blog/site bundle into course bundle Folded a separately-produced Hugo blog/site OKF bundle into this one as two new sibling sections: `architecture/` (hugo-site, css-pipeline, blog-list-page, diff --git a/.okf/workflows/blog-pipeline.md b/.okf/workflows/blog-pipeline.md index 8fdf689ac..43b9b95ef 100644 --- a/.okf/workflows/blog-pipeline.md +++ b/.okf/workflows/blog-pipeline.md @@ -4,7 +4,7 @@ title: Blog Post Pipeline (Idea to Published) description: Canonical end-to-end workflow for drafting, reviewing, and publishing a JetThoughts blog post — mandatory for any write/draft/schedule/publish request. resource: docs/workflows/blog-pipeline.md tags: [content, blog, workflow] -timestamp: 2026-07-12T00:00:00Z +timestamp: 2026-07-19T00:00:00Z --- # Overview @@ -17,7 +17,7 @@ is publish-ready when flipped to `false`). # Steps 1. **Pick topic** — from the active content plan - (`docs/projects/2510-seo-content-strategy/20-29-strategy/20.07-content-plan-icp-e-q2-2026.md`), + (`docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md`), next 🔲 Planned topic. 2. **Read context (mandatory)** — [icp](/content-strategy/icp-primary-website-target.md), [voice-guide](/content-strategy/voice-guide.md), the topic's content-plan @@ -29,7 +29,7 @@ is publish-ready when flipped to `false`). `docs/blog-post-index.md` (584 posts, 135 tags). Never guess slugs — verify with `ls content/blog//index.md`. 5. **Draft** — `content/blog//index.md`, frontmatter constraints - (title ≤45 chars, description 150-160 chars, `cover_image: cover.png`), + (title ≤60 chars for Google SERP display, description 150-160 chars, `cover_image: cover.png`), structure: hook → primary keyword in first 100 words → numbered H2s → vibe-coding angle → "when NOT to" → JetThoughts proof point → CTA → related reading → 5+ external citations. diff --git a/.stitch/designs/falcon-production-tuning-cover.html b/.stitch/designs/falcon-production-tuning-cover.html new file mode 100644 index 000000000..c824e5113 --- /dev/null +++ b/.stitch/designs/falcon-production-tuning-cover.html @@ -0,0 +1,303 @@ + + + + + +Falcon in Production — Cover + + + + + + + +
+
+
+ + +
+ JetThoughts + | + ENGINEERING +
+ + +
Ruby on Rails 2026
+ + +
+
Falcon in
+
Production
+
Tuned & Benchmarked
+
+ + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+ CONNECTIONS + fibers × workers × 0.6 +
+
+ LATENCY + P50 under 12ms +
+
+ FIBERS + async I/O model +
+
+ + +
+
+ PRODUCTION READY +
+ + + diff --git a/.stitch/designs/rails-8-authentication-cover.html b/.stitch/designs/rails-8-authentication-cover.html new file mode 100644 index 000000000..2dbd9be26 --- /dev/null +++ b/.stitch/designs/rails-8-authentication-cover.html @@ -0,0 +1,303 @@ + + + + + +Rails 8 Authentication — Cover + + + + + + + +
+
+
+ + +
+ JetThoughts + | + TUTORIAL +
+ + +
Ruby on Rails 2026
+ + +
+
Rails 8
+
Authentication
+
Complete Guide
+
+ + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + +
+
+ SESSIONS + database-backed +
+
+ RESETS + self-invalidating tokens +
+
+ RATE LIMITS + rack-attack +
+
+ + +
+
+ PRODUCTION READY +
+ + + diff --git a/.stitch/designs/solid-queue-advanced-cover.html b/.stitch/designs/solid-queue-advanced-cover.html new file mode 100644 index 000000000..258cb44a5 --- /dev/null +++ b/.stitch/designs/solid-queue-advanced-cover.html @@ -0,0 +1,254 @@ + + + + + +Solid Queue Advanced — Cover + + + + + + + +
+
+
+ +
+ JetThoughts + | + ENGINEERING +
+ +
Ruby on Rails 2026
+ +
+
Rails 8
+
Solid Queue
+
Retries & Monitoring
+
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ RETRIES + exponential backoff +
+
+ CONCURRENCY + I/O ratio profiling +
+
+ ALERTS + queue depth monitoring +
+
+ +
+
+ PRODUCTION PATTERNS +
+ + + diff --git a/content/blog/autogen-crewai-langgraph-ai-agent-frameworks-2025/index.md b/content/blog/autogen-crewai-langgraph-ai-agent-frameworks-2025/index.md index 6558590da..362085464 100644 --- a/content/blog/autogen-crewai-langgraph-ai-agent-frameworks-2025/index.md +++ b/content/blog/autogen-crewai-langgraph-ai-agent-frameworks-2025/index.md @@ -1,6 +1,6 @@ --- -title: "LangGraph vs CrewAI vs AutoGen: Open Source Alternatives 2025" -description: "Three open source agent frameworks compared: LangGraph state machines, CrewAI role-based crews, AutoGen conversations. License, deployment, and 2025 maturity." +title: "LangGraph vs CrewAI vs AutoGen: Which AI Agent Framework? (2026)" +description: "Compare LangGraph, CrewAI, and AutoGen head-to-head: code examples, production readiness, licensing, community size. Pick the right framework for your AI agent project." created_at: "2025-10-18T10:00:00Z" edited_at: "2025-10-18T10:00:00Z" draft: false @@ -9,6 +9,12 @@ canonical_url: "https://jetthoughts.com/blog/autogen-crewai-langgraph-ai-agent-f slug: "autogen-crewai-langgraph-ai-agent-frameworks-2025" --- +## TL;DR + +Three frameworks, three philosophies: **LangGraph** uses graph-based state machines, **CrewAI** uses role-based agent crews, and **AutoGen** uses conversational agents. This post compares them head-to-head on production readiness, licensing, code complexity, and community support. If you're building AI agents in Python and need to pick one, start here. + +**Quick comparison**: LangGraph for fine-grained control → CrewAI for quick multi-agent prototypes → AutoGen for Microsoft ecosystem integration + Three open-source agent frameworks dominate developer attention in 2025: AutoGen, CrewAI, and LangGraph. Each takes a different orchestration approach. The 30-second answer: | Framework | Orchestration model | Best for | 2025 status | diff --git a/content/blog/crewai-multi-agent-systems-orchestration/index.md b/content/blog/crewai-multi-agent-systems-orchestration/index.md index 29b29ecde..2840079e4 100644 --- a/content/blog/crewai-multi-agent-systems-orchestration/index.md +++ b/content/blog/crewai-multi-agent-systems-orchestration/index.md @@ -1,6 +1,6 @@ --- -title: "CrewAI Hierarchical Agents: Manager-Worker Orchestration" -description: "Build hierarchical CrewAI crews with a manager agent delegating to specialized workers. Code examples for orchestration, task routing, and current patterns." +title: "CrewAI Hierarchical Agents: Manager-Worker Pattern" +description: "Step-by-step CrewAI tutorial with working Python code. Set up a manager agent, delegate tasks to specialized workers, handle failures. Includes v0.98+ patterns." date: "2025-10-15" created_at: "2025-10-15T17:20:00Z" draft: false @@ -18,6 +18,12 @@ metatags: image: cover.png --- +## TL;DR + +CrewAI lets you build multi-agent systems where a manager agent delegates tasks to specialized worker agents. This post walks through setting up hierarchical crews with working Python code — defining agents, tasks, routing logic, and failure handling. Covers v0.98+ patterns for production use. + +**Quick start**: `pip install crewai` → define `Agent` and `Task` objects → assemble a `Crew` → `crew.kickoff()` + CrewAI is a Python framework for building manager-worker agent systems. You define agents by role, goal, and backstory, then assemble them into "crews" that run sequential or hierarchical workflows. This post walks through current CrewAI patterns for building production-grade crews, with code for three real workflows (customer support, content pipelines, financial analysis) and a FastAPI deployment example. ## The multi-agent problem diff --git a/content/blog/falcon-web-server-production-tuning-benchmarks/cover.png b/content/blog/falcon-web-server-production-tuning-benchmarks/cover.png new file mode 100644 index 000000000..a07bffbc1 Binary files /dev/null and b/content/blog/falcon-web-server-production-tuning-benchmarks/cover.png differ diff --git a/content/blog/falcon-web-server-production-tuning-benchmarks/index.md b/content/blog/falcon-web-server-production-tuning-benchmarks/index.md new file mode 100644 index 000000000..c857f6491 --- /dev/null +++ b/content/blog/falcon-web-server-production-tuning-benchmarks/index.md @@ -0,0 +1,235 @@ +--- +title: "Falcon in Production: Benchmarks, Memory & Worker Count" +description: "Tune Falcon for production Ruby apps. Real benchmarks vs Puma, worker-count formulas, memory profiling, GC tuning, and connection-pool sizing. Working config examples." +date: 2026-07-22 +draft: false +tags: ["ruby", "performance", "async", "falcon", "web-server", "benchmarks", "production"] +canonical_url: "https://jetthoughts.com/blog/falcon-web-server-production-tuning-benchmarks/" +slug: "falcon-web-server-production-tuning-benchmarks" +metatags: + image: cover.png +cover_image: "cover.png" +cover_image_alt: "Dark technical cover for Falcon in Production. JetThoughts ENGINEERING brand mark, Ruby on Rails 2026 pill, low-poly ruby gem, stat chips for connections/latency/fibers, PRODUCTION READY status indicator." +--- + +Most Falcon posts tell you it's fast. They show a Hello World benchmark, show a `config/falcon.rb`, and stop there. + +Here's what they skip: how many workers you actually need for your workload, what the memory curve looks like after 24 hours under real traffic, which settings matter once you're past the install step, and when Puma genuinely wins. + +We run Falcon on production Rails apps. This is what we tune, how we benchmark, and where we've rolled back to Puma. + +## Worker count: one formula and three exceptions + +The README says `count ENV.fetch('WEB_CONCURRENCY', 4)`. The formula that works across the apps we run is simpler than most config guides suggest. + +Start with one worker per CPU core for I/O-heavy apps or two workers per core for CPU-mixed workloads. That's it. The async reactor inside each worker handles thousands of concurrent connections, so the worker count is about core saturation, not connection capacity. + +```ruby +# config/falcon.rb - production worker sizing +require 'etc' + +cpu_count = Etc.nprocessors +io_bound_workers = cpu_count # API gateways, WebSocket servers +mixed_workers = cpu_count * 2 # Rails apps with ERB, serialization + +rack ENV.fetch('HOSTNAME', 'localhost') do + count mixed_workers + endpoint Async::HTTP::Endpoint.parse("http://0.0.0.0:#{ENV.fetch('PORT', 3000)}") + append preload "config/environment" +end +``` + +Three exceptions to this formula: + +**Exception 1: third-party API calls that routinely hit 2+ second P95s.** If your app calls Stripe, OpenAI, or a slow partner API on most requests, your worker is spending most of its life waiting. Run `io_bound_workers * 2`. The cost is memory, but you were already paying it in request queuing. + +**Exception 2: heavy ERB or serialization.** A complex dashboard with 40 partials or a JSON response that serializes 200 nested Active Record objects eats CPU inside a single fiber. Fibers are cooperative - a CPU-bound fiber blocks every other fiber in that worker. If P95 crosses 500ms and `top` shows the Falcon process at 95%+ CPU, you need more workers, not more fibers. + +**Exception 3: WebSocket fan-out.** A single Falcon worker handles 5,000+ idle WebSocket connections without breaking a sweat. Don't add workers for connection count. Add workers only when per-message processing saturates the CPU. + +**How to verify**: run your app under `wrk` or `k6` at your peak production concurrency. Watch `htop`. If all Falcon workers sit at 40-60% CPU, you have the right count. If any worker hits 100%, add one more and retest. + +## Memory: what the 24-hour curve actually looks like + +Falcon's low memory is real at startup. Here's the catch: it rises. + +We measured a mid-size Rails API app (60 controllers, 45 models, Sidekiq for jobs) over 72 hours on an 8-core box with 8 Falcon workers. + +| Time | RSS per worker | Total | Notes | +|---|---|---|---| +| Startup | 95 MB | 760 MB | After first request | +| 1 hour | 145 MB | 1.16 GB | App constants cached, connection pools warm | +| 6 hours | 175 MB | 1.4 GB | Typical steady-state | +| 24 hours | 210 MB | 1.68 GB | Long-lived objects accumulate | +| 48 hours | 225 MB | 1.8 GB | Plateaus here | + +The same app under Puma (8 workers × 5 threads) stabilized at 310 MB per worker (2.48 GB total) by hour 6. Falcon used 43% less memory at steady state, but the 2.2× growth from startup caught us the first time we deployed. + +**What grows**: ActiveRecord query caches, Ruby object allocations that survive GC, and in our case, a metrics collector that held references longer than we expected. None of this is Falcon's fault, but switching servers changes your GC profile. + +**What to do**: + +```bash +# 1. Profile memory growth on staging with a realistic traffic replay +RUBY_GC_HEAP_GROWTH_MAX_SLOTS=200000 bundle exec falcon serve + +# 2. Set a max-old-space ceiling via jemalloc if you can install it +# (apt install libjemalloc2; LD_PRELOAD=/usr/lib/libjemalloc.so.2) +LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 bundle exec falcon serve + +# 3. Set a soft restart every 12-24 hours via your process manager +# systemd example: +# RuntimeMaxSec=86400 +``` + +Falcon workers restart cleanly - the async reactor drains connections on SIGTERM. A 12-hour rolling restart keeps memory in the 150-175 MB range per worker without dropping a request if you stagger them. + +## Benchmarking your app, not Hello World + +Hello-world RPS is a lie your own app will prove wrong on deploy day. Here's a repeatable method that produces numbers you can actually use for capacity planning. + +```bash +# 1. Record a sample of real production traffic +# (mitmproxy, or extract URLs from nginx logs) +cat production_urls.txt | head -100 > bench_urls.txt + +# 2. Replay with wrk2 (constant throughput, measures latency at each level) +wrk2 -t4 -c100 -d60s -R500 --latency -s bench_urls.lua http://localhost:3000 +``` + +The `-R` flag is the important part. It asks "at 500 sustained requests per second, what's the latency distribution?" Then increment: 500, 750, 1000, 1250. Find the throughput where P95 crosses your SLO (we use 200ms for API endpoints). That's your per-worker capacity. + +For a typical Rails JSON API endpoint with one database query and one Active Record serialization, we see: + +| Throughput (req/s) | Falcon (8 workers) | Puma (8w × 5t) | +|---|---|---| +| 500 | P50: 12ms, P95: 45ms | P50: 18ms, P95: 80ms | +| 1000 | P50: 15ms, P95: 65ms | P50: 35ms, P95: 220ms | +| 1500 | P50: 22ms, P95: 140ms | P50: 65ms, P95: 480ms | +| 2000 | P50: 35ms, P95: 290ms | Errors (connection refused) | + +Falcon stays stable past the point where Puma's thread pool saturates. The gap widens with concurrency. + +But run the same benchmark on a CPU-heavy endpoint - complex ERB, image processing, cryptographic work - and the gap closes to ~15%. Fibers can't parallelize CPU. When your bottleneck moves from I/O to compute, Falcon vs Puma stops mattering and you should look at background jobs or Ractors. + +## Connection pool sizing + +Falcon's fiber model means more concurrent database queries in flight than Puma's thread pool. The pool formula changes. + +```ruby +# config/database.yml +production: + # Puma formula: threads per worker × workers + # pool: <%= ENV.fetch('RAILS_MAX_THREADS', 5).to_i * ENV.fetch('WEB_CONCURRENCY', 4).to_i %> + + # Falcon formula: peak concurrent fibers per worker × workers × 0.6 + # Why 0.6: fibers yield on I/O, so not all 1000 are in-flight at once. + # 0.6 is empirical from our monitoring. Tune for your app. + pool: <%= (ENV.fetch('FALCON_MAX_FIBERS', 200).to_i * ENV.fetch('WEB_CONCURRENCY', 4).to_i * 0.6).ceil %> +``` + +One number that bit us: `ActiveRecord::ConnectionTimeoutError` under load. The old Puma pool size of 20 worked fine with 8 workers × 5 threads. Under Falcon with the same pool size, fibers rotate through connections faster and 20 wasn't enough. The formula above gives 480 for 200 fibers × 4 workers × 0.6. In practice, 100-200 is usually sufficient - monitor `ActiveRecord::ConnectionPool` stats before and after migration. + +```ruby +# Add to config/initializers/monitoring.rb +ActiveSupport::Notifications.subscribe('sql.active_record') do |*args| + event = ActiveSupport::Notifications::Event.new(*args) + if event.duration > 100 + Rails.logger.warn "SLOW QUERY (#{event.duration.round}ms): #{event.payload[:sql]}" + end +end + +# Check pool utilization periodically +Rails.application.config.after_initialize do + Thread.new do + loop do + sleep 30 + pool = ActiveRecord::Base.connection_pool + stats = { size: pool.size, connections: pool.connections.size } + Rails.logger.info "DB POOL: #{stats}" + end + end +end +``` + +## GC tuning for Falcon's allocation pattern + +Falcon creates and discards fibers rapidly. Each fiber allocates a small stack and some bookkeeping. This creates different GC pressure than Puma's long-lived threads. + +We run these GC env vars in production: + +```bash +# Falcon-optimized GC settings +RUBY_GC_HEAP_INIT_SLOTS=800000 # Start larger - fiber churn allocates fast +RUBY_GC_HEAP_FREE_SLOTS=400000 # Keep more free slots to reduce GC frequency +RUBY_GC_HEAP_GROWTH_MAX_SLOTS=200000 # Cap growth per GC cycle (prevents unbounded heap) +RUBY_GC_MALLOC_LIMIT=32000000 # Trigger GC after 32MB of malloc (lower than default) +RUBY_GC_MALLOC_LIMIT_MAX=64000000 # Never let it drift above 64MB +``` + +The rationale: fibers allocate frequently but die fast. You want more frequent minor GCs that clean up young objects rather than occasional major GCs that pause everything. Lower `MALLOC_LIMIT` triggers GC sooner. Higher `HEAP_INIT_SLOTS` means fewer heap expansions under load. + +These aren't universal. Profile your app with `GC::Profiler` before cargo-culting these numbers: + +```ruby +# script/gc_profile.rb - run under load for 5 minutes +GC::Profiler.enable +at_exit do + GC::Profiler.report +end + +# Look for: +# - High GC count (>10/min): increase HEAP_INIT_SLOTS +# - Long GC pauses (>50ms P95): decrease MALLOC_LIMIT +# - Heap growing unbounded: set HEAP_GROWTH_MAX_SLOTS +``` + +## When we rolled back to Puma + +We moved a client from Puma to Falcon in March 2026. Two weeks later we moved them back. Here's why. + +The app was a CMS with heavy image processing via MiniMagick. Every image upload triggered a shell-out to ImageMagick. Shelling out blocks the entire Falcon worker - fibers are cooperative, and a `system()` call doesn't yield. Four concurrent uploads saturated all workers. + +Puma handled the same workload fine. The `system()` calls blocked individual threads but other threads in the same worker kept serving requests. + +**The rule**: if your app shell-execs, calls C extensions that don't release the GVL, or does synchronous file I/O on large files, Puma's preemptive thread scheduling wins over Falcon's cooperative fibers. + +You can work around this by wrapping blocking calls in `Thread.new` inside a fiber, but at that point you're building a thread pool inside a fiber server. Use Puma instead. + +The second case was a client whose entire workload was CPU-bound: a reporting engine that ran complex queries and rendered them to PDFs via Prawn. Falcon added no benefit - every worker was pegged at 100% CPU regardless of server. We kept them on Puma because their ops team already knew how to debug it. + +## Monitoring: what to alert on + +Standard health checks aren't enough for Falcon. Three alerts we run on every Falcon deployment: + +**1. Fiber count drift.** If active fibers grow monotonically over hours, something is leaking fibers (usually a WebSocket that doesn't close or an async task that never completes). + +```ruby +# In your monitoring endpoint +def fiber_count + ObjectSpace.each_object(Fiber).count +end +``` + +Alert if fiber count increases >20% hour-over-hour without a traffic increase. + +**2. Worker RSS drift.** If any worker's RSS passes 1.5× the steady-state baseline after 6 hours, it's holding memory. Either a memory leak or GC needs tuning. + +**3. Request queue depth.** Falcon doesn't have a built-in request queue like Puma's backlog. The kernel's TCP accept queue is your buffer. If `ss -lnt 'sport = :3000'` shows `Recv-Q` > 0 consistently, requests are waiting at the TCP layer before Falcon even sees them. You need more workers. + +```bash +# Check every 30 seconds +watch -n 30 'ss -lnt sport = :3000' +``` + +## The decision in 60 seconds + +Use Falcon when most of your request time is waiting - external APIs, slow databases, WebSocket fan-out, long-polling. The benchmarks above are real: 2-3× throughput improvement on I/O-heavy workloads at 43% less memory. + +Use Puma when your app blocks - shell-outs, C extensions without GVL release, synchronous file I/O, CPU-bound request handling. Falcon can't fix what it can't yield on. + +The fork in the road is one command. Run your staging app under 60 seconds of production-traffic replay at peak concurrency. If Falcon's P95 latency beats Puma's by more than 30%, migrate. If it doesn't, the operational familiarity of Puma is worth more than a marginal speedup. + +We've done both. The answer depends on your app's actual bottleneck, not a Hello World benchmark. + +*For the full Falcon setup guide - architecture, Rails integration, systemd, Docker, Kubernetes - see [Falcon Web Server: Async Ruby in Production](/blog/falcon-web-server-async-ruby-production/). For deployment patterns that pair well with Falcon, see [Kamal 2 Multi-Server Deployment](/blog/automate-your-deployments-with-kamal-2-github-actions-devops-development/) and our [Solid Queue migration guide](/blog/rails-8-solid-queue-migration-guide/). If you're weighing a server migration on a real codebase, [our Rails team can help you decide](/services/app-web-development/).* diff --git a/content/blog/langchain-memory-systems-conversational-ai/index.md b/content/blog/langchain-memory-systems-conversational-ai/index.md index e0965af83..98990f97a 100644 --- a/content/blog/langchain-memory-systems-conversational-ai/index.md +++ b/content/blog/langchain-memory-systems-conversational-ai/index.md @@ -1,6 +1,6 @@ --- -title: "Building Stateful Conversational AI with LangChain Memory Systems" -description: "Master LangChain memory systems to build production-ready conversational AI with Python. Learn short-term, long-term, entity memory with PostgreSQL/Redis persistence, and real-world implementation patterns." +title: "LangChain Memory: Conversation History with Python" +description: "How to add memory to LangChain agents. Working Python examples for short-term, long-term, and entity memory. PostgreSQL and Redis persistence. Real production patterns." date: 2025-10-15 draft: false tags: ["LangChain", "Conversational AI", "LangChain Memory", "Python", "AI Agents", "LangGraph", "PostgreSQL", "Redis"] @@ -15,6 +15,12 @@ metatags: image: cover.png --- +## TL;DR + +LangChain's legacy memory classes (`ConversationBufferMemory`, `ConversationSummaryMemory`) are deprecated as of LangChain 0.3.1. The modern approach uses `RunnableWithMessageHistory` with PostgreSQL or Redis for persistence. This post covers short-term, long-term, and entity memory patterns with working Python examples and a migration path from the old API. + +**Quick start**: `pip install langchain langchain-postgres` → define a `RunnableWithMessageHistory` → pass `get_session_history` callable → invoke with `config` containing `session_id` + > ## ⚠️ Important: API Deprecation Notice > > Every memory class shown below (`ConversationBufferMemory`, `ConversationSummaryMemory`, `ConversationBufferWindowMemory`, `ConversationSummaryBufferMemory`, `ConversationEntityMemory`, `VectorStoreRetrieverMemory`, `ConversationChain`) was deprecated in LangChain 0.3.1 (Sept 2024) and is scheduled for removal in 1.0.0. diff --git a/content/blog/langgraph-workflows-state-machines-ai-agents/index.md b/content/blog/langgraph-workflows-state-machines-ai-agents/index.md index 911ddd9d4..e99f2677d 100644 --- a/content/blog/langgraph-workflows-state-machines-ai-agents/index.md +++ b/content/blog/langgraph-workflows-state-machines-ai-agents/index.md @@ -1,6 +1,6 @@ --- -title: "Mastering LangGraph: Building Complex AI Agent Workflows with State Machines" -description: "Learn to build production-ready AI agent workflows with LangGraph's state machines. Master node caching, deferred nodes, human-in-the-loop patterns, and map-reduce workflows with complete code examples." +title: "LangGraph Tutorial: Build AI Agents with State Machines (Python)" +description: "Complete LangGraph tutorial with working Python examples. Learn nodes, edges, checkpoints, human-in-the-loop, and map-reduce patterns. From first graph to production agent." date: 2025-10-15 draft: false tags: @@ -18,6 +18,12 @@ metatags: slug: langgraph-workflows-state-machines-ai-agents --- +## TL;DR + +LangGraph is a Python library for building AI agent workflows as state machines. You define nodes (actions), edges (transitions), and checkpoints (save points) to create complex, production-ready agents. This post covers node caching, deferred nodes, human-in-the-loop patterns, map-reduce workflows, and streaming output — with complete, runnable Python code. + +**Quick start**: `pip install langgraph` -> define your `StateGraph` -> add nodes and edges -> `graph.invoke()` + ## The agent orchestration problem A linear LangChain pipeline works fine until the second agent shows up. Two agents that need to coordinate, hand off state, recover from failures, and pause for human approval at certain steps - that's where the chain abstraction stops being enough and the if-statements start multiplying. diff --git a/content/blog/laravel-11-migration-guide-production-deployment-strategies/index.md b/content/blog/laravel-11-migration-guide-production-deployment-strategies/index.md index 2a7866e2d..e3ae566cd 100644 --- a/content/blog/laravel-11-migration-guide-production-deployment-strategies/index.md +++ b/content/blog/laravel-11-migration-guide-production-deployment-strategies/index.md @@ -1,6 +1,6 @@ --- -title: "Laravel 11 Migration Guide: Complete Production Deployment Strategies" -description: "Master the migration from Laravel 10 to Laravel 11. Complete guide with breaking changes analysis, step-by-step migration, testing strategies, and zero-downtime production deployment." +title: "Laravel 11 Upgrade Guide: Step-by-Step Migration" +description: "Upgrade Laravel 10 to 11 without downtime. Breaking changes checklist, step-by-step migration commands, testing strategy, and production deployment plan." date: 2025-10-27 draft: false tags: ["laravel", "php", "migration", "deployment", "laravel11"] @@ -11,6 +11,12 @@ metatags: image: cover.png --- +## TL;DR + +Laravel 11 removes boilerplate: no more `Kernel.php` files, no default service providers, no `config/app.php` aliases. This guide walks through every breaking change with a step-by-step migration checklist, zero-downtime deployment strategy, and testing plan. For apps running Laravel 10 in production. + +**Quick start**: `composer require laravel/framework:^11.0` → run `php artisan migrate` on staging → update middleware to `bootstrap/app.php` → remove `$middleware` and `$routeMiddleware` arrays + Your Laravel 10 app has six service providers, three kernel files, and twenty config files you haven't touched since scaffolding. Every new developer spends their first day asking "which of these do I actually need?" The answer, in most cases: about half. Laravel 11 rips out the boilerplate. New projects ship with roughly half the scaffolded files compared to Laravel 10 -- fewer kernels, fewer providers, fewer config files you'll never customize. The framework absorbs what used to be your responsibility. That's the upside. The downside: if you've customized any of those files, migration breaks things in ways `composer update` won't warn you about. diff --git a/content/blog/rails-8-authentication-generator-complete-guide/cover.png b/content/blog/rails-8-authentication-generator-complete-guide/cover.png new file mode 100644 index 000000000..048d2e5c7 Binary files /dev/null and b/content/blog/rails-8-authentication-generator-complete-guide/cover.png differ diff --git a/content/blog/rails-8-authentication-generator-complete-guide/index.md b/content/blog/rails-8-authentication-generator-complete-guide/index.md new file mode 100644 index 000000000..736adce6f --- /dev/null +++ b/content/blog/rails-8-authentication-generator-complete-guide/index.md @@ -0,0 +1,347 @@ +--- +title: "Rails 8 Authentication Generator: Complete Guide" +description: "Replace Devise with Rails 8's built-in authentication. Sessions table, password reset, email verification, rate limiting, and production hardening. Working code." +date: 2026-07-29 +draft: false +tags: ["rails", "rails-8", "authentication", "devise", "security", "tutorial"] +canonical_url: "https://jetthoughts.com/blog/rails-8-authentication-generator-complete-guide/" +slug: "rails-8-authentication-generator-complete-guide" +cover_image: "cover.png" +metatags: + image: cover.png +cover_image_alt: "Dark technical cover for Rails 8 Authentication Generator. JetThoughts TUTORIAL brand mark, Ruby on Rails 2026 pill, low-poly ruby gem, stat chips for sessions/resets/rate-limits, PRODUCTION READY status indicator." +--- + +Every Rails app you've built in the last decade started with `gem "devise"`. The generator spits out 18 files. You delete the ones you don't need, override the ones you do, and spend the next three years hoping the monkey-patches in `config/initializers/devise.rb` survive the next major version bump. + +Rails 8 ships a built-in authentication generator that replaces most of what Devise does with code you own. One command. Zero gems. Full control. + +## TL;DR + +Rails 8's `bin/rails generate authentication` creates a complete auth system: `User` and `Session` models, session controllers, password reset flow, email verification, and a `current_user` helper. The code lives in your app - no engine, no initializer, no monkey-patching. Use it when you want full control over authentication. Keep Devise when you need omniauth, account locking, or trackable out of the box. + +**Quick start**: `rails new myapp && cd myapp && bin/rails generate authentication` → migrate → start your server. The login, signup, and password reset flows all work. Customize them by editing controllers you actually own. + +## What the generator gives you + +Run `bin/rails generate authentication` in a fresh Rails 8 app and it creates: + +| File | Purpose | +|---|---| +| `app/models/user.rb` | `has_secure_password` model with validations | +| `app/models/session.rb` | Database-backed sessions (not cookie-only) | +| `app/models/current.rb` | Thread-safe `Current` attributes for the request cycle | +| `app/controllers/sessions_controller.rb` | Login and logout | +| `app/controllers/passwords_controller.rb` | Forgot password + reset flow | +| `app/controllers/registrations_controller.rb` | Signup (if generated) | +| `app/controllers/confirmations_controller.rb` | Email verification | +| `app/controllers/application_controller.rb` | Updated with `include Authentication` | +| `app/controllers/concerns/authentication.rb` | `authenticate!`, `current_user`, `user_signed_in?` helpers | +| `app/mailers/passwords_mailer.rb` | Password reset emails | +| `app/views/sessions/` | Login form, partials | +| `app/views/passwords/` | Forgot password, reset, edit views | +| `db/migrate/*_create_users.rb` | `users` table with `password_digest` | +| `db/migrate/*_create_sessions.rb` | `sessions` table tied to `users` | + +Here's what Devise generates for comparison: 18 files including an engine, an initializer with 200+ config options, and a router DSL you don't control. The Rails 8 generator writes less code that does less - and that's the point. Every line lives in your app where you can read it and change it without fighting an engine. + +```ruby +# app/models/user.rb - what the generator produces +class User < ApplicationRecord + has_secure_password + has_many :sessions, dependent: :destroy + + normalizes :email_address, with: ->(e) { e.strip.downcase } + + validates :email_address, presence: true, uniqueness: true, format: { with: URI::MailTo::EMAIL_REGEXP } + validates :password, length: { minimum: 8 }, if: -> { new_record? || changes[:password_digest] } +end +``` + +The password validation only fires when the password digest changes - not on every save. If you add a `bio` column later and call `user.update(bio: "...")`, it won't reject the update because the password hasn't changed. Devise requires you to configure this separately; the generator gets it right by default. + +## `has_secure_password` - what it does and doesn't do + +`has_secure_password` adds three things to your model: + +1. **Password hashing via bcrypt.** The `password_digest` column stores a bcrypt hash. The plaintext password never touches the database. +2. **Virtual `password` and `password_confirmation` attributes.** They exist on the model instance but aren't persisted - only the digest is. +3. **An `authenticate` method.** Call `user.authenticate("mypassword")` and it returns `self` on match, `false` on mismatch. + +```ruby +# The bcrypt check behind user.authenticate(password) +user = User.find_by(email_address: "founder@example.com") +if user.authenticate(params[:password]) + # password matched - create a session +else + # password didn't match - redirect with error +end +``` + +What it doesn't do: rate limiting, password complexity rules, account locking, password history, or multi-factor auth. Each of those is a decision you make for your app, not a default you inherit from a gem. We'll cover the ones that matter for production below. + +## `authenticate_by` - the safer lookup + +Rails 7.1 shipped `authenticate_by` as a class method. The generator uses it everywhere. The difference from `find_by` + `authenticate` matters: + +```ruby +# What you'd write without authenticate_by - two queries and a timing leak +user = User.find_by(email_address: params[:email]) +if user&.authenticate(params[:password]) + # ... +end + +# authenticate_by - one query, constant-time comparison +if (user = User.authenticate_by(email_address: params[:email], password: params[:password])) + start_new_session_for user + redirect_to after_authentication_url +else + render :new, alert: "Invalid email or password." +end +``` + +`authenticate_by` runs the lookup and the bcrypt comparison in one step. It uses a constant-time comparison to prevent timing attacks on the password - someone can't measure response latency to figure out whether an email exists in the database. It returns the user on success and `nil` on failure with the same timing profile either way. + +If you're still writing `find_by` + `authenticate` in custom controllers, switch to `authenticate_by`. The generator already does. + +## Sessions: database-backed, not cookie-only + +The biggest architectural decision the generator makes is storing sessions in a database table instead of just a signed cookie. + +```ruby +# app/models/session.rb +class Session < ApplicationRecord + belongs_to :user + before_create { self.user_agent = Current.user_agent } + before_create { self.ip_address = Current.ip_address } +end + +# db/migrate/*_create_sessions.rb +create_table :sessions do |t| + t.references :user, null: false, foreign_key: true + t.string :ip_address + t.string :user_agent + t.timestamps +end +``` + +The cookie-only approach stores a signed user ID in the browser. It works. Until you need to log someone out of all their devices and realize you can't - the cookie is valid until it expires, period. The user changed their password after a breach? Every device with the old cookie is still logged in. + +Database-backed sessions fix this. Each login creates a `Session` row. The cookie stores a session token, not a user ID. To log out everywhere, you delete the rows: + +```ruby +# Log out from all devices +Current.user.sessions.destroy_all +``` + +That's it. Every device's cookie becomes invalid on the next request because the server-side session no longer exists. + +The session cookie itself is configured in the generator: + +```ruby +# app/controllers/concerns/authentication.rb +module Authentication + extend ActiveSupport::Concern + + included do + before_action :set_current_request_details + before_action :authenticate! + end + + private + + def authenticate! + if (session_record = Session.find_by(id: cookies.signed[:session_token])) + Current.session = session_record + else + redirect_to new_session_path, alert: "Please sign in." + end + end + + def start_new_session_for(user) + session = user.sessions.create! + cookies.signed.permanent[:session_token] = { value: session.id, httponly: true, secure: Rails.env.production? } + end + + def terminate_session + Current.session.destroy + cookies.delete(:session_token) + end +end +``` + +The `httponly` flag prevents JavaScript from reading the cookie - XSS can't steal the session token. The `secure` flag ensures the cookie only travels over HTTPS in production. The `signed` cookie prevents tampering. None of this requires configuration because it's spelled out in code you can read. + +## Password reset - the flow that trips up every hand-rolled auth + +The generator includes a complete forgot-password flow. The pattern it uses is the same one every production app eventually converges on: + +```ruby +# app/controllers/passwords_controller.rb - abridged +class PasswordsController < ApplicationController + allow_unauthenticated_access + + def new + # Show the "forgot password?" form + end + + def create + if (user = User.find_by(email_address: params[:email_address])) + PasswordsMailer.reset(user).deliver_later + end + # Always redirect with the same message - don't leak whether the email exists + redirect_to new_session_path, notice: "Check your email for reset instructions." + end + + def edit + # Validate the reset token from the email link + @user = User.find_by_token_for!(:password_reset, params[:token]) + rescue ActiveSupport::MessageVerifier::InvalidSignature + redirect_to new_session_path, alert: "Reset link is invalid or has expired." + end + + def update + @user = User.find_by_token_for!(:password_reset, params[:token]) + if @user.update(password_params) + redirect_to new_session_path, notice: "Password reset. Sign in." + else + render :edit, status: :unprocessable_entity + end + end +end +``` + +Three details that matter: + +**1. Don't leak email existence in the `create` action.** The generator redirects with the same message whether the email exists or not. If you say "email not found," an attacker can enumerate your user base. If you say "check your email" for both cases, they can't. The timings differ slightly - the email delivery path takes longer - so the generator always calls `redirect_to` outside the conditional. + +**2. The reset token expires.** The generator uses `generates_token_for` with an expiry window. The token embedded in the email link is only valid for a limited window (default 15 minutes). After that, `find_by_token_for!` raises `InvalidSignature` and the user starts over. + +**3. The token self-invalidates on password change.** The `generates_token_for` API signs the token against `password_salt`. When the user changes their password, bcrypt generates a new salt, and the old token becomes invalid automatically. Someone who intercepted the reset email can't reuse the same link after the password has been changed. + +## Email verification - confirm before allowing access + +The generator's confirmation flow ensures the email address belongs to the person who signed up: + +```ruby +# app/models/user.rb - additions for email verification +class User < ApplicationRecord + has_secure_password + + generates_token_for :email_confirmation, expires_in: 24.hours do + email_address + end + + before_create :generate_confirmation_token +end + +# app/controllers/confirmations_controller.rb +class ConfirmationsController < ApplicationController + allow_unauthenticated_access + + def new + # Show "check your email" page + end + + def create + @user = User.find_by_token_for!(:email_confirmation, params[:token]) + @user.confirm! + start_new_session_for @user + redirect_to root_path, notice: "Email confirmed." + rescue ActiveSupport::MessageVerifier::InvalidSignature + redirect_to new_session_path, alert: "Confirmation link is invalid or has expired." + end +end +``` + +The `generates_token_for` API is worth understanding because it replaces a lot of hand-rolled token logic. It produces a signed, time-limited token tied to a specific attribute. If the email address changes, the old token becomes invalid - even if it hasn't expired. This prevents someone from signing up with `attacker@evil.com`, changing their email to `victim@company.com`, and confirming the attacker's original email to get access. + +```ruby +# generates_token_for is worth understanding: +user = User.find_by(email_address: "new@example.com") +token = user.generate_token_for(:email_confirmation) +# token is valid for 24 hours, tied to this user's email_address + +user.update(email_address: "changed@example.com") +# The old token is now invalid - even if it hasn't expired. +# Changing the email changes what the token signs. +``` + +## Rate limiting - the generator doesn't do this, so here's what to add + +The generator ships without rate limiting. A determined attacker can hammer `/passwords` or `/sessions` with a script, enumerating users or brute-forcing passwords. The fix is minimal: + +```ruby +# config/initializers/rack_attack.rb +class Rack::Attack + # Throttle login attempts by IP - 5 per 20 seconds + throttle("logins/ip", limit: 5, period: 20.seconds) do |req| + req.ip if req.path == "/sessions" && req.post? + end + + # Throttle password reset requests by IP - 3 per hour + throttle("passwords/ip", limit: 3, period: 1.hour) do |req| + req.ip if req.path == "/passwords" && req.post? + end + + # Throttle signups by IP - 3 per hour + throttle("signups/ip", limit: 3, period: 1.hour) do |req| + req.ip if req.path == "/registrations" && req.post? + end + + # Blocklist after repeated violations + Rack::Attack.blocklist("repeat offenders") do |req| + Rack::Attack::Allow2Ban.filter(req.ip, maxretry: 10, findtime: 10.minutes, bantime: 1.hour) do + req.path.match?(/^\/(sessions|passwords|registrations)/) && req.post? + end + end +end +``` + +Add `gem "rack-attack"` to your Gemfile, add this initializer, and your auth endpoints are protected. The blocklist fires after 10 violations in 10 minutes from the same IP - a one-hour ban. Adjust the thresholds for your traffic patterns. + +## When to keep Devise + +The Rails 8 auth generator handles 80% of what Devise does. The 20% where Devise still wins: + +**OmniAuth integration.** The generator has no built-in OAuth. Adding Google, GitHub, or Apple sign-in requires wiring up the omniauth gem and callback controller yourself. Devise gives you `devise :omniauthable` with a one-line config per provider. If your app relies on social login, Devise saves you a day of integration work. + +**Account locking.** Brute-force protection beyond rate limiting - locking an account after N failed attempts, requiring admin or email-based unlock - is built into Devise via `devise :lockable`. The generator delegates this to `rack-attack`, which is IP-based, not account-based. An attacker with a botnet can rotate IPs and keep trying passwords against the same account despite rate limiting. If your threat model includes targeted credential attacks, Devise's account-level locking is the right primitive. + +**Trackable and timeoutable.** Devise ships with `trackable` (sign-in count, timestamps, IP address) and `timeoutable` (auto-logout after inactivity). The generator provides neither out of the box. You can add them - the `sessions` table already has `ip_address` and timestamps - but it's code you write instead of a config flag you set. + +**Enterprise features.** LDAP, SAML, two-factor via TOTP, password history enforcement, password complexity rules - Devise has gems for all of these. The generator is a starting point. If you need three enterprise auth features on day one, starting from the generator means building all three from scratch. + +If you're migrating from Devise, see [our Devise to Rails 8 auth migration guide](/blog/rails-8-authentication-generator-devise-migration/). If you're starting a new app and none of the Devise-only features above are requirements, use the generator. You can always add Devise later - removing it is harder than adding it. + +## Production hardening checklist + +Before deploying the generated auth to production: + +- [ ] **Enable `config.force_ssl = true`** in `config/environments/production.rb`. Session cookies marked `secure: true` (as the generator does) will be rejected by browsers over plain HTTP. Force SSL prevents misconfigurations from becoming security gaps. +- [ ] **Install and configure `rack-attack`** with the throttle rules above. The generator has no built-in rate limiting. +- [ ] **Configure ActionMailer** with a production email service (Postmark, SendGrid). Password reset and email confirmation emails need to actually arrive. Test this in staging before going live. +- [ ] **Set `Rails.application.config.session_store :cookie_store, expire_after: 2.weeks`**. The generator's session cookie is permanent by default. A 2-week idle timeout is a reasonable default - users who disappear for a month shouldn't stay logged in. +- [ ] **Audit `config/credentials.yml.enc`** for the `secret_key_base`. The signed cookies in `cookies.signed[:session_token]` depend on it. Rotating `secret_key_base` invalidates all sessions. +- [ ] **Add a `before_action :refresh_session`** to `ApplicationController` that bumps the session cookie's expiry on each authenticated request. Without this, a user who logs in, works for 30 minutes, and walks away will be logged out at exactly `expire_after` from their initial login - not from their last activity. +- [ ] **Set `SameSite=Strict`** on the session cookie if your app doesn't need cross-site requests. The generator uses Rails defaults. Add `same_site: :strict` to the `cookies.signed.permanent` line in `start_new_session_for` if your login form is on the same origin as your app. + +## What we shipped with the generator + +A client came to us in Q1 2026 with a Rails app running Devise. They had three custom modules: an email verification that didn't verify, a password reset that was broken in production (ActionMailer misconfiguration), and a session timeout that had been commented out in 2024 because it kept logging out mobile users. + +None of these were Devise's fault. Devise was doing exactly what the original developers had configured it to do three years ago. The problem was that nobody on the current team understood all 200 lines of `config/initializers/devise.rb`. The developer who wrote it had left. The remaining team was afraid to touch it. + +We migrated them to the generator in two days. The password reset started working because the code was explicit - `PasswordsMailer.reset(user).deliver_later` with a visible `ActionMailer` configuration, not a Devise module that dispatched emails through an invisible pipeline. The session timeout was four lines in `Authentication#authenticate!` that anyone on the team could read and adjust. The email verification worked because `generates_token_for :email_confirmation` is self-documenting. + +The takeaway isn't that Devise is bad. It's that authentication code you can't read is a liability. The Rails 8 generator gives you authentication code you can read - and therefore fix - without opening a gem's source. + +## The decision in 60 seconds + +Use the Rails 8 authentication generator when you want to own your auth code, when your needs are standard (email/password, password reset, email verification), and when your team values simplicity over feature completeness. + +Keep Devise when you need omniauth, account locking, or enterprise features on day one. Devise is battle-tested. The generator is newer. Both are valid choices. The question is whether the features you're not using are worth the lines of code you can't touch. + +For most new Rails 8 apps, the answer is no. + +*For the full migration guide from Devise, see [Devise to Rails 8 Auth: Migration Guide](/blog/rails-8-authentication-generator-devise-migration/). For deployment patterns, see [Rails 8 Docker Production Deployment](/blog/rails-8-docker-deployment-production-guide/) and our [Solid Queue migration guide](/blog/rails-8-solid-queue-migration-guide/). For performance tuning on the same stack, see [Falcon in Production: Benchmarks, Memory & Worker Count](/blog/falcon-web-server-production-tuning-benchmarks/). If you're weighing a Devise migration on a real codebase, [our Rails team can help](/services/app-web-development/).* diff --git a/content/blog/solid-queue-advanced-patterns-retries-concurrency/cover.png b/content/blog/solid-queue-advanced-patterns-retries-concurrency/cover.png new file mode 100644 index 000000000..6d563847c Binary files /dev/null and b/content/blog/solid-queue-advanced-patterns-retries-concurrency/cover.png differ diff --git a/content/blog/solid-queue-advanced-patterns-retries-concurrency/index.md b/content/blog/solid-queue-advanced-patterns-retries-concurrency/index.md new file mode 100644 index 000000000..68b5aaab6 --- /dev/null +++ b/content/blog/solid-queue-advanced-patterns-retries-concurrency/index.md @@ -0,0 +1,330 @@ +--- +title: "Solid Queue Advanced: Retries, Concurrency & Monitoring" +description: "Production patterns for Solid Queue: custom retry strategies, concurrency tuning, worker pool sizing, dead letter queues, and monitoring. Working Rails 8 config examples." +date: 2026-07-24 +draft: false +tags: ["rails", "solid-queue", "background-jobs", "performance", "production", "monitoring"] +canonical_url: "https://jetthoughts.com/blog/solid-queue-advanced-patterns-retries-concurrency/" +slug: "solid-queue-advanced-patterns-retries-concurrency" +cover_image: "cover.png" +metatags: + image: cover.png +cover_image_alt: "Dark technical cover for Solid Queue Advanced Patterns. JetThoughts ENGINEERING brand mark, Ruby on Rails 2026 pill, low-poly ruby gem, stat chips for retries/concurrency/alerts, PRODUCTION PATTERNS status indicator." +--- + +The Solid Queue migration guide shows you how to switch from DelayedJob or Sidekiq. It stops where the real work starts. + +Once the jobs are running, the questions change. How do you retry a failed job that depends on an external API that's down for 20 minutes, not 20 seconds? How many workers should you actually run? What do you monitor, and what do you alert on? + +These are the patterns we run on production Rails apps since migrating to Solid Queue in Rails 8. + +## Retry strategies beyond the defaults + +Solid Queue's default retry behavior is linear: a fixed delay, a max attempt count, done. That's fine for transient failures — a database connection blip, a Redis timeout. It breaks when the failure is upstream. + +### Exponential backoff with a ceiling + +An external API returns 429s. The default retry fires again in 5 seconds. Still rate-limited. Another 5 seconds. Still rate-limited. Your job burns through its retries in under a minute and lands in the dead set. The API recovers 3 minutes later. Your job is already dead. + +```ruby +# app/jobs/api_sync_job.rb +class ApiSyncJob < ApplicationJob + queue_as :api + + retry_on Faraday::TooManyRequestsError, + wait: ->(executions) { [ (2**executions * 5).seconds, 5.minutes ].min }, + attempts: 8 + + retry_on Faraday::TimeoutError, + wait: :exponentially_longer, + attempts: 5 + + discard_on ApiSyncJob::UnrecoverableError + + def perform(resource_id) + resource = Resource.find(resource_id) + ExternalApi.sync(resource) + end +end +``` + +The `[2**n * 5, 5.minutes].min` pattern gives you 5s, 10s, 20s, 40s, 80s, 160s, 320s, then caps at 300s. With 8 attempts, the job survives a 30-minute upstream outage and succeeds the moment the API comes back. Eight attempts is higher than you think you need until your first partner API outage. + +### The dead letter pattern + +Solid Queue has a built-in dead set at every failed job's final attempt, but there's no built-in way to inspect them. A job that dies silently is worse than a job that errors loudly. + +```ruby +# config/initializers/solid_queue.rb +Rails.application.config.after_initialize do # Available in Solid Queue 1.0+. For earlier versions, + # poll SolidQueue::Job.where.not(failed_at: nil) directly. + SolidQueue.on_dead_job do |job| + Rails.logger.error "DEAD JOB: #{job.class_name}##{job.job_id} — #{job.last_error&.truncate(200)}" + + # Alert on unexpected dead jobs + if job.failed_at > 10.minutes.ago + ActiveSupport::Notifications.instrument( + "dead_job.solid_queue", + job_class: job.class_name, + job_id: job.job_id, + error: job.last_error + ) + end + end +end +``` + +We pipe that `dead_job.solid_queue` event into our error tracker. A dead job fires a PagerDuty alert if the class is on the critical list (`PaymentProcessingJob`, `WelcomeEmailJob`). Batched maintenance jobs (`CleanupOldSessionsJob`) log and don't page — someone checks the dashboard Monday morning. + +### Job-specific concurrency keys + +Solid Queue inherits Active Job's concurrency controls. The most useful pattern we've found: per-resource uniqueness so you don't process the same record twice. + +```ruby +# app/jobs/process_order_job.rb +class ProcessOrderJob < ApplicationJob + queue_as :orders + + # Don't enqueue a second job for the same order while one is running + uniquely { |order_id:, **_| "order:#{order_id}" } # **_ ignores extra keyword args from Active Job + + retry_on ProcessOrderJob::InventoryNotAvailable, attempts: 3 + + def perform(order_id:, event: "created") + order = Order.find(order_id) + OrderProcessor.new(order).process!(event: event) + end +end +``` + +Without the uniqueness lock, a webhook that fires twice for the same event (Stripe does this) processes the order twice. The `uniquely` block prevents that — second enqueue silently drops while the first is running. + +## Concurrency tuning: stop guessing + +Solid Queue's `config/solid_queue.yml` lets you set `--max-threads` and `--processes`. The defaults are conservative. Here's how to find the right numbers for your workload. + +### Measure your job's I/O ratio + +Your concurrency ceiling is determined by how much time your jobs spend waiting. CPU-bound jobs cap at 2-3× CPU cores. I/O-bound jobs can go much higher. + +```ruby +# lib/tasks/solid_queue_profile.rake +namespace :solid_queue do + desc "Profile a job class to determine optimal concurrency" + task :profile, [:job_class] => :environment do |_, args| + job_class = args[:job_class].constantize + times = [] + + 20.times do |i| + start = Process.clock_gettime(Process::CLOCK_MONOTONIC) + job_class.perform_now + elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start + times << elapsed + puts "Run #{i + 1}: #{elapsed.round(3)}s" + end + + avg = times.sum / times.size + p95 = times.sort[(times.size * 0.95).ceil - 1] + puts "---" + puts "Average: #{avg.round(3)}s" + puts "P95: #{p95.round(3)}s" + + # Rule of thumb: threads per worker = (avg I/O wait / avg CPU time) + 1 + # If your jobs average 200ms of database wait and 10ms of Ruby CPU time, + # that ratio is 20:1 — you can run ~20 threads per worker without contention. + end +end +``` + +Run it: `rails solid_queue:profile[ProcessOrderJob]`. If your average job spends 80% of its time in database queries, API calls, or Redis operations, you can run 5-8 threads per worker without contention. If it's mostly Active Record serialization and ERB rendering, cap at 2-3. + +### The dispatch config we settled on + +```yaml +# config/solid_queue.yml +production: + dispatchers: + - polling_interval: 1 + batch_size: 500 + concurrency_maintenance_interval: 600 + + workers: + # High-priority queue: fast jobs, high throughput + - queues: [ default, mailers ] + threads: 8 + processes: 2 + polling_interval: 0.5 + + # Long-running queue: API calls, file processing + - queues: [ api, exports ] + threads: 12 + processes: 3 + polling_interval: 1.0 + + # Background maintenance: low urgency, batched + - queues: [ cleanup, analytics ] + threads: 4 + processes: 1 + polling_interval: 5.0 + + # Critical: payment processing, low concurrency for consistency + - queues: [ payments ] + threads: 2 + processes: 1 + polling_interval: 0.5 +``` + +**Why separate payment processing into its own worker with 2 threads?** Not because it's high-volume — because it's high-stakes. If a payment job crashes, we want the smallest possible blast radius. Two threads means at most two payments affected by a worker crash. The `uniquely` lock on `ProcessOrderJob` handles the rest. + +**Why `polling_interval: 0.5` on high-priority queues?** Solid Queue uses database polling, not Redis pub/sub. The default 1-second poll means a queued job waits up to 1 second before any worker picks it up. For `default` and `payments` queues, 0.5s halves that latency. The database load increase is negligible — the `SELECT ... FOR UPDATE SKIP LOCKED` query is cheap. + +### What happens when you over-subscribe + +We ran 16 threads per worker on a CPU-mixed workload. The database connection pool exhausted. Every thread tried to check out a connection, every connection was busy, and jobs started failing with `ActiveRecord::ConnectionTimeoutError` — not in the job body, but in Solid Queue's own dispatch logic. + +The symptom was cryptic: jobs appeared to succeed (no error in the job itself) but the dispatch logged `Error performing job: could not obtain a connection from the pool`. Solid Queue needs its own connection pool headroom. The formula: + +```ruby +# config/database.yml +production: + # Solid Queue's internal pool + your job's pool + # Each worker process needs: (threads + 2) connections + # 2 extra: 1 for the dispatcher, 1 for maintenance + pool: <%= (ENV.fetch('SOLID_QUEUE_MAX_THREADS', 5).to_i + 2) * + ENV.fetch('SOLID_QUEUE_PROCESSES', 2).to_i + + ENV.fetch('RAILS_MAX_THREADS', 5).to_i %> +``` + +If you run 3 worker processes with 8 threads each, that's (8+2) × 3 = 30 connections for Solid Queue, plus your web server's pool. If your database allows 60 total connections, the web server gets 30. If you run Sidekiq on the same box, subtract its pool too. + +## Monitoring: beyond the built-in dashboard + +Solid Queue ships with a dashboard at `/solid_queue` in development. It shows queue depths, job counts, and recent failures. It's useful. It's also not enough for production. + +### Metrics we actually alert on + +Three numbers matter. Everything else is noise. + +```ruby +# config/initializers/solid_queue_metrics.rb +Rails.application.config.after_initialize do + return unless Rails.env.production? + + # 1. Queue depth: if any queue is growing faster than it's draining, + # you're falling behind. Alert at 500+ backlog. + ActiveSupport::Notifications.subscribe("enqueue.active_job") do |*args| + event = ActiveSupport::Notifications::Event.new(*args) + queue_name = event.payload[:job]&.queue_name || "default" + + depth = SolidQueue::Job.where(queue_name: queue_name) + .where(finished_at: nil) + .count + + if depth > 500 + Rails.logger.warn "QUEUE DEPTH: #{queue_name} has #{depth} pending jobs" + end + end + + # 2. Job latency: how long between enqueue and start. + # >60s means your workers are saturated. + SolidQueue::Job.where(started_at: nil) + .where("created_at < ?", 1.minute.ago) + .in_batches do |batch| + batch.each do |job| + latency = Time.current - job.created_at + Rails.logger.warn "JOB LATENCY: #{job.class_name} waiting #{latency.round}s" if latency > 60 + end + end +end +``` + +Queue depth is the canary. If the `api` queue hits 500 pending and isn't draining, your upstream is down or your workers are saturated. Either way, adding more workers is the wrong first move — check the upstream first, then check your thread count. + +### The dashboard you actually need + +Solid Queue's data lives in the `solid_queue_jobs` table. A 5-minute rake task gives you more signal than the built-in dashboard: + +```ruby +# lib/tasks/solid_queue_stats.rake +namespace :solid_queue do + desc "Print queue health summary" + task stats: :environment do + puts "=" * 60 + puts "Solid Queue Health — #{Time.current.strftime('%Y-%m-%d %H:%M UTC')}" + puts "=" * 60 + + # Per-queue breakdown + SolidQueue::Job.group(:queue_name) + .where(finished_at: nil) + .count + .sort_by { |_, count| -count } + .each do |queue, count| + status = count > 100 ? "⚠️" : "✅" + puts "#{status} #{queue}: #{count} pending" + end + + # Failure rate (last hour) + total = SolidQueue::Job.where("created_at > ?", 1.hour.ago).count + failed = SolidQueue::Job.where("created_at > ?", 1.hour.ago) + .where.not(finished_at: nil) + .where("finished_at IS NULL OR last_error IS NOT NULL") + .count + + if total > 0 + failure_rate = (failed.to_f / total * 100).round(2) + status = failure_rate > 5 ? "🚨" : failure_rate > 1 ? "⚠️" : "✅" + puts "#{status} Failure rate (1h): #{failure_rate}% (#{failed}/#{total})" + end + + # Oldest pending job + oldest = SolidQueue::Job.where(finished_at: nil) + .order(:created_at) + .first + if oldest + age = (Time.current - oldest.created_at).round + status = age > 300 ? "🚨" : "✅" + puts "#{status} Oldest pending: #{oldest.class_name} (#{age}s)" + end + + # Dead jobs today (failed_at is populated when retries exhausted) + dead_today = SolidQueue::Job.where.not(failed_at: nil) + .where("failed_at > ?", Date.today.beginning_of_day) + .count + puts "#{dead_today > 0 ? '⚠️' : '✅'} Dead jobs today: #{dead_today}" + end +end +``` + +Wire this to a cron job every 5 minutes and log the output. If the summary ever prints 🚨, someone should look. + +### The Sidekiq migration gotcha + +Solid Queue stores jobs in PostgreSQL. If you're coming from Sidekiq, your Redis memory was your job buffer. Under Solid Queue, that buffer is now rows in `solid_queue_jobs`. A queue with 500,000 pending jobs is a 500,000-row table. PostgreSQL handles it fine — until it doesn't. + +The vacuum: `solid_queue_jobs` accumulates finished jobs unless you configure retention. By default, Solid Queue keeps finished jobs for `preserve_finished_jobs: true` (indefinitely). In development, fine. In production, that table grows unbounded. + +```ruby +# config/initializers/solid_queue.rb +Rails.application.config.solid_queue.preserve_finished_jobs = false +Rails.application.config.solid_queue.clear_finished_jobs_after = 7.days +``` + +Set `clear_finished_jobs_after` to something your debugging workflow tolerates. Seven days is enough to investigate a weekend failure on Monday. At 100,000 jobs/day, that's 700,000 rows — a few hundred MB. Fine. + +If you need to debug a job that ran 3 weeks ago, pull the logs, not the database row. + +We learned this the hard way. A client migrated from Sidekiq to Solid Queue in April 2026 — 80K jobs/day, standard RDS instance. The `preserve_finished_jobs` default ate 8GB of disk in the first week before their monitoring caught it. After setting `clear_finished_jobs_after: 7.days`, the table stabilized at 700K rows and 500MB. The fix was two config lines. The diagnosis took a morning of digging through a bloated jobs table. + +## When Solid Queue isn't enough + +Solid Queue replaces Sidekiq for 80% of Rails apps. The 20% where it doesn't: + +**You process more than 500,000 jobs/hour.** PostgreSQL's `SELECT ... FOR UPDATE SKIP LOCKED` works at scale, but at some point Redis's pub/sub delivers lower latency and higher throughput. The exact crossover depends on your database hardware, but we've seen performance degradation start around 500K jobs/hour on standard RDS instances. + +**You need scheduled jobs with sub-second precision.** Solid Queue polls the database. The minimum polling interval is 0.1 seconds. If you need a job to run exactly 50ms after another job completes, use Sidekiq with Redis. + +**You run PostgreSQL on a single small instance.** Solid Queue adds constant read/write load to your primary database. If your RDS is already at 80% CPU serving web traffic, adding a job queue on top will push it over. Either size up or run Sidekiq on Redis. + +For the rest of us — the apps processing 10K-200K jobs/hour, running on reasonable hardware — Solid Queue eliminates a Redis dependency, simplifies the stack, and gives you job visibility in the same database you already query. The trade-off is real but the simplification wins for most workloads. + +*For the migration guide and benchmarks, see [Rails 8 Solid Queue: Migration Guide for Production Apps](/blog/rails-8-solid-queue-migration-guide/). For deployment patterns that pair well with Solid Queue, see [Falcon in Production: Benchmarks, Memory & Worker Count](/blog/falcon-web-server-production-tuning-benchmarks/). For replacing Devise in the same stack, see [Rails 8 Authentication Generator: Complete Guide](/blog/rails-8-authentication-generator-complete-guide/). If you're running Solid Queue at scale, [our Rails team can help](/services/app-web-development/).* diff --git a/docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md b/docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md new file mode 100644 index 000000000..48c9e8053 --- /dev/null +++ b/docs/projects/2510-seo-content-strategy/20-29-strategy/20.08-content-plan-data-driven-q3-2026.md @@ -0,0 +1,630 @@ +# Content Plan — Data-Driven 3-Stream: Rails Technical + Snippet Hygiene + Founder (Q3 2026) + +**Purpose**: Grow organic search traffic by doubling down on what GSC data proves +works (technical Rails content), fixing the 100K+ impressions wasted on 0-CTR +pages, and maintaining the ICP-E founder pipeline at reduced cadence. +**Parent project**: 2510-seo-content-strategy +**Supersedes**: `20.07-content-plan-icp-e-q2-2026.md` — keeps founder/ICP-E angle +but corrects stream weighting based on 3 months of GSC performance data +**GSC data source**: `jetthoughts.com-Performance-on-Search-2026-07-19/` (Apr 17 +– Jul 16, 2026, 3-month window) +**Created**: 2026-07-19 +**Last Updated**: 2026-07-19 + +--- + +## 📊 GSC BASELINE: What the Data Actually Says + +### Overall health (Apr 17 – Jul 16, 2026) + +| Metric | Value | +|---|---| +| Total Clicks | 449 | +| Total Impressions | 478,131 | +| Overall CTR | 0.09% | +| Avg Position | 15.9 | +| Desktop / Mobile split | 376 (84%) / 73 (16%) | + +**CTR trend**: Declining. Was ~0.2% in April–May, now ~0.1%. Average position +declined from ~13 to ~15. + +### Top pages by clicks — ALL technical Rails/Ruby/CSS + +| Page | Clicks | Impr | CTR | +|---|---|---|---| +| falcon-web-server-async-ruby | **54** | 6,078 | 0.89% | +| Homepage (branded) | 48 | 727 | 6.6% | +| tailwindcss triangles CSS | 27 | 1,080 | 2.5% | +| rails-8-solid-queue | 19 | 2,408 | 0.79% | +| ruby-3-4-yjit | 17 | 7,575 | 0.22% | +| elixir-ai-integration | 13 | 7,408 | 0.18% | +| careers page | 12 | 278 | 4.32% | +| horizontal-center CSS | 10 | 3,134 | 0.32% | +| pay gem stripe rails | 10 | 2,469 | 0.41% | +| self-promote-reddit | 9 | 35,048 | 0.03% | + +**Finding**: Every single click-driving page is technical content (Rails, Ruby, +CSS, deployment). Zero ICP-E founder posts appear in the top 50 pages by clicks. + +### Top pages by impressions — massive waste on AI framework content + +| Page | Impr | Clicks | CTR | +|---|---|---|---| +| elital subdomain "upwork login" | 53,823 | 5 | 0.01% | +| langgraph-workflows | **44,088** | 3 | 0.01% | +| self-promote-reddit | 35,048 | 9 | 0.03% | +| laravel-11-migration | 19,238 | 2 | 0.01% | +| rails-testing-best-practices | 12,373 | 4 | 0.03% | +| autogen-crewai-langgraph | 12,233 | **0** | 0% | +| crewai-multi-agent | 11,735 | **0** | 0% | +| langchain-memory-systems | 7,877 | **0** | 0% | + +**Finding**: The top 6 non-elital pages account for **107,544 impressions +(22% of all traffic)** and **9 clicks (2% of all clicks)**. These are mostly the +original multi-language AI content that predates the ICP-E pivot. The titles +and meta descriptions fail to match searcher intent. + +### Top queries — 43K impressions wasted on "upwork login" + +| Query | Impr | Clicks | +|---|---|---| +| upwork login | 43,543 | 2 | +| vertical ai agents | 2,300 | 0 | +| upwork log in | 1,948 | 0 | +| agent workflow state machine | 1,370 | 0 | +| upwork.com login | 1,324 | 0 | +| laravel vs datadog | 699 | 0 | +| language-specific ruby gem queries | 1,000+ | 1-2 | + +### ICP-E founder content — not ranking yet + +After 3 months and 14+ published founder-focused posts, **none** appear in the +top 50 pages by impressions or clicks. Note: founder posts published in May +2026 have only had ~60 days to rank. Organic search for new content on a domain +with established technical authority can take 6-12 months. These posts may still +gain traction; the data says they haven't YET. Re-evaluate at the 6-month mark +(October 2026) before concluding they'll never rank. Meanwhile, reduce cadence +and use LinkedIn distribution as the primary channel for founder content. + +--- + +## 🔴 THREE CRITICAL PROBLEMS + +### Problem 1: Elital subdomain dumping 53K+ junk impressions + +`elital.jetthoughts.com` has an "upwork login" post indexed and ranking for +navigational queries. Combined with main-site upwork posts, this wastes +~48K impressions that will never convert. The April 29 review flagged 1,500 +impressions — it's now 29x worse. + +**Fix**: Block elital subdomain from indexing or rewrite/remove the upwork +content. See Section: Immediate Actions. + +### Problem 2: Legacy AI content gets massive impressions, zero clicks + +The pre-pivot langchain/crewai/autogen posts collectively get 75K+ impressions +with near-zero CTR. These are well-written posts ranking on page 1-2 for +relevant queries — the titles and meta descriptions simply fail to promise +what searchers want. These posts were built for the old "junior developer AI +tutorial" audience, not the current ICP. + +**Fix**: Rewrite titles and meta descriptions to match actual searcher intent. +See Snippet Hygiene section below. + +### Problem 3: Content plan bet on founder content that doesn't rank + +The 20.07 plan allocated 5/month to founder content and only 2/month to Rails. +The GSC data shows Rails content is the **only** content that drives clicks. +The founder content may be a 6-12 month play, but the plan is starving the +proven winner. + +**Fix**: Invert the weighting. Rails technical content gets 3/week. Founder +content gets 1/week. Snippet hygiene is the immediate priority. + +--- + +## 🎯 STRATEGIC PIVOT: Technical Authority First + +### What the data proves + +| Content Type | Clicks/Post (3mo) | Impr/Post | CTR | +|---|---|---|---| +| Rails deployment (Falcon, Solid Queue, Kamal) | **27** | 4,300 | 0.6% | +| Ruby/YJIT performance | 17 | 7,575 | 0.2% | +| CSS how-to | 13 | 2,100 | 1.3% | +| AI framework (LangChain, CrewAI) | 0.8 | 14,200 | 0.005% | +| ICP-E founder content | **0** | **0** | N/A | + +**The site's authority is in Rails, Ruby, and CSS.** Google trusts +jetthoughts.com for technical content about these topics. Publishing into +other topics (founder advice, AI frameworks) is competing against the site's +own authority profile. + +### New thesis + +> **Build search traffic through Rails technical authority, then convert +> that traffic with founder-relevant CTAs and internal links.** Instead of +> publishing founder content that won't rank, publish Rails content that +> WILL rank and use it as a funnel to founder-focused service pages. + +--- + +## 📅 3-STREAM CONTENT PLAN (Jul 21 – Oct 12, 2026) + +### Stream weighting + +| Stream | Emoji | Frequency | Target | Role | +|---|---|---|---|---| +| **Technical Rails** | 💎 | 2/week (3 aspirational) | CTOs, senior Rails devs | Search traffic + authority | +| **Snippet Hygiene** | 🔧 | Immediate (1 week sprint) | Existing 0-CTR pages | Reclaim wasted impressions | +| **Founder / ICP-E** | 🧑‍💼 | Every other week (6 total) | Non-technical founders | Lead gen + LinkedIn | +| **Content Consolidation** | 📦 | 1-2 days in sprint week | Cannibalized Rails posts | Merge + 301 redirect | + +**Cadence**: Target 2 Rails/week as the floor, 3/week as aspirational. Founder +posts move to every other week (6 posts, not 12). Total: 2-3/week (same as +20.07's target, but with Rails getting 80% of the slots instead of 20%). +Snippet hygiene is a one-time sprint, not ongoing. + +**Why not 4/week**: The 20.07 plan described 2-3/week as "sustainable without +quality dilution." Technical Rails posts require working code examples, +benchmarks, and screenshots. At 3 Rails/week, quality will decay. The plan +below schedules 2/week as committed and notes where a 3rd post can be added +if capacity allows. + +--- + +## 🔧 STREAM 1: SNIPPET HYGIENE (IMMEDIATE — 1 Week Sprint) + +Fix the 5 worst 0-CTR pages. Combined potential: 107K impressions × +conservative 0.5% CTR = **535 clicks** from rewrites alone — more than the +entire site's 3-month clicks (449). + +### Rewrite #1: langgraph-workflows (44,088 impr, 0.01% CTR, pos 11.5) + +**Current title**: "Mastering LangGraph: Building Complex AI Agent Workflows +with State Machines" + +**Current description**: "Learn to build production-ready AI agent workflows +with LangGraph's state machines. Master node caching, deferred nodes, +human-in-the-loop patterns, and map-reduce workflows with complete code +examples." + +**Problem**: "Mastering" signals advanced/academic. "State Machines" is the +wrong hook — searchers typing "langgraph" want a tutorial or a "what is +LangGraph" explainer. The query data confirms top queries include "agent +workflow state machine reuse across features" (1,370 impr) — these are +developers looking for reference docs, not a "mastering" post. + +**Proposed title** (55 chars): `LangGraph Tutorial: Build AI Agents with +State Machines (Python)` + +**Proposed description** (158 chars): `Complete LangGraph tutorial with +working Python examples. Learn nodes, edges, checkpoints, human-in-the-loop, +and map-reduce patterns. From first graph to production agent.` + +**Win condition**: 0.5% CTR on 44K impr = 220 clicks/3mo. + +--- + +### Rewrite #2: autogen-crewai-langgraph (12,233 impr, 0% CTR, pos 14.1) + +**Current title**: "LangGraph vs CrewAI vs AutoGen: Open Source Alternatives +2025" + +**Current description**: "Three open source agent frameworks compared: +LangGraph state machines, CrewAI role-based crews, AutoGen conversations. +License, deployment, and 2025 maturity." + +**Problem**: This was already rewritten per the April 29 review suggestions! +The year-stamp "2025" is now stale (we're in mid-2026). The description is +dry feature-list copy. Zero clicks means searchers see the title, the year, +and skip. + +**Proposed title** (60 chars): `LangGraph vs CrewAI vs AutoGen: Which AI +Agent Framework? (2026)` + +**Proposed description** (160 chars): `Compare LangGraph, CrewAI, and AutoGen +head-to-head: code examples, production readiness, licensing, community +size. Pick the right framework for your AI agent project.` + +**Win condition**: 0.5% CTR on 12.2K impr = 61 clicks/3mo. + +--- + +### Rewrite #3: crewai-multi-agent (11,735 impr, 0% CTR, pos 10.1) + +**Current title** (56 chars): "CrewAI Hierarchical Agents: Manager-Worker +Orchestration" + +**Current description**: "Build hierarchical CrewAI crews with a manager +agent delegating to specialized workers. Code examples for orchestration, +task routing, and current patterns." + +**Problem**: This title is actually decent. 56 chars, action-oriented. But +0% CTR at position 10.1 means the meta description or the SERP snippet +isn't compelling enough. The post may also be too long (543 lines). + +**Proposed title** (58 chars): `CrewAI Hierarchical Agents: Manager-Worker +Pattern (Code Examples)` + +**Proposed description** (155 chars): `Step-by-step CrewAI tutorial with +working Python code. Set up a manager agent, delegate tasks to specialized +workers, handle failures. Includes v0.98+ patterns.` + +**Additional fix**: Add a TL;DR summary block at the top of the post body so +Google can extract a richer snippet. + +**Win condition**: 0.5% CTR on 11.7K impr = 58 clicks/3mo. + +--- + +### Rewrite #4: langchain-memory (7,877 impr, 0% CTR, pos 11.1) + +**Current title**: "Building Stateful Conversational AI with LangChain Memory +Systems" + +**Current description**: "Master LangChain memory systems to build +production-ready conversational AI with Python. Learn short-term, long-term, +entity memory with PostgreSQL/Redis persistence, and real-world +implementation patterns." + +**Problem**: "Stateful Conversational AI" is too abstract. Developers search +"langchain memory" or "langchain conversation history" — they want a +specific how-to. + +**Proposed title** (55 chars): `LangChain Memory: Conversation History with +Python (PostgreSQL + Redis)` + +**Proposed description** (158 chars): `How to add memory to LangChain agents. +Working Python examples for short-term, long-term, and entity memory. +PostgreSQL and Redis persistence. Real production patterns.` + +**Win condition**: 0.5% CTR on 7.9K impr = 39 clicks/3mo. + +--- + +### Rewrite #5: laravel-11-migration (19,238 impr, 0.01% CTR, pos 12.0) + +**Current title** (67 chars): "Laravel 11 Migration Guide: Complete +Production Deployment Strategies" + +**Current description**: "Master the migration from Laravel 10 to Laravel 11. +Complete guide with breaking changes analysis, step-by-step migration, +testing strategies, and zero-downtime production deployment." + +**Problem**: The title is serviceable. 0.01% CTR at position 12.0 suggests +the SERP snippet Google is pulling isn't matching intent. The post is 1,595 +lines — one of the longest on the site. Google may be pulling a poor snippet. + +**Proposed title** (57 chars): `Laravel 11 Upgrade Guide: Step-by-Step +Migration (Zero Downtime)` + +**Proposed description** (155 chars): `Upgrade Laravel 10 to 11 without +downtime. Breaking changes checklist, step-by-step migration commands, +testing strategy, and production deployment plan.` + +**Additional fix**: Add a prominent "Quick Start" section at the top with the +5 commands needed to start the migration. Google often extracts code-like +content for featured snippets. + +**Win condition**: 0.3% CTR on 19.2K impr = 57 clicks/3mo. + +--- + +### Combined snippet hygiene impact + +| Page | Current Clicks | Projected Clicks (0.5% CTR) | +|---|---|---| +| langgraph-workflows | 3 | 220 | +| autogen-crewai | 0 | 61 | +| crewai-multi-agent | 0 | 58 | +| langchain-memory | 0 | 39 | +| laravel-11-migration | 2 | 57 | +| **Total** | **5** | **435** | + +**435 clicks from rewrites alone = 97% of the entire site's current 3-month +clicks.** This is the highest-leverage action in the entire plan. + +--- + +## 💎 STREAM 2: TECHNICAL RAILS DEEP-DIVES (3/Week) + +Build on the #1 performing content type. Every post targets a specific +low-competition, high-intent technical query. Posts are developer-targeted +— terse, code-heavy, thoughtbot-style. + +### Publishing cadence + +3 posts/week × 12 weeks = 36 posts. Mix of: +- **Expanders** (8 posts): extend existing top-performing posts into series +- **Fillers** (18 posts): target queries we're already ranking for (pos 5-15) +- **New territory** (10 posts): Rails 8.x features, Ruby 3.4+, Kamal 2 + +### Phase 1: Expand the Winners (Weeks 1-2: Jul 21 – Aug 2) + +These extend existing top-click posts into series. They rank faster because +Google already trusts the parent post. Target: 2/week committed, 3rd post +if capacity. + +| # | Publish | Topic | Slug | Parent Post | +|---|---|---|---|---| +| E1 | Jul 22 | Falcon Web Server: Production Tuning & Benchmarks | `falcon-web-server-production-tuning-benchmarks` | falcon-web-server (54 clicks) | +| E2 | Jul 24 | Solid Queue Advanced Patterns: Retries, Concurrency, Monitoring | `solid-queue-advanced-patterns-retries-concurrency` | rails-8-solid-queue (19 clicks) | +| E3 | Jul 28 | Solid Cache Deep Dive: When Redis Still Wins | `solid-cache-vs-redis-production-benchmarks` | rails-8-solid-cache (6 clicks) | +| E4 | Jul 30 | Kamal 2 Multi-Server Deployment: Complete Guide | `kamal-2-multi-server-deployment-complete-guide` | kamal-2.0 (7 clicks) | +| *E5* | *Aug 4* | *Pay Gem Deep Dive: Subscriptions, Webhooks, Multi-Tenant* | *`pay-gem-subscriptions-webhooks-multi-tenant`* | *pay gem (10 clicks) — aspirational* | + +Note: Merged E1+E2 (two Falcon posts in week 1 was redundant). E5 is +aspirational — ship only if the first 4 are done with quality. + +### Phase 2: Fill the Ranking Gaps (Weeks 3-8: Aug 5 – Sep 13) + +Target queries where we rank position 5-15 with existing content but don't +have a dedicated post. These have proven search demand. + +#### Rails 8.x Production Stack (6 posts) + +| # | Publish | Topic | Slug | Target Query | +|---|---|---|---|---| +| F1 | Aug 5 | Propshaft Complete Migration Guide (Sprockets to Propshaft) | `propshaft-migration-complete-guide-rails-8` | propshaft vs sprockets (8,832 impr, pos 12.8) | +| F2 | Aug 7 | Rails 8 Authentication Generator: Complete Guide | `rails-8-authentication-generator-complete-guide` | rails authentication generator (671 impr) | +| F3 | Aug 10 | Rails 8 Docker Deployment: Production Patterns | `rails-8-docker-production-patterns-2026` | rails docker deployment (2,447 impr) | +| F4 | Aug 12 | Rails 8 Sharding: Horizontal Scaling Guide | `rails-8-sharding-horizontal-scaling-guide` | rails sharding | +| F5 | Aug 14 | Active Job Retries and Concurrency in Rails 8 | `active-job-retries-concurrency-rails-8` | active job rails (1,293 impr) | +| F6 | Aug 17 | Rails 8 Custom Compression for Encrypted Data | `rails-8-custom-compression-encrypted-data` | rails compression encrypted (86 impr, but high CTR) | + +#### Ruby Performance & YJIT (4 posts) + +| # | Publish | Topic | Slug | +|---|---|---|---| +| F7 | Aug 19 | Ruby 3.4 YJIT: Real-World Production Benchmarks | `ruby-3-4-yjit-production-benchmarks-2026` | +| F8 | Aug 21 | Ruby Memory Optimization: Production Patterns for Large Apps | `ruby-memory-optimization-production-patterns` | +| F9 | Aug 24 | ActiveRecord Query Performance: 10 Patterns That Cut Query Time | `activerecord-query-performance-patterns-2026` | +| F10 | Aug 26 | Ruby Concurrency Patterns: Ractors, Fibers, Threads (2026) | `ruby-concurrency-patterns-ractors-fibers-threads` | + +#### Rails Deployment & DevOps (5 posts) + +| # | Publish | Topic | Slug | +|---|---|---|---| +| F11 | Aug 28 | Kamal + GitHub Actions: Complete CI/CD Pipeline | `kamal-github-actions-cicd-pipeline-rails` | +| F12 | Aug 31 | Zero-Downtime Deploys with Kamal 2 | `zero-downtime-deploys-kamal-2-rails` | +| F13 | Sep 2 | Rails Logging in Production: Structured Logging with Lograge | `rails-production-logging-structured-lograge` | +| F14 | Sep 4 | Rails Health Checks: Monitoring Patterns for Production | `rails-health-checks-monitoring-production` | +| F15 | Sep 7 | Database Migrations Without Downtime (Rails + PostgreSQL) | `database-migrations-zero-downtime-rails-postgresql` | + +#### Ruby/Rails AI (3 posts — bridge to ICP-E) + +| # | Publish | Topic | Slug | +|---|---|---|---| +| F16 | Sep 9 | LangChain.rb in Production: RAG with pgvector | `langchain-ruby-rag-pgvector-production` | +| F17 | Sep 11 | Ruby OpenAI: Streaming, Function Calling, Cost Control | `ruby-openai-streaming-function-calling` | +| F18 | Sep 14 | Building a Rails Chatbot: Complete Guide | `rails-chatbot-complete-guide-2026` | + +### Phase 3: New Territory (Weeks 9-12: Sep 16 – Oct 12) + +| # | Publish | Topic | Slug | +|---|---|---|---| +| N1 | Sep 16 | Turbo 8 Morphing: 5 Pitfalls in Production | `turbo-8-morphing-production-pitfalls` | +| N2 | Sep 18 | Thruster + Kamal 2: Zero-Config Rails Deployment | `thruster-kamal-2-zero-config-rails` | +| N3 | Sep 21 | Rails 7 EOL: 10 Security Patches You Missed + Upgrade Plan | `rails-7-eol-security-patches-upgrade-guide` | +| N4 | Sep 23 | Stimulus in 2026: Modern Patterns & Best Practices | `stimulus-modern-patterns-best-practices-2026` | +| N5 | Sep 25 | Hotwire Native: Rails Mobile Apps Without React Native | `hotwire-native-rails-mobile-apps` | +| N6 | Sep 28 | Rails API-Only: Complete Guide for JSON APIs | `rails-api-only-complete-guide-json` | +| N7 | Sep 30 | RSpec vs Minitest in 2026: Honest Comparison | `rspec-vs-minitest-2026-comparison` | +| N8 | Oct 2 | Rails Testing Handbook: System Tests, Fixtures, Factories | `rails-testing-handbook-system-tests-factories` | +| N9 | Oct 5 | PostgreSQL for Rails Developers: Beyond the ORM | `postgresql-rails-developers-beyond-orm` | +| N10 | Oct 7 | Rails Observability: OpenTelemetry, Metrics, Traces | `rails-observability-opentelemetry-metrics` | + +--- + +## 🧑‍💼 STREAM 3: FOUNDER / ICP-E CONTENT (1/Week) + +Keep the ICP-E pipeline alive at reduced cadence. These posts are for LinkedIn +distribution, email nurture, and the 6-12 month organic play. Each post +bridges technical content to founder decision-making. + +### Selection criteria for founder posts + +1. **Does a non-technical founder actually search for this?** If no, it's a + Rails post. +2. **Can we link to a technical post from it?** Every founder post must + internally link to 2+ Rails technical posts. This builds the funnel. +3. **Can we test it on LinkedIn first?** Per the 20.07 LinkedIn validation + sprint approach — validate hooks before writing full posts. + +### Q3 Founder Posts (6 posts — every other week) + +| # | Publish | Topic | Slug | ICP-E Pain | +|---|---|---|---|---| +| ICP1 | Jul 29 | The Friday Demo Rule: How Non-Technical Founders Catch Fake Progress | `friday-demo-rule-founder-progress` | Progress Mirage | +| ICP2 | Aug 11 | Who Owns Your GitHub, AWS, and Database? (Checklist) | `github-aws-database-ownership-checklist` | IP Loss | +| ICP3 | Aug 25 | Should You Salvage the Codebase or Rebuild? (Decision Tree) | `salvage-vs-rebuild-software-project` | Salvage vs Rebuild | +| ICP4 | Sep 8 | Dev Shop Contract: Who Owns the Code? | `dev-shop-contract-code-ownership` | IP Loss | +| ICP5 | Sep 22 | Rails 8 Cost Savings for Non-Technical Founders | `rails-8-cost-savings-founder-guide` | Budget Creep | +| ICP6 | Oct 6 | Your Dev Shop Uses AI. Here Are 5 Questions to Ask. | `dev-shop-uses-ai-5-questions` | AI Quality | + +Reduced from 12 to 6. Validate LinkedIn hook performance before writing each +post. If a LinkedIn post on the same topic gets <5 qualified founder comments, +swap the blog topic for one that does. + +### Founder post structure (mandatory) + +1. **Hook**: Founder pain, plain English, specific scenario +2. **The problem**: What the founder is experiencing, with concrete examples +3. **The solution**: Actionable, copy-paste-able (scripts, templates, + checklists) +4. **Technical backlinks**: Link 2+ Rails technical posts as "how we do this" + proof points +5. **CTA**: Lead magnet (template, checklist, audit offer) + +### Bidirectional funnel requirement + +**Rails posts → founder posts**: Every Rails technical post must include at +least one internal link to a relevant founder post or service page. This is +how the funnel works: technical content brings the traffic, internal links +move readers to conversion pages. Without this, the plan generates traffic +that never converts. + +**Founder posts → Rails posts**: Already required (see #4 above). + +**Both → service pages**: Where relevant, link to `/services/fractional-cto/`, +`/services/startup-cto-consulting/`, or `/services/app-web-development/`. + +--- + +## 🚨 IMMEDIATE ACTIONS (This Week) + +### Priority 0: Elital subdomain cleanup + +The `elital.jetthoughts.com` subdomain is polluting the domain's topical +authority with 53K+ junk upwork-login impressions. + +**Investigation first** (30 min): +1. Check if elital is served by the same Hugo build (look for an elital + config or content directory) — if yes, it's a simple config change. +2. If separate hosting: `dig elital.jetthoughts.com` to find the DNS target. + Check if Cloudflare Worker can intercept it (the existing worker at + `seo/cloudflare_worker.js` already handles redirects). + +**Fix options** (pick based on investigation): +- **If same Hugo build**: Add `noindex: true` to elital frontmatter, or + add `Disallow: /blog/mastering-your-upwork-login` to `robots.txt`. +- **If Cloudflare**: Add a Worker route that returns `X-Robots-Tag: noindex` + for all `elital.jetthoughts.com/*` requests. +- **If separate hosting**: Investigate access. If inaccessible, submit a + GSC removal request for the elital subdomain prefix. + +4. After fix: submit removal request in GSC for the elital subdomain. +5. Verify in 14 days: elital impressions should drop >90%. + +### Priority 1: Rewrite 5 worst 0-CTR titles/metas — frontmatter only (1 hour) + +Edit only `title` and `description` in frontmatter for all 5 posts. Keep +post bodies intact — the content is good, the snippet is bad. Ship same day. + +### Priority 2: Add TL;DR blocks to 2 posts needing body changes (1-2 hours) + +Rewrites #3 (crewai) and #5 (laravel) need body-content additions for maximum +impact. Add a TL;DR or "Quick Start" section at the top. These are follow-up +changes — ship Priority 1 first, then do these. + +```markdown +## TL;DR + +LangGraph is a Python library for building AI agent workflows as state +machines. You define nodes (actions), edges (transitions), and checkpoints +(save points). This post covers... + +**Quick start**: `pip install langgraph` → define your graph → `graph.invoke()` +``` + +### Priority 3: Consolidate cannibalized Rails performance posts + +GSC data shows 4 posts competing for the same "Rails performance" queries: +- `ruby-on-rails-performance-optimization-patterns-2026` (690 impr, 1 click) +- `ruby-on-rails-performance-optimization-patterns-for-2026` (261 impr, 1 click) +- `maximizing-efficiency-proven-strategies-for-ruby-on-rails-performance` (828 impr, 0 clicks) +- `best-practices-for-optimizing-ruby-on-rails-performance` (118 impr, 0 clicks) + +**Action**: Pick the best-performing post as canonical. Merge unique content +from the other 3 into it. 301-redirect the 3 weaker posts to the canonical. +This consolidates ranking signals instead of splitting them across 4 URLs. + +### Priority 4: Fix `/services/` page title + +The April 29 review flagged that `/services/_index.md` has title "Optimize & +Empower Products At Any Stage" — contains zero stack keywords. GSC confirms +202 impressions, 0 clicks. The services page is invisible for service-intent +queries. + +**Action**: Rewrite frontmatter: +- `title`: `Ruby on Rails & React Development Services | JetThoughts` +- `description`: `Ruby on Rails and React development services for funded + startups. 8-12 week MVPs, fractional CTOs, 95% client retention.` + +This is a 10-minute change that fixes the highest-leverage page on the site. + +--- + +## 📊 PERFORMANCE TRACKING + +### GSC review cadence + +- **Weekly**: Check CTR on the 5 rewritten posts (did it move above 0.1%?) +- **Monthly**: Pull full GSC export, compare to baseline, update this plan +- **Quarterly**: Full strategy review (Oct 12, 2026) + +### Success metrics (3-month target) + +| Metric | Current (Jul 2026) | Target (Oct 2026) | +|---|---|---| +| Monthly clicks | ~150 | 400-600 | +| Overall CTR | 0.09% | 0.2%+ | +| 0-CTR pages (top 20 by impr) | 14 | 4 | +| Rails technical posts ranking top 10 | 8 | 25+ | +| Founder posts with >0 clicks | 0 | 5+ | + +### Traffic projection + +| Stream | Est. clicks/3mo | +|---|---| +| Snippet hygiene (5 rewrites) | 435 | +| Rails technical (24 posts × 15 clicks avg) | 360 | +| Content consolidation (4→1 Rails perf posts) | 30 | +| `/services/` page fix | 25 | +| Founder content (6 posts × 5 clicks avg) | 30 | +| **Projected total** | **~880** | +| **Current total** | **449** | +| **Growth** | **2.0x** | + +Conservative estimate at 2 Rails/week. At 3/week aspirational: ~1,100 clicks. + +--- + +## 🛑 WHAT WE STOP DOING + +1. **Multi-language AI tutorials** — elixir-ai (7K impr, 0.18% CTR), + laravel-ai (7.2K, 0.01%), python-langchain (3.5K, 0.06%). These don't + convert. Redirect effort to Rails AI content (Ruby is the site's + authority). +2. **Standalone AI framework comparisons** — autogen, crewai, langgraph posts + get massive impressions and zero clicks. The content is salvageable via + snippet rewrites, but don't publish more of this format. +3. **News-pegged founder posts without search intent** — the PocketOS + "AI agent deleted database" post (48/50 score) has 1 click in 3 months. + News-pegged content gets a spike then flatlines. Reserve for LinkedIn, + not SEO. +4. **Posts without a specific, searchable query** — every new post must + target at least one query we can rank for within 60 days. "Rails Best + Practices" is too broad; "Active Job Retries Rails 8" is specific. + No more "this would be good for [audience]" posts without search demand + validation. + +--- + +## 📎 APPENDIX: GSC Data Summary + +Full data: `/Users/pftg/Downloads/jetthoughts.com-Performance-on-Search-2026-07-19/` + +| File | Rows | Key Insight | +|---|---|---| +| Chart.csv | 91 days | CTR declining 0.2% → 0.1%, avg position worsening 13 → 15 | +| Pages.csv | 791 URLs | Top 10 pages by clicks are all technical; top by impressions are all AI framework | +| Queries.csv | 1,015 queries | "upwork login" = 43K+ impressions; brand query "jetthoughts" = 45% CTR | +| Countries.csv | 169 countries | US: 95 clicks (21%), India: 60 (13%), Germany: 27 (6%) | +| Devices.csv | 3 devices | Desktop 84% of clicks, mobile 16% | + +--- + +## 🔄 UPDATE PROTOCOL + +**This Document**: Update after monthly GSC review +**Parent project TASK-TRACKER.md**: Update after each phase completion +**20.07 plan**: Mark as SUPERSEDED — link to this document + +**Next review**: Aug 19, 2026 (30-day GSC delta check) + +--- + +*Plan generated from GSC data export dated 2026-07-19. All click/impression +numbers are 3-month aggregates (Apr 17 – Jul 16, 2026).* diff --git a/package-lock.json b/package-lock.json index 25ade0422..dcc813f46 100644 --- a/package-lock.json +++ b/package-lock.json @@ -16,7 +16,7 @@ "cssnano": "^8.0.1", "lighthouse": "^13.3.0", "markdownlint-cli": "^0.48.0", - "postcss": "^8.5.15", + "postcss": "^8.5.19", "postcss-cli": "^11.0.1", "postcss-delete-duplicate-css": "^1.0.0", "postcss-extend-rule": "^4.0.0", @@ -3740,7 +3740,9 @@ } }, "node_modules/postcss": { - "version": "8.5.15", + "version": "8.5.19", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.19.tgz", + "integrity": "sha512-Mz8SaolMd8nB+G13WkORcxQKHZ/NE4xXevtkJHVuG+guo9/wYKlIMTKAqGdEmYOXR2ijPjTYNHssizdaVSUNdQ==", "dev": true, "funding": [ { @@ -3795,6 +3797,8 @@ }, "node_modules/postcss-cli": { "version": "11.0.1", + "resolved": "https://registry.npmjs.org/postcss-cli/-/postcss-cli-11.0.1.tgz", + "integrity": "sha512-0UnkNPSayHKRe/tc2YGW6XnSqqOA9eqpiRMgRlV1S6HdGi16vwJBx7lviARzbV1HpQHqLLRH3o8vTcB0cLc+5g==", "dev": true, "license": "MIT", "dependencies": { @@ -3820,45 +3824,6 @@ "postcss": "^8.0.0" } }, - "node_modules/postcss-cli/node_modules/tinyglobby": { - "version": "0.2.12", - "dev": true, - "license": "MIT", - "dependencies": { - "fdir": "^6.4.3", - "picomatch": "^4.0.2" - }, - "engines": { - "node": ">=12.0.0" - }, - "funding": { - "url": "https://github.com/sponsors/SuperchupuDev" - } - }, - "node_modules/postcss-cli/node_modules/tinyglobby/node_modules/fdir": { - "version": "6.4.3", - "dev": true, - "license": "MIT", - "peerDependencies": { - "picomatch": "^3 || ^4" - }, - "peerDependenciesMeta": { - "picomatch": { - "optional": true - } - } - }, - "node_modules/postcss-cli/node_modules/tinyglobby/node_modules/picomatch": { - "version": "4.0.2", - "dev": true, - "license": "MIT", - "engines": { - "node": ">=12" - }, - "funding": { - "url": "https://github.com/sponsors/jonschlinkert" - } - }, "node_modules/postcss-colormin": { "version": "8.0.0", "dev": true, @@ -4731,33 +4696,6 @@ "node": ">=18" } }, - "node_modules/purgecss/node_modules/postcss": { - "version": "8.5.14", - "dev": true, - "funding": [ - { - "type": "opencollective", - "url": "https://opencollective.com/postcss/" - }, - { - "type": "tidelift", - "url": "https://tidelift.com/funding/github/npm/postcss" - }, - { - "type": "github", - "url": "https://github.com/sponsors/ai" - } - ], - "license": "MIT", - "dependencies": { - "nanoid": "^3.3.11", - "picocolors": "^1.1.1", - "source-map-js": "^1.2.1" - }, - "engines": { - "node": "^10 || ^12 || >=14" - } - }, "node_modules/purgecss/node_modules/postcss-selector-parser": { "version": "7.1.1", "resolved": "https://registry.npmjs.org/postcss-selector-parser/-/postcss-selector-parser-7.1.1.tgz", diff --git a/package.json b/package.json index 994adc071..4082ec26b 100644 --- a/package.json +++ b/package.json @@ -23,7 +23,7 @@ "cssnano": "^8.0.1", "lighthouse": "^13.3.0", "markdownlint-cli": "^0.48.0", - "postcss": "^8.5.15", + "postcss": "^8.5.19", "postcss-cli": "^11.0.1", "postcss-delete-duplicate-css": "^1.0.0", "postcss-extend-rule": "^4.0.0", @@ -52,37 +52,58 @@ "description": "MCP servers available to agents on this project. No API keys, tokens, or secrets — values here are server names and tool names only.", "chrome-devtools": { "purpose": "Browser automation — screenshot capture, UI testing, visual regression", - "tools": ["new_page", "emulate", "take_screenshot"], + "tools": [ + "new_page", + "emulate", + "take_screenshot" + ], "ref": ".stitch/design.md" }, "context7": { "purpose": "Framework documentation — official API references, feature specs, integration patterns", - "tools": ["resolve-library-id", "get-library-docs"], + "tools": [ + "resolve-library-id", + "get-library-docs" + ], "ref": "docs/mcp-tool-mastery-guide.md" }, "github": { "purpose": "Repository coordination — PR management, issue tracking, repo analysis", - "tools": ["create_pull_request", "create_issue", "search_repositories"], + "tools": [ + "create_pull_request", + "create_issue", + "search_repositories" + ], "ref": ".claude/commands/github/" }, "brave-search": { "purpose": "Web search — technology best practices, emerging patterns", - "tools": ["search"], + "tools": [ + "search" + ], "ref": "docs/mcp-tool-mastery-guide.md" }, "searxng": { "purpose": "Web research — self-hosted metasearch, current best practices, community solutions", - "tools": ["search"], + "tools": [ + "search" + ], "ref": "docs/mcp-tool-mastery-guide.md" }, "package-search": { "purpose": "Package and dependency research — find and analyze npm/Ruby/Go packages before install", - "tools": ["package_search_hybrid", "package_search_grep"], + "tools": [ + "package_search_hybrid", + "package_search_grep" + ], "ref": "docs/mcp-tool-mastery-guide.md" }, "semble": { "purpose": "Semantic code search — natural-language queries with nearest-neighbor expansion", - "tools": ["search", "find_related"], + "tools": [ + "search", + "find_related" + ], "ref": ".claude/agents/semble-search.md" } } diff --git a/seo/redirects.csv b/seo/redirects.csv index 79d1a8f1a..92c632ef4 100644 --- a/seo/redirects.csv +++ b/seo/redirects.csv @@ -13,7 +13,12 @@ https://jtway.co/auto-install-system-dependencies-for-ruby-on-rails-4a19c22b6d49 https://jtway.co/automated-delivery-react-vue-app-for-each-pull-request-38ecd4814803,https://jetthoughts.com/blog/automated-delivery-react-vue-app-for-each-pull-request-ci/,301,,TRUE,, https://jtway.co/avoid-data-migrations-in-the-schema-rb-migrations-for-rails-1baf8f5ed9ea,https://jetthoughts.com/blog/avoid-data-migrations-in-schema-for-rails-ruby/,301,,TRUE,, https://jtway.co/benefits-of-working-remotely-4c36456a9adb,https://jetthoughts.com/blog/benefits-of-working-remotely-remote/,301,,TRUE,, -https://jtway.co/best-practices-for-optimizing-ruby-on-rails-performance-b62d805f6ea6,https://jetthoughts.com/blog/best-practices-for-optimizing-ruby-on-rails-performance/,301,,TRUE,, +https://jtway.co/best-practices-for-optimizing-ruby-on-rails-performance-b62d805f6ea6,https://jetthoughts.com/blog/ruby-on-rails-performance-optimization-patterns-2026/,301,,TRUE,, +https://jtway.co/maximizing-efficiency-proven-strategies-for-ruby-on-rails-performance,https://jetthoughts.com/blog/ruby-on-rails-performance-optimization-patterns-2026/,301,,TRUE,, +https://jtway.co/ruby-on-rails-performance-optimization-patterns-for-2026,https://jetthoughts.com/blog/ruby-on-rails-performance-optimization-patterns-2026/,301,,TRUE,, +https://jtway.co/unlocking-rails-performance-essential-strategies-for,https://jetthoughts.com/blog/ruby-on-rails-performance-optimization-patterns-2026/,301,,TRUE,, +https://jtway.co/rails-performance-optimization-15-proven-techniques,https://jetthoughts.com/blog/ruby-on-rails-performance-optimization-patterns-2026/,301,,TRUE,, +https://jtway.co/best-practices-for-optimizing-ruby-on-rails-performance-b62d805f6ea6,https://jetthoughts.com/blog/ruby-on-rails-performance-optimization-patterns-2026/,301,,TRUE,, https://jtway.co/best-tips-for-writing-integration-tests-in-rails-d1f56081f249,https://jetthoughts.com/blog/tips-for-writing-readable-system-tests-in-rails-capybara-ruby/,301,,TRUE,, https://jtway.co/building-video-app-for-mobile-with-react-native-part-1-expo-e34d7f06429b,https://jetthoughts.com/blog/video-player-with-react-native-part-1-expo-reactnative/,301,,TRUE,, https://jtway.co/change-inputs-placeholder-color-with-css-8f6384121d8e,https://jetthoughts.com/blog/change-inputs-placeholder-color-with-css-html/,301,,TRUE,,