fix(trigger): stop prod runs failing on backfill + transient upstreams

[timothygithinji]

Summary

Fixes three independent failure classes seen in recent prod Trigger runs. Already deployed to prod as version 20260605.1.

• scrape-portal (the real bug): a Rightmove backfill page past the result set returns HTML with no __NEXT_DATA__, which threw and killed the whole run — discarding listings already collected from earlier pages. Now an unparseable page past page 0 is a soft pagination-stop (keep what we have, break); page 0 still throws (genuine block → retry).
• enrich-epc: the gov EPC register intermittently serves an HTTP 500 HTML page; searchEpcCerts threw and exhausted retries. Now skips the round on 5xx / no-status (re-runs on cadence); genuine 4xx still throws so real misconfig stays loud.
• enrich-ai / enrich-broadband: neon-http masks DB failures as a generic Failed query: <sql> with the cause stripped. Now surfaces err.cause in the shared upsert and enrich-ai onFailure, and widens these two tasks' retry backoff (5×, 2s..30s) so a transient Neon blip isn't burned through in the global 3×/1s default's ~6s window.

Notes

• Only feat: unblock geo-enrichment (coords) + exact/estimate EPC with manual address override #1 is a true logic fix. feat(notifications): instant mutual-match email + daily digest (Resend + React Email) #2 and Onboard partner: Access OTP allowlist + harden invite accept #3 are diagnosability + resilience — Onboard partner: Access OTP allowlist + harden invite accept #3's cause-logging is what will finally reveal whether the Neon failures are rate-limiting or something else.
• Also gitignores .trigger/ local build state.

Verification

• tsc --noEmit clean
• biome check on changed files — only pre-existing warnings
• Deployed to prod (17 tasks indexed, build passed)

Summary by CodeRabbit

Release Notes

• Improvements
  • Enhanced system resilience with automatic retry mechanisms for transient service failures during property data enrichment
  • Improved error handling when external services are temporarily unavailable—operations now gracefully degrade rather than fail entirely
  • Enhanced diagnostic logging to aid troubleshooting and error investigation

[coderabbitai]

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: eb73a855-3283-4716-867b-e307b5ee6d39

📥 Commits

Reviewing files that changed from the base of the PR and between 4aa2a03 and ab0d7d2.

📒 Files selected for processing (6)

• .gitignore
• src/trigger/enrich-ai.ts
• src/trigger/enrich-broadband.ts
• src/trigger/enrich-epc.ts
• src/trigger/enrich-helpers.ts
• src/trigger/scrape-portal.ts

---

Walkthrough

This PR enhances resilience across the enrichment and scraping pipeline by adding task-level retry policies, conditional error handling for transient upstream failures, and detailed diagnostic logging to distinguish transient server errors from permanent configuration/client errors.

Changes

Enrichment pipeline resilience improvements

Layer / File(s)	Summary
Project configuration .gitignore	Gitignore updated to exclude Trigger.dev local build and temporary state (.trigger/ directory).
Task-level retry policies src/trigger/enrich-ai.ts, src/trigger/enrich-broadband.ts	Both enrichment tasks configured with explicit retry policies: 5 attempts, exponential backoff (factor 2), 2s–30s timeout bounds, and jitter enabled to survive transient Neon HTTP failures during fan-out bursts.
Error logging and diagnostics src/trigger/enrich-helpers.ts, src/trigger/enrich-ai.ts	Enrichment-helpers imports logger and wraps UPSERT in try/catch to log listing ID, attempted patch columns, extracted driver cause, and error message before rethrowing. Enrichment-ai onFailure handler enhanced to include cause/sourceError field in failure diagnostics.
Graceful upstream API error handling src/trigger/enrich-epc.ts	EPC certificate search now extracts upstream HTTP status and conditionally returns empty list (with warning log) for missing status or server errors (≥500), allowing task retry; throws on client errors (4xx) to signal configuration issues.
Portal scraping pagination resilience src/trigger/scrape-portal.ts	Rightmove/Zoopla pagination page parsing wrapped in try/catch: failures on page 0 rethrow to trigger task retry; failures on later pages log warning and stop pagination while preserving collected summaries.

🎯 2 (Simple) | ⏱️ ~12 minutes

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}