Decouple container startup from catalog source rescans

[WilfordGrimley]

Summary

Implements the plan reviewed in journal/2026-07-14-startup-scan-decoupling-plan.md (local, gitignored — summary below) for today's incident: merging PR #11/#14/#15 triggered a migration-gated full catalog rescan on restart, which crashed django (unhandled IntegrityError) and left the API down for ~1.5h until manually caught and restarted.

Investigation findings (see journal for full detail):

• A daily update_database schedule (+ weekly update_dfcs/import_canonical_card_data) already exists via django-q, seeded by migrations 0043/0048, confirmed via production Task history: 7 consecutive successful daily runs. The entrypoint's boot-time block was a redundant duplicate of this for every restart, not a necessary fallback.
• The real cost of a full rescan is ~22 minutes (not the ~2 hours estimated live during the incident — that was a bad extrapolation from django's crashed, partial run). 75% of that is Google Drive listing calls; the per-source loop was fully sequential across ~254 sources even though each source already used an internal 5-worker pool.
• Modified-time-based skipping (raised as a candidate fix) turns out not to help — Drive's modifiedTime is already fetched per file, but the cost is in the listing call itself, which isn't shortened by post-listing filtering.

Changes

1. docker/django/entrypoint.sh: removed the migrate --check-gated block. Now: migrate (always, fast) → import_sources (always, cheap, local CSV) → exec gunicorn, unconditionally. Catalog scanning never blocks the API from binding again, regardless of duration or failure.
2. Bootstrap guard (import_sources.py): after syncing Source rows, if Source.objects.exists() and not Card.objects.exists(), enqueues one async update_database run via django-q. Narrow safety net for a genuinely fresh instance losing the race against the daily schedule's first (already near-immediate) firing — not a BOOTSTRAP env flag.
3. Bounded parallelism (update_database.py): the outer per-source loop is now a ThreadPoolExecutor(max_workers=MAX_SOURCE_WORKERS=8) instead of fully sequential. Drive's 200 req/s quota has ample headroom (today's sequential run was latency-bound, not quota-bound).
4. Per-source failure isolation (update_database.py): wraps each source's scan so one source's exception (e.g. today's duplicate-key race) is logged and skipped rather than aborting the other ~250 sources — this is the direct fix for today's crash, independent of the parallelism change.
5. docs/infrastructure.md: new "Startup vs. scheduled catalog sync" section recording the incident, the fix, and why; updated the rebuild-from-scratch note (manual update_database after a rebuild is now optional, not required).

Test plan

• New tests: test_import_sources.py (3 cases covering the bootstrap guard's emptiness-check logic), test_sources.py (2 new cases: multi-source concurrent scan correctness, and one-source-failure isolation — both using LocalFile sources, no network dependency, using transactional_db since the parallelized code now writes from worker threads on separate DB connections).
• Full backend suite: 487 passed, same 4 pre-existing baseline failures (moxfield network access, snapshot-fixture path issues — unrelated, present on master too).
• bash -n docker/django/entrypoint.sh — syntax clean.
• Pre-commit (ruff/isort/black/mypy/prettier) — clean.

Deliberately not in this PR

• A genuine Google Drive Changes API (cursor-based) — lower priority per the plan; bounded parallelism alone is expected to bring the full rescan from ~22 min to under 5, and gunicorn no longer blocks on it regardless either way.
• Surfacing per-run created/updated/deleted counts somewhere more durable than stdout logs (django-q Task results) — noted as a small follow-up in the plan, not required for this fix.
• A submit-a-drive → pending-moderator-approval workflow — explicitly parked as a separate future proposal in the plan, unrelated to this incident.

Holding as draft for review per your instructions — not merging or deploying to production without separate explicit go-ahead.

🤖 Generated with Claude Code

https://claude.ai/code/session_016i9S7LQsCL3FGaih3ZTRBJ