Phase 3: multi-location bi-directional menu sync, canonical store, Square OAuth, images - #6
Closed
crewcricle wants to merge 5 commits into
Closed
Conversation
* Phase 0: arq task queue, durable webhooks, single-active scheduler, migrations
- Add arq>=0.26.0 + config settings (redis_url, worker_role, dashboard_url,
stripe_portal_config_id)
- New backend/task_queue.py (arq WorkerSettings, get_arq_pool, record_dead_letter,
inbound processing tasks, durable outbound wrappers for all 5 integrations
Twilio/Resend/GBP/Square/DataForSEO, cron entrypoints) + backend/worker.py
(module named task_queue, not queue, to avoid shadowing stdlib queue that
redis imports)
- Migrations 010_webhook_events, 011_dead_letter (both +RLS service_role_all),
012_booking_credentials (moved to Phase 0 per C1)
- Refactor routers/webhooks.py: persist inbound events to webhook_events,
dedupe by idempotency key, return 200 fast, enqueue processing. GBP inbound
decodes Pub/Sub push envelope, uses messageId as key, fetches review resource
before drafting (C3)
- Refactor scheduler.py to enqueue-only; main.py lifespan branches on worker_role
(web = arq pool only; scheduler = APScheduler enqueue-only + arq pool)
- Mandatory utils/reconcile.py reconcile_pending_webhooks (every 5 min, C4)
- Deploy: localmate-redis (AOF, noeviction, 128MB), localmate-worker,
localmate-scheduler on deploy_default; WORKER_ROLE=web on backend; update
deploy_backend.sh, DEPLOY.md, Dockerfile, .env.example (REDIS_URL, WORKER_ROLE,
STRIPE_PORTAL_CONFIG_ID, DASHBOARD_URL)
- Tests: webhook_events, dead_letter, queue_tasks, scheduler_enqueue,
outbound_durable, reconcile, gbp_envelope_decode; new stub env in conftest
- pytest pythonpath config so tests collect without PYTHONPATH
* refactor(phase0): remove dead code, dedupe imports, fix stale docstring
- task_queue: drop unused _mark_event bump param; hoist datetime import
- reconcile: fix stale queue.WorkerSettings docstring ref -> task_queue
- main: dedupe get_arq_pool import across lifespan branches; cache logger
- tests: remove unused imports/params (json, insert_id, redundant webhooks imports)
* fix(phase0): address code-review blocking issues (C4 reliability, C10 gate)
Item 1 (CRITICAL): arq retries + dead-lettering — RETRY_BACKOFF, _retry_defer,
raise arq.Retry on non-final failures, dead-letter on MAX_TRIES; drop
reliance on ctx["max_tries"] (not present in arq ctx).
Item 2: GBP fetch decrypts access+refresh tokens, refreshes empty, retries once
on 401; raises on transport/auth failure instead of silently returning {}.
Item 3: dedup webhook_events SELECT before provider fetch in inbound_review.
Item 4: keep secrets out of Redis/AOF — tasks take client_id, decrypt in-worker
(post_gbp_reply_task, square_sync_task, _load_client, _resolve_gbp_access_token).
Item 5: single reconcile trigger (arq cron only); deterministic _job_id dedup,
stale-processing recovery, atomic pending claim.
Item 6: only ack duplicate on unique-violation (23505); re-raise other DB errors.
Item 7: propagate outbound failures — strict email/dataforseo variants, menu
dispatch inspects sync result, holiday-skip treated as success.
Item 8 (C10): real Postgres+Redis staging gate (scripts/phase0_staging_gate.*),
asyncpg dev dep, DEPLOY.md acceptance checks.
Add test_lifespan.py for main.py worker_role branching; update affected tests.
* fix(phase0): durable outbound setup failures, lease-based stale recovery, real staging gate
Fix 1 (REVIEW BLOCKER): move client loading + credential resolution INSIDE the
coroutine passed to _run_outbound for post_gbp_reply_task and square_sync_task,
so DB outages / missing clients / decrypt / token-refresh failures get arq
retry + dead-letter instead of failing bare. Add retry-then-dead-letter tests for
client-load and decrypt failures on both tasks.
Fix 2 (REVIEW BLOCKER): add processing_started_at lease column to webhook_events
(migration 010, amended in place). Record it on the atomic pending->processing
claim, clear it on retry reset, and base stale-processing recovery on it instead
of created_at so an actively-running event is never reclaimed. Add test that a
row with old created_at but fresh lease stays processing.
Fix 3 (REVIEW BLOCKER): rewrite phase0_staging_gate to import and run the real
task_queue.FUNCTIONS (process_stripe_event, post_gbp_reply_task) and the real
reconcile_pending_webhooks against real Redis/Postgres via a psycopg Supabase
shim, instead of substitute tasks / replicated SQL. Add the single-scheduler
enqueue-exactly-once + restart/resume check using the real create_scheduler.
Fix 4 (TESTING FIND): change FOR INSERT USING to WITH CHECK in 003_reviews.sql
so the INSERT policy is valid Postgres and a clean-DB migration apply works.
Tests: 88 passed.
---------
Co-authored-by: crewcricle <280911048+crewcricle@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…uare OAuth, images
crewcricle
force-pushed
the
vorflux/phase3-menu-sync
branch
from
July 22, 2026 18:38
c8c5ac6 to
26f761d
Compare
…ioning, Yelp posting (#7) Co-authored-by: crewcricle <280911048+crewcricle@users.noreply.github.com>
…#8) Co-authored-by: crewcricle <280911048+crewcricle@users.noreply.github.com>
# Conflicts: # backend/main.py
Contributor
Author
|
Closing: replaced by #9 (merged). This PR targets the stale |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multi-location, bi-directional menu sync engine with canonical store, image sync, and Square OAuth.
Changes
menu_itemstable): all menu changes (Sheets, Square inbound) upsert canonical rows withcontent_hash+origintracking. Content hash issha256(name|price_cents|description|category|active).reconcile_itempushes canonical to each target platform only whenlast_synced_hashdiffers from currentcontent_hash. Square inbound sets Square's hash first so it never echoes back. Deterministic idempotency keys ({item_id}:square:{hash}) prevent duplicate creates on retry.locationstable): GBP identity lives onlocations(per-venuegbp_account_id/gbp_location_id/square_location_id), notclients. Backfills a default location for existing single-venue clients.resolve_client_from_locationrewritten to uselocations.gbp_location_id(C2).approve.pyposts using the draft's location.drafts.location_id+menu_sync_log.location_idcolumns added (C6).square_oauth.py— OAuth code flow, token exchange/refresh,list_locations, auto-map locations by name (mirrors GBP flow)square_catalog.py—UpsertCatalogObjectwith deterministic idempotency,SearchCatalogObjectswatermark reconcile, deletesquare_images.py— multipartCreateCatalogImagegbp_media.py— v4 media upload viasourceUrlmenu_images.py— Supabase storage + cross-platform image syncPOST /webhooks/square/catalogverifies HMAC-SHA256 signature, resolves client bymerchant_id, reconciles viaSearchCatalogObjectswatermark, applies inbound changes without echo.017–022: all new tables withservice_role_allRLS, 5 partial unique indexes, canonical identity._run_outboundwith retry + dead-letter.GET/POST/PATCH /locations(list, create, update target toggles);POST /menu/{location_id}/items/{item_id}/image(multipart upload).Testing
locations,menu_items,menu_item_links,menu_images,square_sync_stateverified;drafts.location_id+menu_sync_log.location_idpresent; default-location backfill created 1 rowGET/POST/PATCHall 200 with correct shapes; ignored fields not echoedDeferred to post-deploy: live Square/GBP API calls, Supabase storage, dashboard UI (Phase 5).
Artifacts
/code/.generated_artifacts/unit_suite.log/code/.generated_artifacts/targeted_loop_guard.log/code/.generated_artifacts/migration.log/code/.generated_artifacts/locations_smoke.log