Skip to content

perf: fetch split quadrants concurrently with bounded Overpass concurrency - #196

Open
anthony-at-pieces wants to merge 1 commit into
FoggedLens:mainfrom
anthony-at-pieces:fix/issue-109-parallel-quadrants
Open

perf: fetch split quadrants concurrently with bounded Overpass concurrency#196
anthony-at-pieces wants to merge 1 commit into
FoggedLens:mainfrom
anthony-at-pieces:fix/issue-109-parallel-quadrants

Conversation

@anthony-at-pieces

Copy link
Copy Markdown
Contributor

Summary

  • fetch split quadrants concurrently (Future.wait) instead of one-at-a-time — worst case drops from 64 sequential Overpass requests (max split depth 3) to ~3 rounds of parallel requests
  • bound total concurrent Overpass requests with a counting semaphore (cap 4, covering nested splits), matching the intent already documented in service_policy.dart ("concurrency managed by NodeDataManager")
  • preserve per-quadrant failure isolation: a failed quadrant logs and contributes no nodes while the remaining quadrants still complete
  • add a @visibleForTesting constructor so NodeDataManager can be built with a faked Overpass client (no production behavior change)

Test plan

  • flutter test test/services/node_data_manager_test.dart — 3 new tests:
    • split quadrant fetches overlap in time (fails on the old sequential loop: only 1 quadrant request ever in flight)
    • nested split fetches stay bounded (≤ 4 in flight when 16 leaf requests are queued)
    • a failed quadrant (exhausted retries + fallback) does not abort the remaining quadrants
  • flutter analyze --no-fatal-infos — no issues
  • flutter test — full suite passes (270 tests)
  • Verified the concurrency test bites: temporarily restoring the sequential loop makes it fail (Expected: <5> ... Actual: <2> — 3 of 4 quadrant requests never started)

Notes

  • Rate-limit behavior is unchanged: RateLimitError still backs off without splitting (splitting into 4x requests against a 429'ing server would be counterproductive), so the semaphore only ever governs NodeLimitError-driven splits.
  • Deliberately scoped to _fetchSplitAreas + the semaphore; intentionally does not fold in the broader smart-fetching work from Smart Overpass fetching, SQLite node cache, and rate-limit handling #133.

Closes #109

…rency

Sequential quadrant fetching made worst-case splitting 64 sequential
Overpass requests (max split depth 3). Fetch the 4 quadrants of each
split round concurrently instead, bounding total concurrent Overpass
requests (including nested splits) with a counting semaphore (cap 4).

Per-quadrant failure isolation is preserved: a failed quadrant logs
and contributes no nodes while the remaining quadrants complete.

Adds a @VisibleForTesting constructor so NodeDataManager can be built
with a faked Overpass client for concurrency tests.

Closes FoggedLens#109
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: Parallelize quadrant fetching during area splitting

1 participant