Consolidate tile rebuild pipeline#99
Merged
Merged
Conversation
process_data (upload, import, delete-recompute, regenerate-map) built its vector tiles inline, so its rebuilds could race the edit chain's regenerate_tiles on the same folder. Its tile step now delegates to the same per-folder lock + dirty-flag path: concurrent rebuilds of one folder are single-flight, rapid changes coalesce into ~one rebuild, and a task still returns only once the tiles cover its own change — so chain/taskinfo semantics are unchanged for every caller.
An exclusion edit knows exactly which locations it excluded from which coverage files, but only the drawn polygon was persisted. Any later recompute (e.g. after deleting another editfile) therefore re-applied edits geometrically — every point inside the polygon, for every linked coverage — which is broader than the user's original picks: a multi-coverage partial pick silently became a blanket exclusion. Editfiles now store their markers (a JSON column, with an Alembic migration), filing copies carry them over, and the recompute filter applies marker-bearing editfiles exactly: only the marked locations, only for the coverage files each pick names. Editfiles created before this change have no markers and keep the geometric behavior, so existing filings reproduce their current results unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two related changes to the tile update/edit/rebuild pipeline. First, we now preserve the exact BSLs edited at edit time rather than just capturing the edit polygon; this keeps later recomputes (e.g. after deleting an edit) from re-applying an edit more broadly than the user's original selection. Existing edits without stored selections keep the old polygon-based behavior. Second, the map update path (e.g., uploading new coverage files) now uses the same rebuild path as the edit flow, which serializes simultaneous rebuilds of a filing and reduces unnecessary rebuilds.