Skip to content

[DOCS] Add blog post on the raster data source tiling reader#3098

Merged
jiayuasu merged 1 commit into
apache:masterfrom
jiayuasu:blog/raster-tiling-reader
Jul 10, 2026
Merged

[DOCS] Add blog post on the raster data source tiling reader#3098
jiayuasu merged 1 commit into
apache:masterfrom
jiayuasu:blog/raster-tiling-reader

Conversation

@jiayuasu

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Is this PR related to a ticket?

  • No: this is a documentation update. The PR name follows the format [DOCS] my subject

What changes were proposed in this PR?

Adds a blog post, "Open Huge GeoTIFFs Without the 2 GB Wall", introducing the raster data source that tiles a GeoTIFF into one row per tile on read (1.9.0). The post uses a 3.5 GB AlphaEarth Foundations embedding scene (8192×8192, 64 bands, COG with 1024×1024 internal tiles) as the running example and covers:

  • why a whole-file read hits Spark's per-record ceiling, and how per-tile reading avoids it;
  • the retile / tileWidth / tileHeight / padWithNoData options and directory globbing (recursiveFileLookup, pathGlobFilter);
  • inspecting the tiled DataFrame and running cross-tile zonal statistics.

New files:

  • docs/blog/posts/raster-tiling-reader.md
  • docs/blog/posts/raster-tiling-reader-cover.svg (hand-authored SVG cover)

How was this patch tested?

Documentation-only change. The code snippets were verified locally by loading a real 3.5 GB GeoTIFF through sedona.read.format("raster") and running the inspection and zonal-statistics queries shown in the post: the file split into 64 tiles (8×8) of 1024×1024×64 bands each, and the tiling-aware zonal-stats roll-up (SUM(sum)/SUM(count)) counted exactly the expected number of pixels across the overlapping tiles. The page renders in the mkdocs blog and passes the repo's markdownlint/codespell pre-commit hooks.

Did this PR include necessary documentation updates?

  • Yes, I have updated the documentation.

Introduce the 1.9.0 raster data source that tiles GeoTIFFs into one row
per tile on read, using a 3.5 GB AlphaEarth Foundations scene as the
running example. Includes inspection and cross-tile zonal-stats snippets
and an SVG cover.
@jiayuasu jiayuasu merged commit c54633a into apache:master Jul 10, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant