Federated Integration of Cloud-Native Geospatial Raster and Vector Data via Virtual Knowledge Graphs

MSc Data Science Dissertation - Heriot-Watt University

Author: Adam Doneo | Supervisor: Prof. Albert Burger

Contents

• Project Summary
• Key Findings
• System architecture overview
• Engine versions
• Quick start guide

Project Summary

This project develops and evaluates a system for the federated integration of cloud-native geospatial (CNG) data, specifically Sentinel-2 satellite raster imagery (Cloud-Optimised GeoTIFFs) and Overture Maps land-use vector data (GeoParquet), to determine crop health for agricultural land parcels. Two different raster integration strategies are developed and compared, and the system is exposed via both a SPARQL endpoint (Ontop VKG) and a spatial REST API (pg_featureserv).

Key technologies: PostgreSQL, PostGIS, DuckDB FDW, PL/Python, RasterIO, NumPy, Docker, STAC API, Ontop VKG, GraphDB, GeoSPARQL, OWL, pg_featureserv

Key Findings

1. The inability to pass user-defined spatial parameters into database functions via Ontop VKG is a fundamental constraint rooted in the R2RML specification; this restricts the VKG to a pre-configured spatial extent and represents the primary blocker to applying VKGs to CNG data integration
2. Whilst the VKG layer provides semantic value through ontological reasoning, schema abstraction and knowledge centralisation, this comes at a severe performance cost driven by redundant self-joins over computationally expensive views
3. The optimal raster integration strategy depends on query selectivity; full federation outperforms temporary scene materialisation for selective queries, whilst the latter dominates for batch queries

System architecture overview

Three-layer system architecture designed for this project illustrated below. The data layer (bottom) comprises remote cloud-native geospatial data sources stored on S3 object stores, including Sentinel-2 L2A COGs and Overture Maps GeoParquet files originating from OpenStreetMap. The engine layer (middle) federates and integrates this data using two raster integration strategies: full federation via out-db rasters (V1) and temporary COG materialisation (V2/V3). The interface layer (top) exposes the engine to end-users via a SPARQL endpoint provided by Ontop VKG through GraphDB, and a spatial REST API provided by pg_featureserv

System architecture diagram

Engine versions

Version	Directory	Description
V1	docker_containers/v1_full_federation/	Full federation via out-db rasters; each centroid triggers a separate HTTP range request via GDAL
V2	docker_containers/v2_temp_cog_materialisation/	Temporary COG materialisation; batch pixel extraction via RasterIO/NumPy, replacing numerous individual HTTP range requests per centroid with 2 COG downloads
V3	docker_containers/v3_stac_api/	Extends V2 with STAC integration, parameterised spatial functions, and pg_featureserv REST API

The Ontop VKG semantic layer (ontology and OBDA mappings in semantic_model/) can be deployed over V1 or V2 via GraphDB Desktop.

Quick start guide

Docker Desktop

Each engine version is in a self-contained Docker configuration. Ensure that Docker Desktop is installed and running before proceeding.

• navigate to the directory of the docker-compose.yml file for the desired engine version
• run docker compose up -d --build to spin up the docker container
• run docker compose down to spin down the docker container

Notes:

• If another PostgreSQL instance is already running on the port specified in the docker-compose.yml file, (port 5434 if left unchanged - this is an alternative PostgreSQL port to not clash with any running instances), the Docker container will fail to start. Stop the existing instance or change the port mapping in the docker-compose.yml file.
• When switching between engine versions, run docker system prune --volumes -f to remove volumes before spinning up a different version (useful if disk space is an issue).
• V3 automatically populates the STAC lookup table on startup. Allow a few minutes for the container to fully initialise before querying.
• pg_featureserv is accessible at http://localhost:9000 once set up and running via V3. Access the URL below to interact with the spatial REST API endpoint for the query_greenspace_ndvi function and see results on a map interface (this takes a few minutes to run - please be patient):

http://localhost:9000/functions/public.query_greenspace_ndvi/items.html?p_xmin=-4.453026&p_ymin=54.051436&p_xmax=-2.847262&p_ymax=55.046881&p_overture_classes=farmland,allotments,dog_park&limit=15000

GraphDB Desktop jdbc and virtual repository config instructions:

Ensure that GraphDB Desktop is installed before proceeding. Note that GraphDB Desktop version 11.4 was used during the development of this project - this version is preferred due to its mapping interface to visualise geospatial results.

To be able to set up an Ontop VKG connection to postgres, a jdbc driver needs to be downloaded and placed in the graphdb directory

• download latest jdbc postgres driver from here: https://jdbc.postgresql.org/download/
• navigate to the GraphDB Desktop install directory (for example: C:\...\GraphDB Desktop\app\lib\)
• create a folder called jdbc
• paste the .jar file in this directory
• set up the connection as per the screenshot below, using the .obda and .ttl files found in the semantic_model folder

Notes:

• The username and password are the same as the ones defined in the docker-compose file (postgres)