This is a cloud-native, serverless ELT pipeline built on Google Cloud Platform. It automatically extracts World Cup football data from TheSportsDB API, processes it through Bronze, Silver and Gold layers using PySpark and Dataproc Serverless, and delivers analysis-ready datasets in BigQuery, fully orchestrated with Cloud Composer.
bigquery pyspark data-proc gcp-services cloud-native-data-engineering serverless-elt-pipeline-design cloud-composer-and-apache-airflow-orchestration
-
Updated
Aug 22, 2026 - Python