DynamoDB Streams to capture data change in Amazon DynamoDB by CRUD with REST API in Amazon API Gateway
DFS is an efficient mysql data synchronisation framework that consists of a millisecond incremental data capture module (CDC), a sqlite-based persistent queue, an automatic relational processing module (rel_mgr), and a complex data transformation module (DPU), encapsulated using docker containers.
This project showcases the use of Kafka connectors for real-time data streaming from Postgres databases using Spring Boot. Two branches are included, one for JSON converter and another for Avro converter."
Lightweight MSSQL Change Data Capture in Go
DevOps & Data Engineer | Kafka, PostgreSQL, Kubernetes, Docker, Odoo, BigQuery, and real-time data pipelines.
Experiment with Kafka, Debezium. Source Connectors: Monitor MySQL & MongoDb changes, push messages to Kafka. springboot service: Listen to Kafka.
Personal knowledge hub built with data-engineering rigor, "idempotent ETL pipelines", "Bronze/Silver/Gold layers", and "lineage tracking", to gather the high scattered documents in data center. It feed a dashboard & vector-search RAG technique, turning into a single source of truth.
Add a description, image, and links to the capture-data-change topic page so that developers can more easily learn about it.
To associate your repository with the capture-data-change topic, visit your repo's landing page and select "manage topics."