Day-Zero Data Quality Gatekeeper — manifest-driven pre-ingestion checks built on GX #11806
Closed
huacenxu
started this conversation in
Integrations
Replies: 1 comment
|
Thanks for sharing @huacenxu. One important caveat for anyone considering adopting it: this tool is built on GX 0.18.x, which is a legacy version, and no longer receives updates or bug fixes. If you're starting a new project, we'd strongly recommend building against the current GX release (docs here) to get active support, security patches, and the current API. @huacenxu looks like the lift would be low to migrate to the v1 API, but let me know if you have questions. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We built an open-source tool called Day-Zero Gatekeeper on top of GX 0.18.x for teams who need data quality checks before ingestion — with zero historical data required.
The approach: encode the full data contract in a YAML manifest (schema, health, business rules) and gate the pipeline on Day One, before any history exists.
We added a Layer 4 completeness check that GX doesn't have natively — compares source-to-raw record counts via a vendor control file, with a same-vendor taxonomy proxy fallback when no manifest is available.
Sharing as a community extension for anyone building pre-ingestion DQ pipelines.
👉 https://github.com/huacenxu/day-zero-gatekeeper
All reactions