Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
76 lines (61 loc) · 3.57 KB

CONTRIBUTING.md

File metadata and controls

76 lines (61 loc) · 3.57 KB
Raw

Contributing — jolarca-data

Data engineers, analysts, and the DPO. Every merge here changes either the dataset governance record or the pipeline that transforms personal data — both are auditable surfaces. Commits follow Conventional Commits; notable changes land in CHANGELOG.md.

Seed doctrine (seed/)

  1. Synthetic only. Fixtures are generated by faker-seeded generators (synthetic/generators/); no real person, company, or address may be committed. Regeneration with a new seed must produce new identities.
  2. Deterministic & versioned. Seed files are committed artifacts with schema validation (seed/validators/); CI blocks malformed taxonomy. Changes to taxonomy are MINOR/MAJOR changes for consumers (jolarca) — describe impact in the PR.
  3. Translations are complete. Category names exist in lt/lv/et/en (ru reserved); a missing translation is a CI failure, not a TODO.

Warehouse doctrine (warehouse/)

  1. PII is stripped in staging. Staging models hash identifiers and drop names/emails — a model that re-introduces an identifier is a defect of the highest class.
  2. Every model is documented. models/**/_models.yml entries with owner + tests (unique, not_null, relationships) are mandatory; undocumented models fail dbt-ci.
  3. Tests encode the invariants. Custom tests (warehouse/tests/) guard no-null-pii-columns, id-hash-format, eur-only, vat-rate-bounds.
  4. No production credentials. Warehouse access is via scoped service accounts from env vars (ADR-0002); nothing in-repo can reach production, and extraction is read-replica only.

Governance is mandatory

  • Every new dataset lands with: a governance/data-catalog.md entry, an owner + steward row in ownership-register.csv, a classification tier, a RoPA purpose reference, and a retention class. catalog-lint blocks orphans.
  • Retention class changes require DPO review (CODEOWNERS routes it).
  • Policy text lives in jolarca-compliance; this repo implements it.

Personal data — absolute rules

  • Never commit personal data, real or "temporarily". Synthetic only.
  • Never commit warehouse credentials, connection strings with passwords, or API keys — gitleaks enforces.
  • scripts/check-personal-data.sh and the PII pattern scan are tripwires, not a license: passing them does not make a commit lawful.

Workflow

  1. Issue first. dataset_request for new datasets, pii_incident for PII findings (never put the data itself in the issue).
  2. Branch per change. One dataset/pipeline concern per PR.
  3. CI is a merge gate. ci, compliance-check, dbt-ci (parse + slim build + tests) must be green.
  4. CODEOWNERS routing is binding. DPO review on anything touching RESTRICTED paths, lifecycle jobs, and compliance marts.

Solo-era operation (current): the org operates with a single data operator, so human review gates ride on automated checks + CODEOWNERS routing until the second operator onboards. Tracked deviation, not an exemption.

Change-risk classes

Class Examples Gate
Low Docs, reserved scaffolding, dashboards definitions 1 review
Med New staging model, seed translation additions 1 review + CI
High New dataset (catalog entry), mart logic, retention job change CI + owner review; DPO if RESTRICTED
Crit Pseudonymizer change, erasure/legal-hold logic, classification tier change DPO + data platform owner; adversarial verification

If you cannot say which retention class the data falls under and how it is erased, the change is not ready.