Skip to content

fix: constraint- and length-aware data masking (issue #18)#21

Merged
valehdba merged 1 commit into
mainfrom
fix/masking-hardening
Jul 1, 2026
Merged

fix: constraint- and length-aware data masking (issue #18)#21
valehdba merged 1 commit into
mainfrom
fix/masking-hardening

Conversation

@valehdba

@valehdba valehdba commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Builds on the type-aware masking from #17. A mask could still abort or corrupt a clone in ways type-checking alone did not catch; all are now handled by skipping the offending mask (with a WARNING) or clamping it, so the clone always succeeds:

  • length: a constant/partial/hash/random_int value longer than a varchar(N)/char(N) column failed with "value too long". Output for a length-limited string column is now clamped with left(..., N).
  • constant on a numeric column: the default 'REDACTED' (or any non-numeric text) failed to parse. A constant is applied to a non-string column only when the literal is a valid number, otherwise skipped.
  • null on a NOT NULL column: skipped.
  • UNIQUE/PRIMARY KEY column: only the injective "hash" strategy is applied; value-collapsing (name/constant/null) or collision-prone (random_int) strategies are skipped.
  • FOREIGN KEY column: masking is skipped to preserve referential integrity.

discover_sensitive() and masking_report() now only suggest strategies the engine will apply: FK columns are omitted, and UNIQUE/PK or NOT NULL sensitive columns are steered to "hash". A masked view (create_masking_policy) enforces no constraints, so only the type/constant checks apply there.

New helpers: ColMaskMeta + pgclone_column_maskmeta / pgclone_maskmeta_from_row (shared PGCLONE_MASKMETA_COLS catalog fragment), pgclone_mask_skip_reason (single decision point), pgclone_append_mask_expr_clamped, pgclone_looks_numeric, pgclone_discover_strategy.

Tests: pgTAP group 27 (12 tests, plan 95 -> 107) plus mask18_parent/mask18_child fixtures covering length, constant-on-numeric, uniqueness, NOT NULL and FK. Docs: USAGE length/constraint safety section. Version bumped to 4.4.2 with sql/pgclone--4.4.1--4.4.2.sql (no SQL signature changes).

Summary

Closes #

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that changes existing behavior)
  • 🏗️ Refactor / internal cleanup (no functional change)
  • ⚡ Performance improvement
  • 📖 Documentation only
  • 🔧 Build / CI / tooling
  • 🧪 Test-only change

Description of changes

Database / SQL changes

  • New or modified SQL functions are declared in sql/pgclone--<version>.sql
  • PG_FUNCTION_INFO_V1 names in C match the AS '<lib>', '<sym>' clauses in SQL
  • Return types in .sql match the C PG_RETURN_* paths
  • COMMENT ON FUNCTION ... added for new public functions
  • Function volatility (VOLATILE / STABLE / IMMUTABLE) and PARALLEL safety set correctly

Testing

PostgreSQL versions tested locally:

  • pgTAP tests added or updated in test/pgclone_test.sql
  • plan() count exactly matches the actual number of assertions
  • Shell tests added/updated in test/test_async.sh or test/test_database_create.sh (if applicable)
  • test/fixtures/seed.sql updated if new test objects are needed
  • CI is green on all of PG 14, 15, 16, 17, 18
# Output of pre_deploy_checks.sh (paste the summary line)
# e.g. "22 passed, 0 failed"

C code safety checklist

  • palloc / palloc0 / pfree only — no malloc/free
  • Every PQconnectdb has a matching PQfinish in all paths (success and error)
  • Every PQexec result is PQclear-ed in all paths
  • Dynamic SQL uses quote_literal_cstr() and quote_identifier()
  • StringInfo used for dynamic strings; no fixed stack buffers for SQL
  • strlcpy instead of strcpy / strncpy
  • ereport / elog with appropriate level — connection strings never logged at LOG level or above
  • PG-version-specific APIs are guarded with #if PG_VERSION_NUM >= XXXXXX
  • No new uses of removed APIs (d.adsrc, pre-PG15 shmem-request pattern, etc.)
  • Shared-memory state protected by LWLock where needed

Documentation

  • CHANGELOG.md updated under the appropriate version heading
  • Relevant doc updated: docs/USAGE.md / docs/ASYNC.md / docs/ARCHITECTURE.md / docs/TESTING.md
  • README.md updated if the change affects the user-facing feature list or quick-start

Version bump

  • pgclone.control (default_version)
  • META.json (version and provides.pgclone.version + file)
  • README.md version badge
  • New sql/pgclone--<old>--<new>.sql migration script (if applicable)
  • CHANGELOG.md has a heading for the new version

Backward compatibility

Screenshots / sample output

Additional notes

Builds on the type-aware masking from #17. A mask could still abort or
corrupt a clone in ways type-checking alone did not catch; all are now
handled by skipping the offending mask (with a WARNING) or clamping it,
so the clone always succeeds:

- length: a constant/partial/hash/random_int value longer than a
  varchar(N)/char(N) column failed with "value too long". Output for a
  length-limited string column is now clamped with left(..., N).
- constant on a numeric column: the default 'REDACTED' (or any non-numeric
  text) failed to parse. A constant is applied to a non-string column only
  when the literal is a valid number, otherwise skipped.
- null on a NOT NULL column: skipped.
- UNIQUE/PRIMARY KEY column: only the injective "hash" strategy is applied;
  value-collapsing (name/constant/null) or collision-prone (random_int)
  strategies are skipped.
- FOREIGN KEY column: masking is skipped to preserve referential integrity.

discover_sensitive() and masking_report() now only suggest strategies the
engine will apply: FK columns are omitted, and UNIQUE/PK or NOT NULL
sensitive columns are steered to "hash". A masked view (create_masking_policy)
enforces no constraints, so only the type/constant checks apply there.

New helpers: ColMaskMeta + pgclone_column_maskmeta / pgclone_maskmeta_from_row
(shared PGCLONE_MASKMETA_COLS catalog fragment), pgclone_mask_skip_reason
(single decision point), pgclone_append_mask_expr_clamped, pgclone_looks_numeric,
pgclone_discover_strategy.

Tests: pgTAP group 27 (12 tests, plan 95 -> 107) plus mask18_parent/mask18_child
fixtures covering length, constant-on-numeric, uniqueness, NOT NULL and FK.
Docs: USAGE length/constraint safety section. Version bumped to 4.4.2 with
sql/pgclone--4.4.1--4.4.2.sql (no SQL signature changes).
@valehdba valehdba merged commit 3828e16 into main Jul 1, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant