Repository navigation
fix: three asks a platform raised against 26.09.06 on the day it shipped — v26.09.07 - #169
Merged
Merged
Conversation
…ped — v26.09.07 Carve identifier shapes (uuid, W3C trace/span id, ULID) out of a log message before any PII pattern runs, and make PHONE mean a phone number: it was guarded by digits only, so it mangled about one uuid in four and never matched compact E.164, giving up searchability and privacy at once. Probe before running the Postgres outbox DDL, so a serving process needs no CREATE on the schema and no ownership of framework tables, with pyfly.eda.postgres.auto-create-tables as the opt-out. Subscribe the CQRS cache-invalidation bridge one event type per registered rule instead of the wildcard, so a process with no rule is not a consumer of the bus and the drain loop's cursor guard works as written.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three asks a platform raised against
26.09.06on the day it shipped. Each was found by running a service on the framework, not by reading it.An identifier in a log message survives redaction, and
PHONEmeans a phone numberThe built-in
PHONErule was guarded by digits only —(?<!\d)and(?!\d)— and a hex letter or a hyphen satisfies a digit-only guard, so it matched inside a uuid. Measured on this checkout over 200 000 generated ids per shape: 24.9% of uuid4 and 24.8% of uuid7 were mangled, producing lines such asrun 01a0d4d8-bc85-<PHONE>-1ff4a6a3a450. A run id grepped out of an application therefore never matched its own log line — silently and only sometimes, which is worse than always.The same rule missed compact E.164 (
+34911234567,+442071838750,+8613800138000): eleven contiguous digits cannot be split by\d{3}\d{4}. It gave up searchability and privacy at once, so this is not a trade.PHONEis re-anchored on word boundaries and matches compact E.164, separated international,(212) 555-0143,202-555-0143,555-0143. Behaviour change: a national number written as seven bare digits with no separator is deliberately no longer matched.pyfly.logging.redaction.preserve-patterns(named-regex map, symmetric withextra-patterns) for a consumer's own id shape; an invalid entry is warned about and ignored.After: 0.00% of uuid4 and uuid7 mangled, no true positive lost, three gained.
A serving process boots the Postgres event bus with no right to create schema
PostgresEventBus.start()ran the outboxCREATE TABLE/INDEX IF NOT EXISTSin every process at every boot, including a pure publisher (publish()lazily starts the bus). PostgreSQL checksCREATEon the schema beforeIF NOT EXISTS, andCREATE INDEX IF NOT EXISTSon a table you do not own fails even when the index exists — aGRANT CREATE ON SCHEMAdoes not fix it. The framework was asking every deployment to make its serving role an owner of two framework-internal tables.start()now probes with oneto_regclassround trip (needs onlyUSAGEon the schema) and issues DDL only when a table is missing. A first boot against an empty database still creates everything.pyfly.eda.postgres.auto-create-tables: falseis the opt-out for a deployment whose schema belongs to its migrations. The consumer group's cursor row stays outside the skip: it is data, not schema.The CQRS cache-invalidation bridge subscribes one handler per rule, not the wildcard
EdaCacheInvalidationBridge.subscribe()registered"*"the moment a CQRS context had an EDA bus, whether or not any rule was registered — a process subscribed to everything in order to discard it. On the Postgres bus that is not passive:_drainrefuses to advance the group's cursor while no handler is registered, precisely so events published before a worker subscribed are not lost, and a wildcard handler that dispatches nothing defeated that guard. An API sharing a worker's consumer group drained the worker's queue into a no-op and its jobs stayed queued.The bridge now subscribes an event type when a rule for it is registered, and nothing while it has no rule; registering before or after the framework attaches it both work.
pyfly.cqrs.cache.invalidation.enabled: falserefuses the bean without disabling CQRS.Tests
tests/logging/test_identifiers_survive_redaction.pyLogRecord; allow-list and span-splicing edge casestests/eda/test_postgres_outbox_ddl_gate.pystart()against a fake asyncpg, asserting on the statements senttests/integration/test_eda_postgres_least_privilege.pyUSAGE+SELECT/INSERT/UPDATEand nothing else; fails on the old adapter withpermission denied for schema publictests/cqrs/test_cache_bridge_subscribes_narrowly.pytests/eda/test_cursor_guard_with_cache_bridge.pytests/logging/test_redaction_processor.py::test_trace_ids_are_never_redacted_even_with_phone_patternhad a sanity line asserting thatPHONEwould corrupt a trace id if unguarded. That is no longer true, which is the point; the assertion is inverted and the field guard it protects is untouched.Gates
uv run ruff format --check src/ tests/anduv run ruff check src/ tests/— cleanuv run mypy src/pyfly --strict— 723 files, no issues5150 passed, 7 skipped, 59 deselected(baseline onmainwith the same selection: 5097 collected; +60 tests, none lost)python -m mkdocs build --strict— cleanpytest -m integration tests/integration/test_eda_postgres_least_privilege.pyagainst a real PostgreSQL — 3 passedDocs updated in
docs/modules/logging.md,docs/modules/events.mdanddocs/modules/cqrs.md; version bumped to26.09.07per the repository's release habit.