1 — Create without householdType returns 202 but never persists
Create accepted with 202; _search returns TotalCount = 0 after a 240s settle. householdType is DEFAULT 'FAMILY' NOT NULL, but the insert names the column and binds the payload value, and an explicit NULL bind defeats a column DEFAULT in Postgres — so an omitted householdType becomes a NOT NULL violation instead of taking the default. The persister correctly dead-lettered it (NOT NULL violation, 5 bounded attempts) — nothing lost silently, but the caller is told the create succeeded when it did not.
2 — _delete returns 202 but is a silent no-op
Plain _delete returns 202; at 30/90/150/210/270s the row is still present with isDeleted=false and rowVersion=1 unchanged. includeDeleted=true shows the same. The delete path never sets isDeleted — it only bumps rowVersion and audit details when the request already carries isDeleted=true — so the record is republished to the delete topic with isDeleted=false and the row is rewritten to exactly the values it already had.
3 — Later edits to an offline-created member are silently discarded
Update returns 202 and the persister logs "persisted", but the row's rowVersion stays 1 and the edited field never appears. The update matches on parent columns that are legitimately NULL, so it changes zero rows — the payload carries the member's own server id, but the statement does not key on it. Everything that goes through this update path is lost the same way: later edits, status changes, and a soft-delete flagged on an update. The member's own _delete is a separate statement keyed on the row's server id and is not lost by this mechanism.
4 — Persister reports rows persisted when zero rows changed
The write layer logs the count it was asked to write, not the count the database changed. Observed a driver returning [0,1] while the log said 2. The per-row counts are discarded and nothing else in the pipeline looks at them either, so a statement that matched nothing commits and is acknowledged exactly like a successful write — no dead-letter, no park, no retry, no error anywhere. This is what made 3 and 6 invisible; it is the visibility defect, and the zero-row writes in 3 and 6 are separate defects in their own right.
5 — Relationship edits: new ones dropped, existing ones overwritten
A relationship added while editing an existing member is never written. And editing one relationship rewrites all of that member's relationships to the last relative's values — observed both rows overwritten, destroying the second.
6 — Address attached to an existing delivery is never stored
The update path can only modify an existing address row, not insert one. Returns 202, no row written, delivery left pointing at an address id that does not exist.
7 — A mixed-key member batch is discarded whole
Validation picks one parent-key interpretation — householdId or householdClientReferenceId — sampled from a single record, then throws on the first record for which that key is null ("element cannot be mapped to a null key"), which aborts the whole batch. The record that gets sampled decides the interpretation for every record, so the same two records need not behave the same way in a different order. Batch of 2 → 202, both absent. A uniform batch persists fine.
8 — A referral batch is discarded whole when one referral omits recipientType
recipientType is optional in the model but dereferenced unguarded. Batch of 1 valid + 1 malformed → 202, both absent.
9 — Referral/side-effect update erases its own beneficiary link
The update overwrites every column, so an update omitting the link NULLs it. Observed: link present after create, NULL after the update, rowVersion 2. For side-effect the same overwrite hits a NOT NULL column and fails outright.
10 — Four consumers discard a failed batch with no recoverable record, and eleven more discard it quietly
Twelve methods across four consumers rethrow. The shared platform's Kafka error handler extends the framework's default handler but overrides its failure path without delegating to it, and the property that would make it publish ships false and is overridden by no service — so a rethrowing method gets zero redeliveries, the failed record is acknowledged, and every record after it in the same poll is dropped with no log line. The other eleven consumers — thirty-three methods — catch and return an empty list, which is the quieter version of the same loss: one log line, then the record is gone. Every bulk create/update/delete path across the six services is one or the other.
11 — Deliveries rejected when the beneficiary has only a clientReferenceId
Flagged INVALID_RELATED_ENTITY_ID / NON_RECOVERABLE — the exact offline shape that should be accepted. NON_RECOVERABLE means the device never retries. Batch of 3 → only 1 persisted.
12 — A never-persisted record is searchable, with a server id, for ~45–75s
Records are cached with their assigned server id at publish time, before the write. A _search whose only criterion is clientReferenceId — the exact call the device makes to confirm an upload — reads that cache before the table: TotalCount=1 with a server id at t+5/20/45s, then 0 at t+75/130s, consistent with the 60s cache TTL rather than a fixed window. A search by the server id, or by any other criterion, does not hit that cache entry and correctly returns 0. The device's only failed-upload detection therefore false-positives, so loss is indistinguishable from success.
13 — Parked records are terminal, unmonitored and not viewable at all
Nothing consumes the park topic, and the dead-letter dashboard's matcher is end-anchored so it matches the dead-letter topic but not the park topic. Observed 63 parked, 34 already aged out, no consumer group. Neither the dead-letter topic nor the park topic appears in any indexer config either, so neither is searchable through operational tooling — these failures are not viewable at all, not merely unmonitored.
14 — A duplicate-key rejection from a semantic unique is treated as success
The classifier maps unique-violation (23505) to benign unconditionally, whatever constraint fired. It does not bite on the top-level device key — a re-sent clientReferenceId is rejected at the service layer first as a duplicate entity, and that check searches with deleted rows included, so it never reaches the persister. It bites on every unique that check cannot see: the project beneficiary voucher tag, the individual's generated individualId, the per-individual identifier and address mappings, and the clientReferenceId uniques on the sub-entities — identifier, skill, address, task resource — which the top-level duplicate check never looks at. Soft-delete a record holding a printed voucher, re-register the same voucher — the voucher-tag check looks only at non-deleted rows while the constraint ignores isDeleted, so the violation is raised, classified benign, and the record discarded with zero artifact.
15 — One table has no uniqueness backstop on its link column
No PK on id, no unique constraint or index on clientReferenceId, bare insert. Observed two live rows sharing one clientReferenceId. The sibling table correctly raises a violation.
16 — Uniqueness guard is blind to the offline-shaped row
The "already a member" check filters on the individual's server id, so a stored row whose individualId is NULL — the normal offline shape — is invisible to it. Observed two membership rows for one individual. Same blind spot on the head-of-household check, which reads the existing members by whichever single household key column the request happens to carry. Two residuals the guard cannot close at all: it reads only rows already stored, so two DISTINCT records in the SAME batch naming the same person, each with its own clientReferenceId, both persist a membership row; and when the individual cannot be resolved the check is skipped outright rather than deferred.
17 — Deleting a member leaves the relationships the delete payload omits active
The server never looks up the member's stored relationship set; it only touches the relationship rows the request itself resends, and that field is optional on the request. Omit memberRelationships from a member _delete and zero relationship statements run — the rows stay isDeleted=false, pointing at a deleted person, and keep being reported, with a 202 either way and no signal that the cascade did not happen. Resend them and the cascade does run, but keyed on the member's selfId it clears every relationship of that member, including ones the request did not name.
18 — Link columns carry no index
Four clientReferenceId link columns are unindexed, across two tables: on referral, projectBeneficiaryClientReferenceId and sideEffectClientReferenceId; on side_effect, taskClientReferenceId and projectBeneficiaryClientReferenceId (catalog read: none of the four carries an index). The equivalent link columns on household_member, project_task and project_beneficiary ARE indexed, so this is confined to referral and side_effect. Presents as downsync/search slowdown at scale, not as an error.
19 — Whitespace-only clientReferenceId accepted with 202
An empty string is rejected with 400, whitespace-only is accepted. It is now the sole linking key, so that is a broken link key. On POST /stock/reconciliation/v1/_create the request body is taken without validation, so on that one endpoint NO constraint on the payload is evaluated at all — not the whitespace case, and not the pre-existing 2-64 length check that produces the 400 elsewhere; a zero-length clientReferenceId, or a missing facilityId, is accepted there. It is the only endpoint of its kind across all six services — the bulk reconciliation create and every other create, update and search body is validated.
20 — The app writes fabricated server ids into server-id columns
(a) an identifier's parent id is set to the parent's own client reference; (b) each relationship is stamped with the member's OWN server id as the relative's id, and one id triple is applied to all of them, cross-linking them; (c) the local read drops the three stored server ids and writes those nulls back on the next edit. No back-fill or repair exists for rows already written with fabricated ids — nothing in the app's local-store upgrade path and no server-side migration touches these columns — so correcting the write sites leaves every row already written wrong.
21 — Validation-error responses echo the caller's credentials
Reproduced. Filing separately as an advisory rather than describing it here.
1 — Create without householdType returns 202 but never persists
Create accepted with 202; _search returns TotalCount = 0 after a 240s settle. householdType is DEFAULT 'FAMILY' NOT NULL, but the insert names the column and binds the payload value, and an explicit NULL bind defeats a column DEFAULT in Postgres — so an omitted householdType becomes a NOT NULL violation instead of taking the default. The persister correctly dead-lettered it (NOT NULL violation, 5 bounded attempts) — nothing lost silently, but the caller is told the create succeeded when it did not.
2 — _delete returns 202 but is a silent no-op
Plain _delete returns 202; at 30/90/150/210/270s the row is still present with isDeleted=false and rowVersion=1 unchanged. includeDeleted=true shows the same. The delete path never sets isDeleted — it only bumps rowVersion and audit details when the request already carries isDeleted=true — so the record is republished to the delete topic with isDeleted=false and the row is rewritten to exactly the values it already had.
3 — Later edits to an offline-created member are silently discarded
Update returns 202 and the persister logs "persisted", but the row's rowVersion stays 1 and the edited field never appears. The update matches on parent columns that are legitimately NULL, so it changes zero rows — the payload carries the member's own server id, but the statement does not key on it. Everything that goes through this update path is lost the same way: later edits, status changes, and a soft-delete flagged on an update. The member's own _delete is a separate statement keyed on the row's server id and is not lost by this mechanism.
4 — Persister reports rows persisted when zero rows changed
The write layer logs the count it was asked to write, not the count the database changed. Observed a driver returning [0,1] while the log said 2. The per-row counts are discarded and nothing else in the pipeline looks at them either, so a statement that matched nothing commits and is acknowledged exactly like a successful write — no dead-letter, no park, no retry, no error anywhere. This is what made 3 and 6 invisible; it is the visibility defect, and the zero-row writes in 3 and 6 are separate defects in their own right.
5 — Relationship edits: new ones dropped, existing ones overwritten
A relationship added while editing an existing member is never written. And editing one relationship rewrites all of that member's relationships to the last relative's values — observed both rows overwritten, destroying the second.
6 — Address attached to an existing delivery is never stored
The update path can only modify an existing address row, not insert one. Returns 202, no row written, delivery left pointing at an address id that does not exist.
7 — A mixed-key member batch is discarded whole
Validation picks one parent-key interpretation — householdId or householdClientReferenceId — sampled from a single record, then throws on the first record for which that key is null ("element cannot be mapped to a null key"), which aborts the whole batch. The record that gets sampled decides the interpretation for every record, so the same two records need not behave the same way in a different order. Batch of 2 → 202, both absent. A uniform batch persists fine.
8 — A referral batch is discarded whole when one referral omits recipientType
recipientType is optional in the model but dereferenced unguarded. Batch of 1 valid + 1 malformed → 202, both absent.
9 — Referral/side-effect update erases its own beneficiary link
The update overwrites every column, so an update omitting the link NULLs it. Observed: link present after create, NULL after the update, rowVersion 2. For side-effect the same overwrite hits a NOT NULL column and fails outright.
10 — Four consumers discard a failed batch with no recoverable record, and eleven more discard it quietly
Twelve methods across four consumers rethrow. The shared platform's Kafka error handler extends the framework's default handler but overrides its failure path without delegating to it, and the property that would make it publish ships false and is overridden by no service — so a rethrowing method gets zero redeliveries, the failed record is acknowledged, and every record after it in the same poll is dropped with no log line. The other eleven consumers — thirty-three methods — catch and return an empty list, which is the quieter version of the same loss: one log line, then the record is gone. Every bulk create/update/delete path across the six services is one or the other.
11 — Deliveries rejected when the beneficiary has only a clientReferenceId
Flagged INVALID_RELATED_ENTITY_ID / NON_RECOVERABLE — the exact offline shape that should be accepted. NON_RECOVERABLE means the device never retries. Batch of 3 → only 1 persisted.
12 — A never-persisted record is searchable, with a server id, for ~45–75s
Records are cached with their assigned server id at publish time, before the write. A _search whose only criterion is clientReferenceId — the exact call the device makes to confirm an upload — reads that cache before the table: TotalCount=1 with a server id at t+5/20/45s, then 0 at t+75/130s, consistent with the 60s cache TTL rather than a fixed window. A search by the server id, or by any other criterion, does not hit that cache entry and correctly returns 0. The device's only failed-upload detection therefore false-positives, so loss is indistinguishable from success.
13 — Parked records are terminal, unmonitored and not viewable at all
Nothing consumes the park topic, and the dead-letter dashboard's matcher is end-anchored so it matches the dead-letter topic but not the park topic. Observed 63 parked, 34 already aged out, no consumer group. Neither the dead-letter topic nor the park topic appears in any indexer config either, so neither is searchable through operational tooling — these failures are not viewable at all, not merely unmonitored.
14 — A duplicate-key rejection from a semantic unique is treated as success
The classifier maps unique-violation (23505) to benign unconditionally, whatever constraint fired. It does not bite on the top-level device key — a re-sent clientReferenceId is rejected at the service layer first as a duplicate entity, and that check searches with deleted rows included, so it never reaches the persister. It bites on every unique that check cannot see: the project beneficiary voucher tag, the individual's generated individualId, the per-individual identifier and address mappings, and the clientReferenceId uniques on the sub-entities — identifier, skill, address, task resource — which the top-level duplicate check never looks at. Soft-delete a record holding a printed voucher, re-register the same voucher — the voucher-tag check looks only at non-deleted rows while the constraint ignores isDeleted, so the violation is raised, classified benign, and the record discarded with zero artifact.
15 — One table has no uniqueness backstop on its link column
No PK on id, no unique constraint or index on clientReferenceId, bare insert. Observed two live rows sharing one clientReferenceId. The sibling table correctly raises a violation.
16 — Uniqueness guard is blind to the offline-shaped row
The "already a member" check filters on the individual's server id, so a stored row whose individualId is NULL — the normal offline shape — is invisible to it. Observed two membership rows for one individual. Same blind spot on the head-of-household check, which reads the existing members by whichever single household key column the request happens to carry. Two residuals the guard cannot close at all: it reads only rows already stored, so two DISTINCT records in the SAME batch naming the same person, each with its own clientReferenceId, both persist a membership row; and when the individual cannot be resolved the check is skipped outright rather than deferred.
17 — Deleting a member leaves the relationships the delete payload omits active
The server never looks up the member's stored relationship set; it only touches the relationship rows the request itself resends, and that field is optional on the request. Omit memberRelationships from a member _delete and zero relationship statements run — the rows stay isDeleted=false, pointing at a deleted person, and keep being reported, with a 202 either way and no signal that the cascade did not happen. Resend them and the cascade does run, but keyed on the member's selfId it clears every relationship of that member, including ones the request did not name.
18 — Link columns carry no index
Four clientReferenceId link columns are unindexed, across two tables: on referral, projectBeneficiaryClientReferenceId and sideEffectClientReferenceId; on side_effect, taskClientReferenceId and projectBeneficiaryClientReferenceId (catalog read: none of the four carries an index). The equivalent link columns on household_member, project_task and project_beneficiary ARE indexed, so this is confined to referral and side_effect. Presents as downsync/search slowdown at scale, not as an error.
19 — Whitespace-only clientReferenceId accepted with 202
An empty string is rejected with 400, whitespace-only is accepted. It is now the sole linking key, so that is a broken link key. On POST /stock/reconciliation/v1/_create the request body is taken without validation, so on that one endpoint NO constraint on the payload is evaluated at all — not the whitespace case, and not the pre-existing 2-64 length check that produces the 400 elsewhere; a zero-length clientReferenceId, or a missing facilityId, is accepted there. It is the only endpoint of its kind across all six services — the bulk reconciliation create and every other create, update and search body is validated.
20 — The app writes fabricated server ids into server-id columns
(a) an identifier's parent id is set to the parent's own client reference; (b) each relationship is stamped with the member's OWN server id as the relative's id, and one id triple is applied to all of them, cross-linking them; (c) the local read drops the three stored server ids and writes those nulls back on the next edit. No back-fill or repair exists for rows already written with fabricated ids — nothing in the app's local-store upgrade path and no server-side migration touches these columns — so correcting the write sites leaves every row already written wrong.
21 — Validation-error responses echo the caller's credentials
Reproduced. Filing separately as an advisory rather than describing it here.