Skip to content

Reject lossy generated DAML decoding#431

Draft
HardlyDifficult wants to merge 21 commits into
codex/decoder-backed-cancellation-readersfrom
codex/lossless-daml-decoding
Draft

Reject lossy generated DAML decoding#431
HardlyDifficult wants to merge 21 commits into
codex/decoder-backed-cancellation-readersfrom
codex/lossless-daml-decoding

Conversation

@HardlyDifficult

@HardlyDifficult HardlyDifficult commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Why this belongs in the SDK

DAML is authoritative for domain correctness: quantities, cardinality, identifier relationships, pricing rules, vesting graphs, and other business invariants should be accepted or rejected by the contract.

This PR addresses a different boundary. After contract data crosses into JavaScript, DAML cannot guarantee that the generated decoder:

  • observes one stable input rather than caller mutation or accessor/proxy side effects
  • preserves the complete wire value instead of silently dropping or coercing structure
  • keeps DAML maps, null-prototype records, dangerous own keys, timestamps/numerics, and negative zero lossless
  • returns detached, deeply readonly data that callers cannot mutate back into shared decoder state
  • reports codec/transport failures as bounded structured SDK errors

The intended check is therefore structural fidelity: snapshot the untrusted input once, give decoder and encoder isolated clones, decode exactly once, and compare independent source/decoded/re-encoded evidence. It does not decide whether the decoded contract data is semantically valid.

Scope after the DAML-validation pivot

Keep:

  • generated template-correlated codec typing
  • per-call ownership isolation and immutable detached results
  • lossless structural round-trip checks at the JSON/generated-DAML boundary
  • structured codec failures without invoking hostile getters or proxy traps

Remove:

  • positive/non-empty/cardinality checks
  • Party, identifier, pricing, relationship, or other domain validation
  • acceptance/cancellation semantic checks inherited from parent PRs

Current stack status

The current branch is stacked on #430 and still contains inherited semantic validation. It must not merge as-is. The useful #431 transport/ownership slice will be extracted onto a clean base after the approved pivot PRs land, and will receive fresh exact-head validation and review before any merge decision.

Existing branch validation

  • npm run test:ci — 99 suites / 5,794 tests passed
  • declaration, package-consumer, typecheck, build, lint, format, and package dry-run checks passed on the existing head
  • those results describe the historical stacked head, not the future cleaned extraction

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0448502a-ae3e-4fe7-ae86-ff1b4f81166b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/lossless-daml-decoding

Comment @coderabbitai help to get the list of available commands.

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

@coderabbitai pause

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown
✅ Action performed

Reviews paused.

…ssless-daml-decoding

# Conflicts:
#	src/functions/OpenCapTable/capTable/damlEntityData.ts
…ossless-daml-decoding

# Conflicts:
#	src/functions/OpenCapTable/capTable/acceptanceContractData.ts
#	src/functions/OpenCapTable/capTable/cancellationContractData.ts
#	src/functions/OpenCapTable/capTable/damlCodecLosslessness.ts
#	src/functions/OpenCapTable/capTable/damlEntityData.ts
#	test/functions/cancellationReaders.test.ts
…n-readers' into codex/lossless-daml-decoding
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Reconciled the final #430 parent and tightened this PR at exact head a1bdbe24f9fb63fa18eb39431ac4e78985f793a0. The stacked delta is now exactly one file: test/functions/damlEntityDataLosslessness.test.ts.

Improvements after independent audit:

  • every negative case asserts OcpParseError, lossy_daml_decode, exact public source, and exact context.fieldPath/decoder path
  • redundant acceptance/cancellation wrapper cases were removed because Validate cancellation reader payloads #430 covers them more comprehensively
  • added a real public OcpClient.OpenCapTable.stockTransfer.get boundary regression proving exact diagnostics survive the consumer API
  • retained focused optional-scalar/object, unknown-field, nested-record, and lossless-valid coverage

Validation on @fairmint/canton-node-sdk@0.0.227 is green: build, declarations, exact public config, lint, format, 92 suites / 4,757 tests, and a final focused 8-test run after fixture cleanup. The worktree is clean. This PR remains draft pending fresh exact-head CI and Copilot-first review.

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Final stack reconciliation complete at exact head e079e00, incorporating #430 at 1abca30. The stacked delta remains test-only: test/functions/damlEntityDataLosslessness.test.ts is the sole diff from the updated parent, with no production-code delta. Full validation passed on canton-node-sdk 0.0.227: build, declaration checks, exact public config, source typecheck, ESLint, Prettier, focused losslessness suite (1 suite / 8 tests), and full Jest coverage (92 suites / 4,810 tests). The temporary node_modules symlink was removed; the worktree is clean. The PR remains draft and unassigned while CI runs.

@HardlyDifficult HardlyDifficult marked this pull request as ready for review July 11, 2026 21:21
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

@copilot review

Please review exact head e079e00. Copilot is the first requested reviewer for this material revision. The stacked delta against codex/decoder-backed-cancellation-readers is exactly one losslessness test file: test/functions/damlEntityDataLosslessness.test.ts. Local full validation passed 92 suites / 4,810 tests. Exact build-and-test and full QuickStart are SUCCESS, GitHub reports CLEAN/MERGEABLE, the PR is unassigned, and there are zero unresolved review threads.

Copilot AI commented Jul 11, 2026

Copy link
Copy Markdown

@copilot review

Please review exact head e079e00. Copilot is the first requested reviewer for this material revision. The stacked delta against codex/decoder-backed-c...

Reviewed exact head e079e001f75837ebf07a387d0793203571e25ab1 against codex/decoder-backed-cancellation-readers. The stacked delta is still only test/functions/damlEntityDataLosslessness.test.ts; focused suite passes locally (8/8), and I did not find issues requiring code changes in this revision.

@HardlyDifficult HardlyDifficult marked this pull request as draft July 11, 2026 21:24
…n-readers' into codex/lossless-daml-decoding
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Propagated the corrected #426#430 ancestry into this lossless-decoding proof branch at exact head 8b04e25.

Ancestry: #430 head 206699f and #426 fix d999c29 are ancestors of this head.

Stack invariant reverified against updated #430:

  • the delta is exactly test/functions/damlEntityDataLosslessness.test.ts
  • no other file is present in the stacked diff

Validation on @fairmint/canton-node-sdk 0.0.227:

  • focused losslessness: 8/8
  • focused transaction ordering: 72/72
  • typecheck passed
  • build passed
  • built declaration checks passed
  • exact public config checks passed
  • lint and format passed
  • test:ci passed
  • git diff --check passed

The merge was conflict-free. The temporary dependency symlink was removed and the worktree is clean. This PR remains draft and unassigned; no reviewer was requested.

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Propagated exact #430 head 3558b0159dda80ed48696ecc493f2f1446a104af with a normal merge commit. New exact #431 head: 0c71a1339436225ab3a281b38e27bc6334cbee76.

The stacked delta remains exactly test/functions/damlEntityDataLosslessness.test.ts with unchanged patch ID 25fd9fa6; no source or other test file leaked into the slice. Validation on @fairmint/canton-node-sdk@0.0.227 is fully green: 93 focused losslessness+sorting tests; build; declarations; exact public config; source typecheck; lint; formatting; and test:ci with 92 suites / 4,824 tests. The temporary dependency symlink was removed and the worktree is clean. PR remains draft/unassigned; no reviewer requested.

…n-readers' into codex/lossless-daml-decoding
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Propagated exact #430 head bbae333 with a normal merge commit. New exact #431 head: 6719439.

The stacked delta remains exactly test/functions/damlEntityDataLosslessness.test.ts with unchanged patch ID 25fd9fa6; no source or other test file leaked into the slice. Validation on @fairmint/canton-node-sdk@0.0.227 is fully green: acceptance readers 85/85; focused losslessness 8/8; build; declarations; exact public config; source typecheck; lint; formatting; and test:ci with 92 suites / 4,824 tests. The temporary dependency symlink was removed and the worktree is clean. PR remains draft/unassigned; no reviewer requested.

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Material parent propagation complete at exact head e2e40e8, incorporating #430 at 2c33641. Patch stability is exact: the sole stacked delta remains A test/functions/damlEntityDataLosslessness.test.ts, with no production-code delta. Full validation passed on canton-node-sdk 0.0.227: build, declarations, exact public config, explicit typecheck, ESLint, Prettier, focused losslessness suite (1 suite / 8 tests), and full test:ci (92 suites / 4,836 tests). The temporary node_modules symlink was removed and the worktree is clean. The PR remains draft and unassigned while CI runs.

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Fresh lossless-decoder audit hardening is pushed at exact head c1c03571386fd315be6feb2c093c178ec6c03c00.

The audit reproduced a global identity-trust bypass: once any object had decoded successfully, later same- or cross-codec calls skipped decoding entirely, so deleting required fields or reusing the object with another codec could pass. The global WeakSet/recursive marking is removed. A current-call-only validateDecodedGeneratedDamlValue now validates the structured decoder result immediately, avoiding only the generic dispatch’s synchronous second decode; every later decode call always runs the supplied codec. There is no persistent token/cache, freeze, or mutable-type lie.

Regressions cover required-field deletion, nested mutation and exact paths, current-call handoff, repeated same-codec calls, cross-codec reuse, clones, and public reader behavior. Direct proof reports deletion=decode, cross=decode, firstCalls=2, secondCalls=1. Numeric/Time string preservation remains intentionally lossless at this layer; native converters enforce semantic validity.

Validation: focused 5 suites / 431 tests; full 92 suites / 4,841 tests; build; typecheck; declarations; exact public-config; lint; format. The PR remains draft/unassigned; no parent merge or reviews were requested.

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Corrected parent stack propagated through PR #431 at exact head 40428ed. Lossless generated-DAML validation keeps its decoded-value handoff confined to the current synchronous call: there is no global decoded-object WeakSet or identity cache, every later decode runs the selected codec again, and mutation/cross-codec/cloned-value diagnostics retain exact paths. Acceptance, cancellation, strict-map, exact wrapper/template/contract-ID, sorter, and single-fetch parent invariants remain intact. I also moved the public-boundary losslessness fixture onto the current canton-node-sdk client shape required by the corrected parent.

Local validation: focused losslessness/acceptance/cancellation/dispatcher/single-contract/map/sorter suite 8 suites / 579 tests; full coverage suite 97 suites / 5,661 tests; clean build and declaration checks; exact public-config checks; built cardinality; package-consumer declarations; TypeScript; ESLint; Prettier; diff checks; canonical inventory 55 variants / 0 ingestion aliases (56 reachable objects, 160 resources). The worktree is clean and local/remote heads match. PR remains draft, unassigned, with no review requests while fresh CI runs. Per propagation scope, no work was performed on #432 or later.

@HardlyDifficult HardlyDifficult marked this pull request as ready for review July 12, 2026 07:29
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

@copilot review

Please review exact current head 40428ed307abfb87fa8551097609390ad50bfa9a. Pre-review gate: both required checks are successful, GitHub is CLEAN/MERGEABLE, the PR is unassigned, and zero review threads are unresolved. Focus on lossless DAML decoding, generated wrapper/data boundaries, hostile proxy/getter/thenable handling, diagnostic precedence, and public typing. Report every actionable current-head issue before the other reviewers.

Copilot AI commented Jul 12, 2026

Copy link
Copy Markdown

@copilot review

Please review exact current head 40428ed307abfb87fa8551097609390ad50bfa9a. Pre-review gate: both required checks are successful, GitHub is CLEAN/MERGEABLE, the PR is unassigned, and...

Reviewed exact head 40428ed307abfb87fa8551097609390ad50bfa9a against codex/decoder-backed-cancellation-readers, including lossless decode flow, wrapper/data boundaries, hostile-structure handling, diagnostic precedence, and public typing. I did not find actionable current-head issues. I also re-ran focused coverage locally: test/functions/damlEntityDataLosslessness.test.ts and test/converters/genericConversionReadBoundaries.test.ts (110 tests passed).

@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

@cursor review

Please review exact current head 40428ed307abfb87fa8551097609390ad50bfa9a. Copilot completed its required exact-head pass with no actionable findings: #431 (comment). Both required checks are green, GitHub is CLEAN/MERGEABLE, and zero review threads are unresolved. Focus on lossless DAML decoding, generated wrapper/data boundaries, hostile proxy/getter/thenable handling, diagnostic precedence, and public typing.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 40428ed. Configure here.

@HardlyDifficult HardlyDifficult marked this pull request as draft July 12, 2026 08:35
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Parent propagation is complete at exact head 855d5c1 via a normal conflict-free merge of #430 4647d3d. The non-overlapping parent and child layers remain byte-identical. Validation passes: 15 focused suites / 1,198 tests, full 97 suites / 5,666 tests, typecheck, lint, formatting, declarations, exact-public/cardinality, and package-consumer gates. Independent semantic inspection found no issue. Keeping this PR in draft until fresh remote CI is green.

…n-readers' into codex/lossless-daml-decoding

# Conflicts:
#	src/functions/OpenCapTable/capTable/acceptanceContractData.ts
#	src/functions/OpenCapTable/capTable/damlEntityData.ts
#	src/functions/OpenCapTable/capTable/damlToOcf.ts
#	src/utils/generatedDamlValidation.ts
#	test/utils/generatedDamlValidation.test.ts
@HardlyDifficult

Copy link
Copy Markdown
Collaborator Author

Decoder-ownership remediation and the refreshed #430 parent are pushed at exact head a419033dd9919c05578728339bc9bc4f7dededed. Full validation passes: 99 suites / 5,794 tests, declarations, build, package-consumer, exact-public-config, lint, formatting, and package dry-run. Hosted CI and OCP QuickStart are running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants