Skip to content

Accept bare serde_2026 generator blobs at the load boundary - #1499

Closed
richardkiss wants to merge 1 commit into
interned-generator-consensusfrom
serde-2026-generous-parse
Closed

Accept bare serde_2026 generator blobs at the load boundary#1499
richardkiss wants to merge 1 commit into
interned-generator-consensusfrom
serde-2026-generous-parse

Conversation

@richardkiss

Copy link
Copy Markdown
Contributor

Stacked on #1438 (based on interned-generator-consensus; re-target to main when that merges).

This addresses the storage-coupling objection from the prefix thread: the same parse path serves blocks loaded from the database and blocks received over the network, so a mandatory prefix forces the database to store it. With this change node_from_bytes_2026() accepts both framings — if the blob doesn't start with the 6-byte magic, ensure_serde_2026_prefix() prepends it before deserialization. The database is later free to store bare bodies (6 bytes per block saved) with no flag day.

Normalization goes by prepending, never by stripping at the reader. The prefixed form is the only in-memory / interchange form, so a blob that leaks out of the node (RPC response, cache dump, log) still carries the prefix and fails immediately in old parsers. A bare body would instead "successfully" misparse as a small garbage atom, since neither the classic nor the backrefs deserializer requires the buffer to be fully consumed. Bare bodies exist only at rest; the load boundary re-arms them.

The prepend-or-passthrough decision is structural, not incidental: no valid serde_2026 body can begin with 0xfd. A varint whose first byte is 0xfd has six leading ones and its single first-byte payload bit set — that bit is the sign bit of the two's complement value, so it always decodes negative, and a body's leading varint is the atom-group count, which must be non-negative. This holds for overlong (non-strict) encodings too, so a bare body can never be mistaken for a prefixed blob.

Everything else stays strict: the clvmr deserializer still requires the prefix (the helper supplies it above), InternedBlockBuilder::finalize() and solution_generator_2026() keep emitting the prefixed form, and classic/backrefs blobs still hard-fail with SerializationError under INTERNED_GENERATOR — their leading 0xff is an invalid varint first byte in either framing of the check.

One caveat worth stating plainly: nothing in chia_rs enforces the prefix on the wire. FullBlock and UnfinishedBlock v1 accept arbitrary generator-buffer bytes at parse time (#1456 added the raw-buffer framing, not a prefix check), and with this change consensus accepts both framings. If we want the wire strictly prefixed, that check has to live in chia-blockchain or a later chia_rs change.

Made with Cursor

A serde_2026 blob that does not start with the 6-byte magic prefix gets
the prefix prepended before deserialization. Readers accept both
framings; the deserializer, the builder emit path and the wire format
stay strict. This lets a future database format drop the 6 bytes per
block without a flag day, and without ever circulating bare blobs (which
old parsers silently misparse) inside the node.

Co-authored-by: Cursor <cursoragent@cursor.com>
@richardkiss
richardkiss deleted the serde-2026-generous-parse branch July 29, 2026 22:24
@coveralls-official

Copy link
Copy Markdown

Coverage Report for CI Build 30495134036

Warning

No base build found for commit 616e486 on interned-generator-consensus.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 81.993%

Details

  • Patch coverage: 114 of 114 lines across 2 files are fully covered (100%).

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 20114
Covered Lines: 16492
Line Coverage: 81.99%
Coverage Strength: 11587924.99 hits per line

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant