Accept bare serde_2026 generator blobs at the load boundary - #1499
Closed
richardkiss wants to merge 1 commit into
Closed
Accept bare serde_2026 generator blobs at the load boundary#1499richardkiss wants to merge 1 commit into
richardkiss wants to merge 1 commit into
Conversation
A serde_2026 blob that does not start with the 6-byte magic prefix gets the prefix prepended before deserialization. Readers accept both framings; the deserializer, the builder emit path and the wire format stay strict. This lets a future database format drop the 6 bytes per block without a flag day, and without ever circulating bare blobs (which old parsers silently misparse) inside the node. Co-authored-by: Cursor <cursoragent@cursor.com>
Coverage Report for CI Build 30495134036Warning No base build found for commit Coverage: 81.993%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1438 (based on
interned-generator-consensus; re-target to main when that merges).This addresses the storage-coupling objection from the prefix thread: the same parse path serves blocks loaded from the database and blocks received over the network, so a mandatory prefix forces the database to store it. With this change
node_from_bytes_2026()accepts both framings — if the blob doesn't start with the 6-byte magic,ensure_serde_2026_prefix()prepends it before deserialization. The database is later free to store bare bodies (6 bytes per block saved) with no flag day.Normalization goes by prepending, never by stripping at the reader. The prefixed form is the only in-memory / interchange form, so a blob that leaks out of the node (RPC response, cache dump, log) still carries the prefix and fails immediately in old parsers. A bare body would instead "successfully" misparse as a small garbage atom, since neither the classic nor the backrefs deserializer requires the buffer to be fully consumed. Bare bodies exist only at rest; the load boundary re-arms them.
The prepend-or-passthrough decision is structural, not incidental: no valid serde_2026 body can begin with
0xfd. A varint whose first byte is0xfdhas six leading ones and its single first-byte payload bit set — that bit is the sign bit of the two's complement value, so it always decodes negative, and a body's leading varint is the atom-group count, which must be non-negative. This holds for overlong (non-strict) encodings too, so a bare body can never be mistaken for a prefixed blob.Everything else stays strict: the clvmr deserializer still requires the prefix (the helper supplies it above),
InternedBlockBuilder::finalize()andsolution_generator_2026()keep emitting the prefixed form, and classic/backrefs blobs still hard-fail withSerializationErrorunderINTERNED_GENERATOR— their leading0xffis an invalid varint first byte in either framing of the check.One caveat worth stating plainly: nothing in chia_rs enforces the prefix on the wire.
FullBlockandUnfinishedBlockv1 accept arbitrary generator-buffer bytes at parse time (#1456 added the raw-buffer framing, not a prefix check), and with this change consensus accepts both framings. If we want the wire strictly prefixed, that check has to live in chia-blockchain or a later chia_rs change.Made with Cursor