A record can cross the lane as a document instead of a string - #1347
Merged
Conversation
The lane carries a record's stored JSON as a STRING inside the JSON envelope, so the same bytes are converted to JSON twice: the proxy escapes every quote in the payload into a fresh allocation on the way out, and the reader parses the envelope and then parses each record's string again. The stored bytes are already JSON -- both sides are converting JSON to JSON. It is the largest avoidable cost on a lane where JSON decode is most of the gateway's CPU: a sampled profile put raw_decode at 49.4% of gateway self time with another 15.3% in the lane reader. RecordPayload::Inline embeds the stored bytes verbatim with serde_json's RawValue. Nothing is escaped on the way out and the reader parses once. A caller gets it only by asking (records_inline_json), so a reader that still expects a string sees the byte-identical shape it always saw. The two sides deploy independently, so this is a negotiation, not a flag flip -- a lane whose codec changes under a running reader is an outage rather than a speedup. Validation before wrapping is not optional. RawValue is emitted VERBATIM, so a payload that is not JSON would make the WHOLE batch malformed rather than one record; a payload that does not parse falls back to the string shape. The readers accept both shapes now (lane_record_payload), which is what lets the sides be switched over one at a time. That half is inert today: every value is still a string and the helper returns the same dict the try/except produced. Gates. cargo test --bin matrixark_rust_proxy: 3 passed -- this file compiles through src/bin, so --lib does not run these at all and reports success while executing none of them. The default stays a string and survives the reader's second parse; an inline payload carries the identical record, asserted equal to what the string path produces; a non-JSON payload falls back rather than corrupting the batch. Disabling the inline path fails one of the three, so they discriminate. tools/test_a_lane_record_reads_the_same_either_shape.py: 3 passed. No end-to-end CPU delta is claimed. Three attempts to measure lane bytes per call were each contaminated -- by a concurrent soak, then by the gateway's own background passes, which produced 1.34 GB for one ingest against a 328 MB store. Process counters cannot isolate a request here, so the number is left unreported rather than reported badly. The case is structural: the second parse and the escaping are both visible in the code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The lane carries a record's stored JSON as a string inside the JSON envelope, so the same
bytes are converted to JSON twice. The proxy escapes every quote in the payload into a fresh
allocation on the way out, and the reader parses the envelope and then parses each record's
string again. The stored bytes are already JSON: both sides are converting JSON to JSON.
That matters because JSON decode is where the gateway's CPU goes. A sampled profile under load
put
raw_decodeat 49.4% of gateway self time with another 15.3% in the lane reader, and theproxy's own hottest frames are the construction and teardown of the matching values.
What changes
RecordPayload::Inlineembeds the stored bytes verbatim using serde_json'sRawValue. Nothingis escaped on the way out, and the reader parses once.
A caller only gets it by asking (
records_inline_json). A reader that still expects a stringsees the byte-identical shape it always saw. The two sides deploy independently, so this is a
negotiation rather than a flag flip -- a lane whose codec changes under a running reader is an
outage, not a speedup.
Validation before wrapping is not optional.
RawValueis emitted verbatim, so a payload thatis not JSON would make the whole batch malformed rather than one record. A payload that does not
parse falls back to the string shape.
The readers accept both shapes now (
lane_record_payload), which is what lets the two sides beswitched over one at a time. That half is inert today: every value is still a string, and the
helper returns the same dict the callers'
try/exceptproduced.Gates
cargo test --bin matrixark_rust_proxy-- 3 passed. This file compiles throughsrc/bin, so--libdoes not run these tests at all; running it that way reports success while executingnone of them.
produces, so the shape changes and the content does not
tools/test_a_lane_record_reads_the_same_either_shape.py-- 3 passed: both shapes read thesame, the document shape is returned without a parse, junk stays empty rather than raising.
What this does not claim
No end-to-end CPU delta is quoted. Three attempts to measure lane bytes per call were each
contaminated -- by a concurrent soak, and then by the gateway's own background passes, which
produced a reading of 1.34 GB for a single ingest against a 328 MB store. Process-level counters
cannot isolate one request here, so the number is not reported rather than reported badly. The
case for this change is structural: the second parse and the escaping are visible in the code.
The enable flag stays off until the
batch_hgetconsumers are audited; several of them still dostr(row.get("value"))and would read a document as its repr.