Skip to content

A record can cross the lane as a document instead of a string - #1347

Merged
bjmeetsfo merged 1 commit into
mainfrom
perf/a-record-crosses-the-lane-as-a-document
Sep 8, 2026
Merged

A record can cross the lane as a document instead of a string#1347
bjmeetsfo merged 1 commit into
mainfrom
perf/a-record-crosses-the-lane-as-a-document

Conversation

@bjmeetsfo

Copy link
Copy Markdown
Collaborator

The lane carries a record's stored JSON as a string inside the JSON envelope, so the same
bytes are converted to JSON twice. The proxy escapes every quote in the payload into a fresh
allocation on the way out, and the reader parses the envelope and then parses each record's
string again. The stored bytes are already JSON: both sides are converting JSON to JSON.

That matters because JSON decode is where the gateway's CPU goes. A sampled profile under load
put raw_decode at 49.4% of gateway self time with another 15.3% in the lane reader, and the
proxy's own hottest frames are the construction and teardown of the matching values.

What changes

RecordPayload::Inline embeds the stored bytes verbatim using serde_json's RawValue. Nothing
is escaped on the way out, and the reader parses once.

A caller only gets it by asking (records_inline_json). A reader that still expects a string
sees the byte-identical shape it always saw. The two sides deploy independently, so this is a
negotiation rather than a flag flip -- a lane whose codec changes under a running reader is an
outage, not a speedup.

Validation before wrapping is not optional. RawValue is emitted verbatim, so a payload that
is not JSON would make the whole batch malformed rather than one record. A payload that does not
parse falls back to the string shape.

The readers accept both shapes now (lane_record_payload), which is what lets the two sides be
switched over one at a time. That half is inert today: every value is still a string, and the
helper returns the same dict the callers' try/except produced.

Gates

  • cargo test --bin matrixark_rust_proxy -- 3 passed. This file compiles through src/bin, so
    --lib does not run these tests at all; running it that way reports success while executing
    none of them.
    • the default stays a string and still survives the reader's second parse
    • an inline payload carries the identical record -- asserted equal to what the string path
      produces, so the shape changes and the content does not
    • a non-JSON payload falls back rather than corrupting the batch
    • disabling the inline path fails one of the three, so they discriminate
  • tools/test_a_lane_record_reads_the_same_either_shape.py -- 3 passed: both shapes read the
    same, the document shape is returned without a parse, junk stays empty rather than raising.

What this does not claim

No end-to-end CPU delta is quoted. Three attempts to measure lane bytes per call were each
contaminated -- by a concurrent soak, and then by the gateway's own background passes, which
produced a reading of 1.34 GB for a single ingest against a 328 MB store. Process-level counters
cannot isolate one request here, so the number is not reported rather than reported badly. The
case for this change is structural: the second parse and the escaping are visible in the code.

The enable flag stays off until the batch_hget consumers are audited; several of them still do
str(row.get("value")) and would read a document as its repr.

The lane carries a record's stored JSON as a STRING inside the JSON envelope,
so the same bytes are converted to JSON twice: the proxy escapes every quote in
the payload into a fresh allocation on the way out, and the reader parses the
envelope and then parses each record's string again. The stored bytes are
already JSON -- both sides are converting JSON to JSON. It is the largest
avoidable cost on a lane where JSON decode is most of the gateway's CPU: a
sampled profile put raw_decode at 49.4% of gateway self time with another 15.3%
in the lane reader.

RecordPayload::Inline embeds the stored bytes verbatim with serde_json's
RawValue. Nothing is escaped on the way out and the reader parses once. A
caller gets it only by asking (records_inline_json), so a reader that still
expects a string sees the byte-identical shape it always saw. The two sides
deploy independently, so this is a negotiation, not a flag flip -- a lane whose
codec changes under a running reader is an outage rather than a speedup.

Validation before wrapping is not optional. RawValue is emitted VERBATIM, so a
payload that is not JSON would make the WHOLE batch malformed rather than one
record; a payload that does not parse falls back to the string shape.

The readers accept both shapes now (lane_record_payload), which is what lets the
sides be switched over one at a time. That half is inert today: every value is
still a string and the helper returns the same dict the try/except produced.

Gates. cargo test --bin matrixark_rust_proxy: 3 passed -- this file compiles
through src/bin, so --lib does not run these at all and reports success while
executing none of them. The default stays a string and survives the reader's
second parse; an inline payload carries the identical record, asserted equal to
what the string path produces; a non-JSON payload falls back rather than
corrupting the batch. Disabling the inline path fails one of the three, so they
discriminate. tools/test_a_lane_record_reads_the_same_either_shape.py: 3 passed.

No end-to-end CPU delta is claimed. Three attempts to measure lane bytes per
call were each contaminated -- by a concurrent soak, then by the gateway's own
background passes, which produced 1.34 GB for one ingest against a 328 MB store.
Process counters cannot isolate a request here, so the number is left unreported
rather than reported badly. The case is structural: the second parse and the
escaping are both visible in the code.
@bjmeetsfo
bjmeetsfo merged commit bc88565 into main Sep 8, 2026
7 checks passed
@bjmeetsfo
bjmeetsfo deleted the perf/a-record-crosses-the-lane-as-a-document branch September 8, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant