Fc: stream JSON responses directly, bypassing the variant tree

[heifner]

Emits HTTP JSON responses token-by-token into the output buffer through fc::json_writer instead of building an intermediate fc::variant tree and re-serializing it with fc::json::to_string.

Core pieces

• fc::json_writer (fc/io/json_stream.hpp): append-only token writer with object/array frame tracking, checkpoint()/rewind() for atomic rollback of partial emission, locale-independent numeric emission via std::to_chars, and the same 64-bit integer quoting threshold fc::json::to_string uses.
• to_json_stream overload family: primitives and standard containers are co-located with their to_variant siblings (variant.hpp, container/flat.hpp) so shape and MAX_NUM_ARRAY_ELEMENTS guards stay in lock-step; a reflector visitor gives every FC_REFLECT'd struct a streaming serializer for free; a depth-bounded streaming walker covers fc::variant / variant_object values (json_stream_max_depth matches json.cpp's recursion cap).
• FC_SERIALIZE_AS_STRING trait: string-shaped types (name, asset, symbol, symbol_code, block_timestamp, the sha/ripemd160/keccak256/blake3 hashes, BLS keys/signatures, bitset, url) declare one to_string()/from_string() pair that drives to_variant, from_variant, and to_json_stream, replacing the per-type hand-rolled overload triplets. The hash from_string factories reject odd-length hex, keeping the strictness the previous vector<char> from_variant path enforced. Dead fc::fixed_string is removed. to_json_stream(bls::private_key) is deleted so key material cannot be streamed into a response.
• abi_serializer::binary_to_json_stream: a single _binary_walk<Sink> traversal drives both the variant_sink (the existing binary_to_variant) and the new stream_sink (tokens straight into the writer), so shape parity between the two paths is structural rather than by convention. Action-data decode is atomic: unpack_action_data_field checkpoints, rewinds on failure, and the caller emits the hex fallback under the same key.
• http_plugin streaming callback path + bind_stream<>: registration template with compile-time signature pinning; response serialization happens on the http thread pool while the api thread hands off the result struct by move. chain_api, producer_api, net_api, db_size_api, and trace_api get_block/get_transaction_trace are migrated; endpoint sets, queues, categories, and response codes are unchanged. Stored exceptions rethrow with their dynamic type so the status classifier keeps mapping tx_duplicate to 409, unknown_block to 400, etc., and queued streaming request bodies take the same bytes_in_flight reservation as the variant path.
• Deferred ABI serialization: get_block and trx_retry capture raw ABI blobs at block time (captured_abis) and build serializers at emit time on the http pool, keeping ABI decode off the main thread; memory for captured blobs is accounted in the trx_retry budget.

Performance

RelWithDebInfo, i9-12900K, benchmark -f trace_api_json / -f abi_serializer:

• trace_api get_block: -26% wall-clock at the 25k-transaction block shape and -44% on data-heavy multi-action blocks, versus the variant + json::to_string pipeline.
• abi_serializer JSON production: -25% to -36% across transfer/newaccount/regproducer/block shapes via binary_to_json_stream versus binary_to_variant + to_string.
• get_table_rows decodes rows straight from binary into the writer with no per-row variant tree.

Parity

Streamed output is pinned byte-identical to the variant path: every abi_tests round-trip case also runs verify_stream_matches_variant, endpoint-level byte-identical assertions cover get_account/get_block/get_table_rows, and trace_api has stream-vs-variant response parity cases including the decode-failure hex-fallback shapes.

[huangminghuang]

Inline review findings from the streaming JSON parity and resource-accounting review.

[huangminghuang]

[P1] Enforce the memory budget during emission

The response is fully materialized by emitter(w) before body.size() is charged to bytes_in_flight. Concurrent large streaming requests are therefore invisible to --http-max-bytes-in-flight-mb while allocating and serializing, and are rejected only after the memory cost has occurred. Please account growth incrementally or enforce a budget-aware writer limit before allowing the buffer to expand.

[heifner]

Now charged incrementally: json_writer grew an optional growth guard invoked at token boundaries every 64KiB of emission; the handler settles the buffer into bytes_in_flight at each checkpoint and aborts with the busy response once verify_max_bytes_in_flight rejects, so over-budget emissions stop within a stride of the cap instead of materializing. The abort re-arms across catch-and-continue serializers (abi hex-fallback) so it cannot be absorbed. Covered by stream_response_bytes_in_flight (mid-emission budget visibility) and stream_response_over_budget_aborts_early (503 + early abort + drain), both verified failing against the previous code.

[huangminghuang]

[P1] Preserve absent-policy null fields

Changing these fields from fc::variant to std::optional changes the endpoint schema: the streaming reflector omits disengaged optionals, whereas the previous default variants emitted active_finalizer_policy and pending_finalizer_policy as explicit null values. A missing pending policy is normal, so clients that access the key before testing for null can now fail. Please preserve explicit nulls with a custom emitter or equivalent nullable representation and add an absent-policy parity test.

[heifner]

Kept the typed fields via a new fc::nullable whose disengaged state serializes as explicit null under its key on both the variant and streaming paths (trio co-located with the std::optional overloads). Absent-policy parity test added: get_finalizer_info on a fresh chain asserts byte-identical variant/stream output containing pending_finalizer_policy:null.

[huangminghuang]

[P2] Preserve large-duration quoting

value_int64 bypasses the established 64-bit JSON quoting rule. For example, microseconds{0x100000000} now emits 4294967296, while the previous to_variant plus json::to_string path emits "4294967296". Please delegate to the guarded int64_t streaming overload and add positive/negative boundary tests around 0xffffffff.

[heifner]

Delegated to the guarded int64_t overload; boundary tests added on both sides of +-0xffffffff plus byte-parity against the variant path, verified failing against the previous code. Also routed stream_sink::value_int64/uint64 through the guarded overloads -- ABI built-in ints already streamed through the guarded generic in get_built_in_stream_unpacks, so no live divergence existed there, but the sink interface itself now cannot diverge for direct callers.

[huangminghuang]

[P1] Enforce the response budget within a single JSON token

guard_check() only runs from value_prefix()/key(), before this token is appended. A single large reflected string (for example an action console), value_hex, or raw_value can therefore append its entire contents after one pre-check, and the HTTP handler only rejects it in the final settle_to after the full buffer has already been allocated. The new over-budget test emits many separate 64 KiB tokens, so it does not exercise this case. Please invoke the guard while the escape/hex/raw loops cross each stride (including escape expansion), and add a regression where one token alone exceeds the response budget.

[heifner]

Closed: guarded writers now check within tokens. value_string/key route the escape loop's periodic yield (every 128 input chars) into the same stride-gated guard check, measuring the output buffer so escape expansion counts; value_hex and raw_value emit in 64KiB chunks with a guard check between chunks (unguarded writers keep the single tight loop and the empty yield they already passed). A guard throw mid-token is safe under the existing re-arm plus checkpoint/rewind contract. Tests: growth_guard_intra_token_abort (each of the three token kinds aborts within one chunk of the threshold on a 1-2MiB token), growth_guard_measures_escape_expansion (32KiB of control chars crossing a 100KiB output threshold the raw input never reaches), and stream_response_single_token_over_budget (single 32MiB string value against a 1MiB budget: 503 with the emitter's post-token statement never reached), all verified failing against the token-boundary-only code.

[huangminghuang]

Four inline findings from the current-head review.

[huangminghuang]

[P1] Avoid reserving the full guarded string

The guarded writer calls escape_string, which immediately reserves input.size() + 13 bytes before its periodic yield can enforce the growth budget. A single 32 MiB string therefore allocates roughly 32 MiB even under a 1 MiB cap. The guarded path should reserve incrementally, with a regression test checking capacity/allocation rather than only out.size().

[heifner]

Fixed by checking before the reserve rather than reserving incrementally: value_string/key know the token's input length -- a guaranteed lower bound on its peak emission, since escaping never shrinks a byte pre-prune -- so guarded writers now consult the guard with the prospective size before escape_string runs. An over-budget string aborts with zero allocation and zero buffer mutation (capacity stays at the writer's initial slack), while an approved string keeps the single full-size reserve, so the legitimate path pays no geometric-growth tax and unguarded/legacy callers are untouched. Escape expansion beyond the approved size remains covered by the intra-loop checks from the previous round, and the http guard's signed settle_to reconciliation handles the prospective charge -- reserve-then-allocate, trued up at the next checkpoint. Regression: growth_guard_bounds_allocation drives a 32 MiB string and key at a 64 KiB threshold asserting out.capacity() stays below the threshold (was ~32 MiB), plus approved-path and unguarded controls pinning the full reserve, verified failing against the previous code.

[huangminghuang]

[P1] Charge queued emitter payloads

The typed result is captured by the emitter and queued on the HTTP thread without reserving any bytes-in-flight; accounting starts only when emission executes. Multiple large results can consequently accumulate outside http-max-bytes-in-flight-mb, unlike the variant path which charges before dispatch. Carry a source-size reservation through the queue and release it as the captured result is consumed.

[heifner]

bind_stream now reserves the typed result's packed-size estimate at the moment of capture -- dispatch::sync and the async typed-result arm, the two shapes with a real cross-thread queue window (post/post_direct produce on the http pool and the dispatch short-circuits inline). The RAII reservation (http_plugin::reserve_bytes_in_flight) rides inside the emitter closure and releases exactly when the captured result is destroyed after emission, so the source stays charged alongside the incrementally-charged output buffer -- both genuinely coexist, matching the variant path holding its in_flight_sizeof estimate while the response string builds. The dispatched handler also re-checks the budget before running the emitter, draining over-budget queued work with the busy response like the variant path's pre-serialization verify. Trace-bearing results report their retained memory (action data, console, return values, captured ABI bytes) through a queued_payload_size customization point, since streamed_processed_trace is emitted but never wire-packed; fc::nullable gained raw pack/unpack parity with std::optional so reflected results containing it can be sized. Regression: stream_queued_payload_reserved parks the single pool thread in a latched emitter and asserts the sampled bytes_in_flight covers a 1 MiB queued bind_stream payload while it waits behind the stall -- reads 0 without the reservation, verified -- then drains to zero.

[huangminghuang]

[P1] Use the guarded writer for trace responses

This builds the entire trace response in an unguarded std::string before passing it through the legacy json_raw callback, so the HTTP memory cap observes it only after allocation. Large or concurrent trace responses can bypass the new protection. These endpoints should emit directly into the streaming callback’s guarded writer, with a low-budget regression test.

[heifner]

Both endpoints moved off the json_raw pass-through onto add_async_api_stream: the request handler now produces an emitter -- misses (absent block, or trxid absent per the new contains_transaction) resolve to their 404 before a 200 emitter is committed -- and the emitter writes into the guarded json_writer the streaming callback provides, so the budget observes and can abort trace responses mid-serialize. Error responses emit error_results through the same streaming path handle_exception_stream uses. response_formatter grew writer-taking process_*_to_stream forms; the string-building forms remain as thin wrappers over them for tests and the benchmark. Tests: streaming_block_response_budget_abort drives the block emitter into a small-threshold guarded writer over a 32-transaction block and asserts the abort lands within a token of the threshold with the full body several times larger; streaming_emitter_miss_resolution_and_parity pins nullopt on every miss and byte-identical emitter output against the wrappers; the existing stream/variant parity suites are unchanged, and trace_plugin_test.py passes end-to-end over http.

[huangminghuang]

[P2] Preserve the sign of NaN

The streaming formatter always emits "nan" for NaN, while the legacy formatter emits "-nan" for a negative NaN. Arbitrary ABI float bit patterns can expose this, breaking the promised byte-for-byte serialization parity. Preserve std::signbit(d) and add positive/negative NaN parity coverage.

[heifner]

Fixed with signbit for both the nan and inf branches. Parity coverage: libfc leaf checks for positive and negative quiet NaN, both infinities, and a negative float NaN against the platform's variant output, plus abi_tests/non_finite_float_sign_parity packing raw negative-NaN float64/float32 bit patterns -- unreachable from JSON input -- and asserting binary_to_json_stream matches binary_to_variant + to_string byte-for-byte with the legacy "-nan" shapes pinned, verified failing against the previous code.