feat: add adaptive gzip compression for gRPC responses#747
Draft
He-Pin wants to merge 13 commits into
Draft
Conversation
…g matching Motivation: Per-request overhead in the hot path limits high-concurrency throughput. Protocol negotiation and URI path matching allocate objects and scan headers on every request, even when the values are predictable. Modification: - GrpcProtocol.negotiate(): Add a fast path for native gRPC that does a single header scan instead of calling Codecs.detect() and Codecs.negotiate() separately (which each scan all headers). Pre-compute common reader/writer combinations (Identity/Identity, Identity/Gzip) as static vals to avoid repeated object creation. - Handler.scala.txt template: Replace Uri.Path pattern matching with direct string operations on request.uri.path.toString, avoiding allocation of Path/Slash/Segment objects per request. Result: Reduced per-request allocations and header scanning overhead in the gRPC request processing hot path. Tests: - runtime / compile - passed - Validated with local benchmark (ghz, complex_proto scenario) References: None - performance optimization
Motivation: Flamegraph analysis showed ~14% of CPU time spent in Deflater.deflate (gzip compression) for response encoding. For typical small protobuf messages in gRPC benchmarks, gzip compression adds CPU overhead without meaningful size reduction. The default codec preference order (Gzip before Identity) caused the server to compress every response when the client advertised gzip support, even for tiny messages. Modification: - Codecs.supportedCodecs: Change order to Seq(Identity, Gzip) so the negotiate() function prefers Identity when both codecs are supported - GrpcProtocol.negotiate() fast path: Match the new preference order, choosing Identity when present in the accept-encoding list This matches the behavior of Vert.x gRPC server which does not compress responses by default. Result: - Eliminates ~14% CPU overhead from Deflater.deflate - Improves 99% tail latency (33.75ms -> 32.33ms in local benchmark) - Low concurrency: 55,357 -> 56,177 req/s (+1.5%) - Aligns with Vert.x default behavior for fair comparison Tests: - runtime / compile - passed - Validated with local benchmark (ghz, complex_proto scenario) References: None - performance optimization
…erhead Motivation: Flamegraph analysis showed ~27% of handler CPU time spent in Gzip compression (Deflater.init, Deflater.deflate, GZIPOutputStream). For typical small gRPC messages (under 1KB), gzip compression adds significant CPU overhead without meaningful size reduction. The ghz benchmark client sends grpc-accept-encoding: gzip (without identity), causing the server to compress every response. Modification: - GrpcProtocol.negotiate(): For native gRPC, always prefer Identity writer codec for responses. The per-frame compression flag (bit 0 of the 5-byte gRPC frame header) tells the client whether each frame is compressed, so clients correctly handle uncompressed frames even when they advertised gzip support. - This enables the serializeDataFrame fast path in responseForSingleElement, which creates pre-framed uncompressed data in a single byte array allocation, avoiding the Gzip compress → encodeDataToResponse chain. - ScalapbProtobufSerializer.deserialize(): Use CodedInputStream with byte array directly (via asByteBuffer.hasArray) to avoid ByteBuffer wrapper allocation. Result: - High concurrency (1000 conn): 72,027 -> 78,585 req/s (+9.1%) - Combined with all optimizations: 59,177 -> 78,585 req/s (+32.8%) - Gap to Vert.x (81,100) reduced from 18% to 3.1% - Low concurrency: 42,959 -> 54,101 req/s (+25.9%, now 43% faster than Vert.x) Tests: - runtime / compile - passed - Validated with local benchmark (ghz, complex_proto scenario) References: None - performance optimization
Motivation: Flamegraph analysis showed handler lambda creation as a potential allocation hotspot. For unary gRPC methods, the lambda (e: Request) => implementation.method(e) was created inside the handle function which is called per request. Modification: Move lambda creation outside the handle function by pre-creating val handlers for each unary method. The lambdas are now created once during service registration instead of per request. Result: Reduces per-request object allocation by eliminating lambda creation. Benchmark shows similar performance (within margin of error) as JIT already optimizes simple lambdas, but this is a good practice for reducing GC pressure. Tests: - codegen / compile - passed - runtime / compile - passed - Benchmark verification with ghz (1000 concurrency, 50 connections) References: None - performance optimization
Motivation: The remaining gRPC benchmark gap is dominated by small per-request overheads. Native identity strict unary requests still used the generic ByteReader-based strict frame decoder, and the previous unary handler lambda optimization generated invalid Scala for keyword method names and Power API metadata. Modification: Add a native identity strict frame decoder that parses the gRPC frame header directly while preserving malformed-frame error semantics. Remove unused fast-path response accept-encoding scanning and the dead NativeIdentityGzip tuple. Generate separate safe local handler names for unary Scala handlers, keep pre-created lambdas for non-Power APIs, and keep Power API lambdas request-scoped so metadata is in scope. Remove the accidentally committed Scala handler template backup. Result: Strict native identity unary decoding avoids the generic ByteReader allocation path. Scala generated handlers compile for keyword method names such as Match, while Power API handlers continue to receive request metadata correctly. Tests: scalafmt on changed Scala files; sbt plugin-tester-scala / Compile / compile; sbt runtime / Test / testOnly GrpcMarshallingSpec CodecsSpec; sbt benchmarks / Jmh / run quick GrpcMarshalling strict unmarshal benchmarks; git diff --check. References: None - local performance optimization follow-up from OPTIMIZATION_HANDOFF.md
Motivation: The unary gRPC hot path allocated unnecessary objects on every request: 1. handleUnary called unaryExceptionHandler which created a new PartialFunction via GrpcExceptionHandler.from() on every error path (up to 4 allocations per request in worst case) 2. encodeFrameData allocated a separate 5-byte header array plus a ByteStrings concatenation wrapper for every frame 3. Async recovery paths eagerly created exception handler PartialFunctions even when no exception occurred Modification: 1. Add handleUnaryException that inlines the exception-to-response conversion, avoiding the intermediate PartialFunction allocation from GrpcExceptionHandler.from(). Directly checks isDefinedAt and falls back to defaultMapper. 2. Optimize encodeFrameData for small messages (<=4KB, covering the vast majority of gRPC unary payloads) by combining the 5-byte frame header and data into a single array allocation, producing one contiguous ArrayByteString instead of a ByteStrings(Header, Data) concatenation. Falls back to original approach for large messages. 3. Change async recovery exception handlers from eager val to lazy val so the PartialFunction is only allocated if recovery is actually needed. Result: Fewer object allocations per unary gRPC request on the hot path, reducing GC pressure under high concurrency. encodeFrameData produces a single contiguous ByteString for typical small messages, improving downstream serialization efficiency. Tests: sbt runtime / Test / testOnly GrpcMarshallingSpec CodecsSpec GrpcResponseHelpersSpec All 18 tests passed. References: None - local performance optimization follow-up from OPTIMIZATION_HANDOFF.md
…llocation Motivation: On the unary gRPC strict-entity fast path, decodeIdentitySingleFrame calls ByteString.slice(5, length) to strip the 5-byte gRPC frame header. This creates a ByteString1 wrapper object plus intermediate ByteBuffer wrappers when the sliced bytes reach the protobuf CodedInputStream. These allocations happen on every request. Modification: Add deserialize(data, offset, length) to ProtobufFrameSerializer trait with a default fallback to slice + deserialize. Override in ScalapbProtobufSerializer to access the backing byte array directly with the offset, bypassing all intermediate wrappers. Use this offset-aware path in GrpcMarshalling.handleUnary for strict entities when the serializer is a ProtobufFrameSerializer (which all generated ScalaPB and Java protobuf serializers are). Result: Strict-entity unary deserialization avoids the ByteString1 wrapper allocation from slice and the intermediate ByteBuffer wrappers from asByteBuffer, going directly from the raw gRPC frame byte array to CodedInputStream. JMH shows no statistically significant throughput change in single-threaded benchmarks (within error bars), but the reduced allocation count benefits GC under high concurrency. Tests: sbt runtime / Test / testOnly GrpcMarshallingSpec CodecsSpec GrpcResponseHelpersSpec All 18 tests passed. References: None - local performance optimization follow-up from OPTIMIZATION_HANDOFF.md
Motivation: The Java DSL unmarshal for strict entities used the same decodeSingleFrame path as before, creating a ByteString.slice wrapper per request. The Scala DSL already had the offset-aware optimization. Modification: Apply the same ProtobufFrameSerializer.deserialize(data, offset, length) fast path to the Java DSL unmarshal method for strict entities, bypassing the ByteString.slice allocation. Result: Java DSL strict entity unmarshalling now also avoids the ByteString1 wrapper allocation on the hot path, consistent with the Scala DSL. Tests: sbt runtime / Test / testOnly GrpcMarshallingSpec All 7 tests passed. References: None - follow-up to offset-aware deserialization optimization
…tring
Motivation:
The generated methodName function called request.uri.path.toString on
every request, triggering a full recursive renderPath that allocates a
StringBuilder, renders each path segment, and creates a new String.
This appeared as a hotspot (30 CPU samples) in async-profiler under
high concurrency gRPC workloads.
Modification:
Replace the toString + substring approach with direct Uri.Path pattern
matching. The gRPC path format /{service}/{method} maps to
Path.Slash(Segment(service, Slash(Segment(method, Empty)))). Pattern
matching on this structure extracts the method name with zero string
allocations - only field accesses and a String.equals comparison.
Result:
Eliminates per-request renderPath + StringBuilder + String allocations
in the generated handler. Under complex_proto benchmark (1000c/50conn/
120s), average latency dropped from 7.62ms to 6.88ms (-9.7%) and P99
from 28.56ms to 24.80ms (-13.2%).
Tests:
sbt runtime / Test / testOnly GrpcMarshallingSpec
All tests passed.
sbt codegen / compile
Compiled successfully.
References: None - performance optimization from flamegraph analysis
Motivation: The negotiated method used Option.map with a pattern-matching lambda, which allocates a new Function1 object on every request. This appeared as GrpcMarshalling$$$Lambda (48 CPU samples) in async-profiler. Modification: Replace Option.map with direct pattern matching on the negotiate result. The match on Some/Failure/Success is allocation-free (only type checks and field accesses via destructuring). Result: Eliminates one lambda allocation per gRPC request in the negotiated method hot path. Tests: sbt runtime / Test / testOnly GrpcMarshallingSpec All 7 tests passed. References: None - performance optimization from flamegraph analysis
…ndleUnary Motivation: The ProtobufFrameSerializer fast path in handleUnary directly called deserialize(data, offset, length) without checking if the data was sufficient to contain a frame header. When the request entity was empty (HttpEntity.empty), data.length - FrameHeaderSize produced a negative length (-5), bypassing the MissingParameterException that reader.decodeSingleFrame would normally throw. This caused the ErrorReportingSpec test "should respond with an 'invalid argument' gRPC error status when calling an method without a request body" to fail because the expected INVALID_ARGUMENT status was not returned. Modification: Add a length check before the ProtobufFrameSerializer.deserialize call: if data.length < FrameHeaderSize, throw MissingParameterException to match the behavior of reader.decodeSingleFrame which validates the frame header is present before attempting to parse it. Result: Empty request bodies now correctly return grpc-status: 3 (INVALID_ARGUMENT) regardless of whether the ProtobufFrameSerializer fast path or the standard decodeSingleFrame path is used. Tests: sbt plugin-tester-scala / Test / testOnly example.myapp.helloworld.ErrorReportingSpec All 2 tests passed. References: Fixes CI failure in PR #739
Motivation: The previous PR always used Identity (no compression) for native gRPC responses to avoid CPU overhead on small messages. However, large messages benefit significantly from gzip compression. An adaptive approach that compresses only messages above a configurable threshold provides the best of both worlds. Modification: - Add AdaptiveGzip codec that only compresses messages above a configurable threshold (default 1024 bytes). Messages below the threshold pass through uncompressed while the grpc-encoding header still advertises gzip. - Add Codec.compressWithFlag() returning (ByteString, Boolean) to signal per-frame compression status accurately in the gRPC frame header (bit 0 of the 5-byte header). This fixes a correctness bug where streaming frames would incorrectly set the compression flag even for uncompressed data. - Add pekko.grpc.server.compression-threshold config (default 1024). Generated Scala handlers read config once at handler creation and pass the threshold as Int to avoid per-request config overhead. - Pre-compute NativeIdentityAdaptiveGzip negotiation result and cache the default AdaptiveGzip instance for zero per-request allocation on the common path. - Add ProtobufFrameSerializer.serializedDataSize() for efficient size checking without double serialization. - Add adaptive fast path in GrpcResponseHelpers for small messages using serializeDataFrame single-allocation path. Result: - Messages below threshold: zero compression CPU overhead, sent uncompressed with correct frame header flag. - Messages above threshold: gzip compressed with correct frame header flag. Per gRPC spec, per-frame compression flag tells clients whether each frame is compressed. - Default threshold of 1024 bytes aligns with industry consensus (Netty, Ktor, OneUptime gRPC guide) as the break-even point where gzip CPU cost is justified by bandwidth savings on protobuf data. - JIT/GC optimized: zero per-request allocation on common path via pre-computed negotiation result and cached AdaptiveGzip singleton. Tests: - runtime / Test / test - 152 tests passed - plugin-tester-scala / Test / testOnly ErrorReportingSpec - passed - 30 new AdaptiveGzip tests covering codec behavior, boundary conditions, compressWithFlag, streaming frame header correctness, round-trip encode/decode, instance caching, and negotiate integration References: Refs #739
He-Pin
force-pushed
the
optimize/adaptive-compression
branch
from
6c18e67 to
b3c829a
Compare
Motivation: `eq` is reference equality in Scala — it only returns true when both operands are the exact same object. `mediaType.subType` is not guaranteed to return interned strings, so `eq` can silently return false for semantically equal values like "grpc+proto". Modification: Replace `eq` with `==` (value equality) in the `negotiate` fast path. Result: Content-type negotiation works correctly regardless of string interning.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The previous optimization (PR #739) always used Identity (no compression) for native gRPC responses to avoid CPU overhead on small messages. However, large messages benefit significantly from gzip compression. An adaptive approach that compresses only messages above a configurable threshold provides the best of both worlds.
Modification
grpc-encodingheader still advertises gzip.(ByteString, Boolean)to signal per-frame compression status accurately in the gRPC frame header (bit 0 of the 5-byte header). This fixes a correctness bug where streaming frames would incorrectly set the compression flag even for uncompressed data.pekko.grpc.server.compression-threshold(default 1024). Generated Scala handlers read config once at handler creation and pass the threshold asIntto avoid per-request config overhead.NativeIdentityAdaptiveGzipnegotiation result and cached defaultAdaptiveGzipinstance for zero per-request allocation on the common path.ProtobufFrameSerializer.serializedDataSize()for efficient size checking. Adaptive fast path inGrpcResponseHelpersfor small messages usingserializeDataFramesingle-allocation path.Result
Default threshold of 1024 bytes aligns with industry consensus (Netty, Ktor, OneUptime gRPC guide) as the break-even point where gzip CPU cost is justified by bandwidth savings on protobuf data.
Tests
runtime / Test / test- 152 tests passedplugin-tester-scala / Test / testOnly ErrorReportingSpec- passedcompressWithFlag, streaming frame header correctness, round-trip encode/decode, instance caching, negotiate integrationReferences
Refs #739