perf(policy-engine): drop per-request deep copy of policy params#2617
Open
renuka-fernando wants to merge 1 commit into
Open
perf(policy-engine): drop per-request deep copy of policy params#2617renuka-fernando wants to merge 1 commit into
renuka-fernando wants to merge 1 commit into
Conversation
deepCopyParams did a full JSON marshal/unmarshal round-trip on spec.Parameters.Raw at all six execution phases on every request. Params are an immutable snapshot published once at chain-build time and shared read-only across concurrent requests, so the copy was pure overhead — it accounted for ~30% of the ext_proc CPU tree. - Pass spec.Parameters.Raw directly to each policy On* method - Remove deepCopyParams and the now-unused encoding/json import - Drop the three orphaned TestDeepCopyParams_* tests Verified with go test -race on the executor and kernel packages (kernel covers the production chain-build/publish path). An A/B benchmark with a realistic nested param map showed ~30-40x faster execution and ~14x fewer allocations per policy.
renuka-fernando
requested review from
AnuGayan,
Arshardh,
CrowleyRajapakse,
HeshanSudarshana,
HiranyaKavishani,
Krishanx92,
PasanT9,
RakhithaRR,
Tharsanan1,
Thushani-Jayasekera,
VirajSalaka,
ashera96,
chamilaadhi,
dushaniw,
hisanhunais,
lasanthaS,
malinthaprasan,
pubudu538,
senthuran16,
tgtshanika,
tharikaGitHub,
tharindu1st and
thivindu
as code owners
July 13, 2026 03:42
Contributor
📝 WalkthroughWalkthroughPolicy execution no longer deep-copies parameters for request, response, or streaming callbacks. Each callback receives ChangesPolicy parameter sharing
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain modules listed in go.work or their selected dependencies" Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@gateway/gateway-runtime/policy-engine/internal/executor/chain.go`:
- Around line 143-145: Update the policy dispatch around OnRequestHeaders so
policies cannot mutate the shared spec.Parameters.Raw map: replace the raw-map
argument with the established read-only accessor, or create a defensive
per-request clone before invoking the callback. Preserve the existing parameter
values while ensuring concurrent requests never share a mutable map instance.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 1498b86e-8d1b-4be6-b24b-db619f6e8ef6
📒 Files selected for processing (2)
gateway/gateway-runtime/policy-engine/internal/executor/chain.gogateway/gateway-runtime/policy-engine/internal/executor/chain_test.go
💤 Files with no reviewable changes (1)
- gateway/gateway-runtime/policy-engine/internal/executor/chain_test.go
Comment on lines
+143
to
+145
| // spec.Parameters.Raw is an immutable snapshot published at chain-build time and | ||
| // shared read-only across concurrent requests; policies must not mutate it. | ||
| action := headerPol.OnRequestHeaders(ctx, reqCtx, spec.Parameters.Raw) |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Confirm the concrete type of Parameters.Raw and look for any existing immutability guard
rg -nP -g '*.go' '\btype\s+Parameters\b' -A 10
rg -nP -g '*.go' 'Raw\s+map\[string\]interface\{\}'
# Search for any policy implementation (outside tests) that writes into a params-like map argument
rg -nP -g '*.go' -g '!*_test.go' '\b(params|parameters)\s*\[[^\]]+\]\s*='Repository: wso2/api-platform
Length of output: 155
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map the relevant files before reading them
git ls-files 'gateway/gateway-runtime/policy-engine/internal/executor/chain.go' \
'gateway/gateway-runtime/policy-engine/internal/**' \
| sed -n '1,200p'
# Inspect the chain implementation around the cited lines
wc -l gateway/gateway-runtime/policy-engine/internal/executor/chain.go
cat -n gateway/gateway-runtime/policy-engine/internal/executor/chain.go | sed -n '120,180p'
# Find the Parameters type and the Raw field definition
rg -n --hidden --glob '*.go' '\btype\s+Parameters\b|Raw\s+map\[string\]interface\{\}|type\s+.*Parameters' gateway/gateway-runtime/policy-engine/internal
# Find policy callback signatures to see whether the map is exposed directly everywhere
rg -n --hidden --glob '*.go' 'OnRequestHeaders|OnRequestBody|OnResponseHeaders|OnResponseBody|OnRequestBodyChunk|OnResponseBodyChunk' gateway/gateway-runtime/policy-engine/internal
# Look for any code in the policy engine that mutates the shared map or nested values
rg -n --hidden --glob '*.go' --glob '!**/*_test.go' 'spec\.Parameters\.Raw|Parameters\.Raw|\[[^]]+\]\s*=|append\(' gateway/gateway-runtime/policy-engine/internalRepository: wso2/api-platform
Length of output: 11339
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the Parameters type and the Raw field
rg -n --hidden --glob '*.go' 'type\s+Parameters\b|Raw\s+map\[string\]interface\{\}' gateway/gateway-runtime/policy-engine/internal
# Read the file(s) that define Parameters in full enough to inspect the surrounding context
for f in $(rg -l --hidden --glob '*.go' 'type\s+Parameters\b|Raw\s+map\[string\]interface\{\}' gateway/gateway-runtime/policy-engine/internal); do
echo "===== $f ====="
wc -l "$f"
cat -n "$f" | sed -n '1,260p'
done
# Inspect the policy callback interfaces and any wrappers around the params argument
rg -n --hidden --glob '*.go' 'type\s+.*interface\b|OnRequestHeaders|OnRequestBody|OnResponseHeaders|OnResponseBody|OnRequestBodyChunk|OnResponseBodyChunk' gateway/gateway-runtime/policy-engine/internalRepository: wso2/api-platform
Length of output: 155
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find where spec.Parameters is defined and how Raw is typed
rg -n --hidden --glob '*.go' 'spec\.Parameters|Parameters\s+struct|Raw\s+map\[string\]interface\{\}|map\[string\]any|map\[string\]interface\{\}' gateway/gateway-runtime/policy-engine
# Find policy callback interface definitions and implementations
rg -n --hidden --glob '*.go' 'OnRequestHeaders\(|OnRequestBody\(|OnResponseHeaders\(|OnResponseBody\(|OnRequestBodyChunk\(|OnResponseBodyChunk\(' gateway/gateway-runtime/policy-engine
# Inspect likely definition files if found
for f in \
$(rg -l --hidden --glob '*.go' 'spec\.Parameters|Raw\s+map\[string\]interface\{\}|map\[string\]any' gateway/gateway-runtime/policy-engine | head -n 20); do
echo "===== $f ====="
wc -l "$f"
cat -n "$f" | sed -n '1,220p'
doneRepository: wso2/api-platform
Length of output: 50374
Shared spec.Parameters.Raw is still a mutable map[string]interface{}
chain.go passes the same map instance into every policy callback, but the policy API exposes it as a raw map with no defensive copy or read-only wrapper. Any mutation can leak state across requests or trigger concurrent map writes in the gateway. Replace it with a read-only accessor or clone/freeze params before dispatch.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@gateway/gateway-runtime/policy-engine/internal/executor/chain.go` around
lines 143 - 145, Update the policy dispatch around OnRequestHeaders so policies
cannot mutate the shared spec.Parameters.Raw map: replace the raw-map argument
with the established read-only accessor, or create a defensive per-request clone
before invoking the callback. Preserve the existing parameter values while
ensuring concurrent requests never share a mutable map instance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
The policy chain executor deep-copied each policy's parameters on every request, at all six execution phases (request/response headers, request/response body, streaming request/response body). The copy was a full JSON marshal/unmarshal round-trip over
spec.Parameters.Raw. Profiling attributed roughly 30% of the ext_proc CPU tree to this deep copy, along with the corresponding allocation churn.Policy parameters are an immutable snapshot: they are published once at chain-build time (on the xDS push / config load) and shared read-only across concurrent request handlers. Config updates build entirely new chain objects and swap them in atomically, so a live chain's parameters are never mutated. The per-request copy was therefore pure overhead.
Goals
Eliminate the redundant per-request parameter copy and the allocations it produced, without changing observable behavior.
Approach
spec.Parameters.Rawdirectly to each policyOn*method instead of a per-request deep copy.deepCopyParamshelper and the now-unusedencoding/jsonimport.TestDeepCopyParams_*unit tests.User stories
N/A
Documentation
N/A — internal performance optimization with no user-facing or API changes.
Automation tests
Security checks
Samples
N/A
Related PRs
N/A
Test environment
Go (policy-engine module); linux/amd64 and darwin/arm64.
Related Issues
Related #2615
Checklist
Remarks
An A/B benchmark with a realistic nested parameter map (a combined JWT-validation + rate-limit config) through a body-buffering policy showed roughly 30-40x faster chain execution and ~14x fewer allocations per policy once the copy was removed. The repo's existing executor benchmarks use empty parameter maps, where the old code short-circuited before the JSON round-trip, so they did not surface this cost.
The safety of sharing the map now rests on the convention that policies treat parameters as read-only. A hard guardrail (e.g. a read-only accessor type in the SDK) would be a separate, breaking follow-up and is out of scope here.