fix(providers): summarize per-provider errors instead of dumping err.stack#1441
Open
droplet-rl wants to merge 1 commit into
Open
fix(providers): summarize per-provider errors instead of dumping err.stack#1441droplet-rl wants to merge 1 commit into
droplet-rl wants to merge 1 commit into
Conversation
…stack
RetryProvider.send and the Solana QuorumFallback factory accumulate
per-provider failures via `err.stack || err.toString()`. For ethers
transport errors (SERVER_ERROR, etc.), `err.stack` and the stringified
form embed the failed RPC URL with its API key in cleartext, plus a
multi-line ethers stack trace. That accumulator then gets concatenated
into the aggregate "Not enough providers succeeded" error message and
into the `logQuorumMismatchOrFailureDetails` warn log — both reach
Slack/Datadog and leak the key.
Introduce `summarizeProviderError(err)` which produces a single-line,
log-safe summary. Preference order:
1. Parsed JSON-RPC error message (e.g. "execution reverted: ERC20:
burn amount exceeds balance"), drilling into `err.error` if
necessary — this is what callers actually want.
2. `err.reason` — ethers' short, URL-free reason string.
3. `err.message` for non-ethers errors (no `err.code` present).
4. `err.code` (e.g. "SERVER_ERROR", "NETWORK_ERROR") as a last resort.
`err.message` is deliberately skipped for ethers-shaped errors because
that's exactly where transport-layer errors splice in the failed URL.
The original error remains reachable via `error.cause` on the aggregate
thrown by `createSendErrorWithMessage`, so forensic detail isn't lost.
Wire it into both error-collection sites in `retryProvider.ts` and both
in `solana/quorumFallbackRpcFactory.ts`. No behavior change beyond the
shape of the recorded error string.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RetryProvider.send(and the SolanaQuorumFallbackSolanaRpcFactorycounterpart) accumulate per-provider failures via(err as any)?.stack || err?.toString(). For ethers transport errors (SERVER_ERROR,NETWORK_ERROR, etc.), botherr.stackand the stringified form embed the failed RPC URL with its API key in cleartext, alongside a multi-line ethers stack trace. That accumulator then gets concatenated into the aggregateNot enough providers succeeded. Errors: …error message and into thelogQuorumMismatchOrFailureDetailswarn log — both reach Slack/Datadog. So in addition to being unreadable, this is leaking provider API keys to log destinations.Real example surfaced from the Across relayer warning logs (URL/key redacted here):
After this change, the same failure produces:
Approach
New
summarizeProviderError(err)helper insrc/providers/utils.tsthat produces a single-line, log-safe summary. Preference order:parseJsonRpcError(drilling intoerr.errorif needed) — this is the actual revert reason that callers want.err.reason— ethers' short, URL-free reason string (processing response error,execution reverted, etc.).err.message, only for non-ethers errors (noerr.codepresent) — safe because there's no transport-layer URL to leak.err.code(SERVER_ERROR,NETWORK_ERROR, …) as a last resort.err.messageis deliberately skipped for ethers-shaped errors because that's where ethers'Logger.makeErrorsplices the failed URL into the message. The original error remains reachable viaerror.causeon the aggregate thrown bycreateSendErrorWithMessage, so any forensic detail isn't lost — it just stops being on the user-visible log path.Wired into both
errors.push([provider, …])sites inretryProvider.tsand both insolana/quorumFallbackRpcFactory.ts. No other behavior change:failImmediatestill callsparseJsonRpcErroron the raw error; the originalerris still thrown/cascaded as before; classification downstream (e.g. relayer'sMultiCallerClient#canIgnoreRevertReason) is.includes()based and continues to match the cleaner string.Test plan
summarizeProviderErrorintest/providers/utils.test.tscovering: null/undefined, string inputs, JSON-RPC body extraction (top-level + nestederr.error),.reasonfallback,.codefallback, non-ethers.message, and an explicit URL-leak guard.createSendErrorWithMessage+providers.test.tssuites still pass.prettier --checkandeslintclean on touched files.tsc -p tsconfig.build.json --noEmitclean.Context
Found while investigating noisy
MultiCallerClient#LogSimulationFailureswarnings in the Across relayer. A relayer-side regex sanitizer was considered first (across-protocol/relayer#3411, now closed) but fixing it once at the source — here — is cleaner and benefits every consumer (relayer, dataworker, finalizer, monitor) with a single version bump.🤖 Generated with Claude Code