Epic: Improve RPC query path performance #1055

Summary

Optimize RPC's query path for performance reasons.

I. XDR Views

First, replace RPC's decoding-heavy backend with views to minimize the amount of needless decoding / marshaling.

Then, update the relevant interfaces to have / implement view-idiomatic methods.

Cap this section off by creating a semi-temporary differential test suite to exhaustively demonstrate parity between the old and new final endpoint responses.

II. Cacheing and memoization

Several endpoints are constantly recomputing data to serve identical responses. In some cases (e.g. getLatestLedger), the entire final serialized response can be saved, amortizing the cost of computing the response across all responses in a given window. In others, key components of the response can be saved as memos per-sequence.

Additionally, investigate RPCv1's on-ledger-ingest cache lock holding behavior. If it's holding the lock more greedily than is necessary, correct this.

III. Raw-response passthrough

We use creachadair/jrpc2 as our jsonrpc library. As is common with all mainstream jsonrpc libraries, this library forces some marshaling/decoding where we don't need it.

Thus, here, either by forking https://github.com/creachadair/jrpc2 or implementing the dispatcher in stellar-rpc, rework the handler/server to never marshal/decode beyond what's necessary.

• if it ends up being low-hanging fruit afterwards, open a PR in https://github.com/creachadair/jrpc2 to allow the maintainers to decide if they'd like this in their upstream

IV. Benchmarking

Finish off this super-epic by looking back on your progress!

Using RPCv1's GHA performance evaluation tool, create a before/after comparison of the results of your optimizations.

Acceptance criteria

• endpoint backends are migrated to XDR views, interfaces updated, differential suite in + passing
• endpoint backends take advantage of cacheing/memoization opportunities
• as much as is reasonable, jsonrpc is made maximally efficient
• results are benchmarked and demonstrate at least a reasonable speedup across the board

This epic is being made retroactively after a considerable number of optimizations have been merged or are currently in progress.