Skip to content

Measure replication over a real network, not loopback #86

Description

@jrosskopf

Successor to #68, which closed as answered.

Every performance measurement in #68 was taken with SAP and the erpl-rev server
on the same host. That single fact is what disqualified its largest lever:

  • Payload compression achieves a real, reproducible 47.9× ratio, yet cost
    16% end to end on loopback, because the payload term turned out to be
    dominated by server-side ingest rather than transfer. Over a link where 184 MB
    actually takes time, the sign of that result plausibly flips.
  • The in-HANA columnar option was ranked partly on removing payload bytes. That
    half of its case only exists if bytes cost something.

So the open question is not "which optimisation" but what the profile looks
like when the wire is real
.

What to measure

The same harness as #67/#68 — alternating arms, fresh DuckDB per sample, medians
of 5–7 rounds, load average recorded — with SAP and the server on separate hosts:

  1. Full-width 100k-row load, compression off vs on (gv_gzip). If on
    wins, compression should stop being default-off and the docs should say when
    to enable it.
  2. Where the wall clock goes: SAP-side select + transform, transfer, server
    ingest. Attack the SAP read side: it now owns the replication wall clock #68 has the loopback split (~36% serial chain) to compare against.
  3. Whether mv_depth starts mattering. On loopback the pipeline rarely blocked
    on it; with real latency it may become the throttle, which would make the
    nearly-free option 4 from Attack the SAP read side: it now owns the replication wall clock #68 worth doing after all.

Why it is not trivial

It needs two hosts with an SAP system reachable from the other, which the A4H
Docker trial on a laptop is not. Options in increasing order of fidelity: two
VMs on one hypervisor with a shaped link; a cloud SAP instance; a customer
system with permission to benchmark.

Definition of done

A table of medians per arm at a stated latency and bandwidth, and a decision on
whether gv_gzip should default on for networked deployments. A negative result
is a fine outcome and should be recorded the same way #68's was.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions