edge-relay-gateway

A public HTTP endpoint for a private service the gateway is never allowed to connect to.

The usual way to expose a service that lives on a private network is to poke a hole towards it: a port forward, an inbound firewall rule, a NAT entry, a VPN tunnel. Every one of those is a standing invitation — the hole stays open whether or not anyone is using it, and it is the private network that carries the risk.

This gateway inverts the direction. It sits in the DMZ with a public HTTP port, and it holds no route into the private network at all. Instead the private service dials out to the gateway and keeps that connection open. Inbound requests are pushed down the connection the private side already opened. The security posture is what makes it worth building: the private network needs zero inbound rules, publishes no address, and can drop the connection at any time to take itself off the internet.

The trick that makes it work is turning a one-way push into request/response using correlation IDs.

About this project. An original reference implementation, written to demonstrate the architecture of production systems I've worked on — built with a team, and not an extract from any employer's codebase. No proprietary code, configuration, credentials or customer detail appears in it. It runs on JDK 17 and Maven with no external infrastructure, and every command below is shown with real captured output.

It is also one half of a pair: the private-side peer, with the auth and multi-tenant routing that sits behind it, is the companion project multitenant-relay-router. This one runs standalone too.

---

Architecture

flowchart LR
    client([HTTP client])

    subgraph dmz["DMZ — public"]
        http["RelayController<br/>/relay/**"]
        pending[("PendingRequests<br/>correlationId → future")]
        registry[("PeerRegistry<br/>peerId → connection")]
        rsocket["RSocket server<br/>:7000"]
    end

    subgraph private["Private network &mdash; no inbound access"]
        peer["Relay peer"]
        svc["Internal services"]
    end

    client -->|"1 . HTTP request"| http
    http -->|"2 . park future"| pending
    http -->|"3 . look up peer"| registry
    http -->|"4 . push envelope"| rsocket
    rsocket -->|"5 . over the peer's own connection"| peer
    peer --> svc
    peer -->|"6 . result + correlationId"| rsocket
    rsocket -->|"7 . complete future"| pending
    pending -->|"8 . HTTP response"| client

    peer -.->|"connection is always dialled outwards"| rsocket

    style private fill:#eef7ee,stroke:#5a8f5a
    style dmz fill:#eef2f8,stroke:#5a7a9f

Loading

Read the dotted line first: it is the only connection between the two zones, and the private side is always the one that opens it.

---

Core concepts

1. The peer registry is the only door

PeerRegistry maps a peer id to the connection that peer opened. The gateway cannot create an entry — only an inbound SETUP frame can. If a peer is absent, the honest answer to a request is 503, and there is no fallback path that would let the gateway reach in and connect.

Two lifecycle details carry real weight:

• Reconnect displaces. When a peer restarts, its new connection frequently arrives before the gateway notices the old one died. Keeping both would leave requests being written into a half-dead socket that will never answer, so register disposes the connection it replaces.
• Late closes must not evict. The displaced connection's close signal arrives moments later. unregister removes the peer only if the closing connection is still the registered one — without that identity check, the stale close would evict the healthy replacement and silently blackhole every subsequent request.

Both are covered by PeerRegistryTest. They are the kind of bug that only shows up under a real restart, which is exactly why they are worth pinning down in a test.

2. Correlation IDs turn a push into request/response

RSocket gives a duplex connection, not a request/response channel in the direction the gateway needs. So RelayController:

1. mints a correlation ID,
2. parks a CompletableFuture in PendingRequests under that ID,
3. pushes a RelayEnvelope down the peer's connection fire-and-forget, and
4. returns the future to Spring MVC.

The peer answers later on a different route (relay.result), carrying the ID it was asked about. PendingRequests.complete finds the parked future and the HTTP exchange resumes.

Returning a CompletableFuture rather than blocking matters more here than in a normal service. Relayed calls are slow by construction — two network hops plus whatever the private service does — so a gateway that burned a container thread per in-flight request would exhaust its thread pool long before it exhausted anything else.

3. Every failure mode has to be an answer

A relay can fail in a way a normal proxy cannot: the peer accepts the work and then never speaks again. Nothing downstream will ever close that loop.

Situation	Response	Why
Peer not connected	503	Fail fast; there is nothing to wait for
Peer never answers	504 after relay.request-timeout	The only thing preventing a permanently parked exchange
Send fails on the wire	502	Surfaced immediately rather than waiting out the timeout
Peer answers 404	404	The peer reports the downstream status; the edge does not flatten it

The timeout also frees the parked entry. Completing an exchange by any route calls PendingRequests.forget, which is what stops the map growing without bound under load.

4. The header allow-list

RelayProperties.forwardedHeaders names the inbound headers permitted to cross into the private network. Everything else — cookies, hop-by-hop headers, anything a caller invents — stops at the edge. On an internet-facing component an allow-list is the only defensible default; a deny-list is a list of the attacks you already thought of.

5. Binary responses pick their own route

File downloads and exports come back on relay.result.binary, with the correlation ID in frame metadata because there is nowhere to put it inside an opaque byte array.

The gateway does not need to be told in advance which shape a response will take. Both routes complete the same parked future, and the HTTP layer renders whatever it finds — RelayResult becomes a JSON response, byte[] becomes an octet-stream. This keeps the special case out of the request path entirely, where it would otherwise have to be driven by some header the client sets.

---

Run it

Requires JDK 17+ and Maven. No external infrastructure.

mvn spring-boot:run -Dspring-boot.run.profiles=mock-peer

The mock-peer profile starts a stand-in private peer that dials out to the gateway, so the whole loop is exercisable on one machine. It is a real client over a real socket, not a mock object — the interesting behaviour here lives in the connection lifecycle, and a stubbed registry would not exercise any of it.

Confirm the peer registered:

curl -s http://localhost:8080/gateway/peers

{"connectedPeers":["private-router"],"inFlightRequests":0}

Relay a GET, with one allow-listed header and one that is not:

curl -s -H 'Authorization: Bearer demo-token' -H 'X-Secret-Internal: must-not-cross' 'http://localhost:8080/relay/orders/42?page=1'

{"servedBy":"mock-peer","method":"GET","path":"/orders/42","correlationId":"8632fed4-d40f-4629-8058-a26dba359093","query":"page=1","forwardedHeaders":{"Authorization":"Bearer demo-token"}}

Note the /relay mount point stripped from the path, and X-Secret-Internal absent — it never left the DMZ.

Relay a POST with a body:

curl -s -X POST http://localhost:8080/relay/orders -H 'Content-Type: application/json' -d '{"item":"widget","quantity":3}'

{"servedBy":"mock-peer","method":"POST","path":"/orders","correlationId":"f7712372-d5d3-4536-9d2e-5e3e1c438a1d","forwardedHeaders":{},"receivedBody":{"item":"widget","quantity":3}}

A downstream 404 stays a 404:

curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8080/relay/missing/thing

404

A binary answer comes back as bytes:

curl -s -i http://localhost:8080/relay/reports/monthly/download | head -3

HTTP/1.1 200
Content-Type: application/octet-stream
Content-Length: 37

Ask for a peer that never connected:

curl -s -H 'X-Peer-Id: nope' http://localhost:8080/relay/orders/1

{"error":"Service Unavailable","detail":"Peer 'nope' is not connected","status":503}

Watch a request time out. The mock peer deliberately drops anything under /blackhole, so this hangs for relay.request-timeout (20s by default) and then answers:

curl -s http://localhost:8080/relay/blackhole/thing

{"error":"Gateway Timeout","detail":"The private peer did not answer within PT20S","status":504}

Tests

mvn test

19 tests. PeerRegistryTest and PendingRequestsTest cover the state machines directly; RelayRoundTripTest and RelayFailureModeTest drive the whole loop over a real socket.

Pairing with the real peer

Run this gateway without the mock-peer profile and start multitenant-relay-router alongside it. The router dials out to localhost:7000, registers as private-router, and serves traffic arriving at http://localhost:8080/relay/**.

---

Design notes

Why RSocket rather than a WebSocket or gRPC stream? All three would carry the traffic. RSocket is the best fit because it is symmetric by design: once connected, either end can be the requester. The peer's callback is an ordinary request from the peer to the gateway rather than a message squeezed back through a channel built for the other direction. Application-level keep-alive and backpressure come with it.

Why a typed envelope? The request is packaged as a RelayEnvelope record — method, path, query, headers, body as explicit fields. It is tempting to concatenate everything into a query string and parse it on the far side; that works right up until a value contains an &, and then it fails in a way that is genuinely painful to diagnose. Typed fields cost nothing and remove the whole class of problem.

Why is the peer id a record rather than a string? PeerIdentity keeps the SETUP payload unambiguous — no question of whether the id arrives JSON-quoted or raw — and leaves room to add a protocol version or capability flags without breaking existing peers.

Trade-off: state is in memory. PendingRequests and PeerRegistry are per-instance. That is correct for what they hold — a parked HTTP exchange belongs to the node serving it, and a socket belongs to the node it terminates on — but it means gateway instances are not interchangeable mid-request. See below.

---

Simplified from production

What a real deployment of this pattern adds:

• Horizontal scale. With several gateway instances, a peer connects to one of them, so a request arriving at a different instance has no local peer. Real systems solve this by having peers connect to every instance, or by routing requests to the instance holding the connection via a shared registry. Here, one instance.
• Authentication of peers. The SETUP frame should carry a credential the gateway verifies before registering; here any client that reaches the port may claim any peer id.
• TLS on both hops, with the RSocket connection mutually authenticated.
• Backpressure and admission control. PendingRequests is unbounded — a peer that stops answering while traffic continues will grow it until the timeouts catch up. Production wants a cap and a fast rejection above it.
• Per-route timeouts. One global timeout is a compromise; a report export and a status check do not deserve the same budget.
• Observability. In-flight count, per-peer latency histograms and reconnect counters as real metrics rather than a status endpoint.

The concepts above are the ones worth reading; the rest is deployment.

---

Part of a set

Four standalone projects, each isolating one problem from systems I've worked on in production. They live in separate repositories and each runs on its own.

Project	Language	The problem
edge-relay-gateway — you are here	Java	Serving public HTTP traffic for a private service the gateway is not allowed to connect to
multitenant-relay-router	Java	Passwordless auth, per-tenant credentials, and one request answered by many backends at once
event-correlation-engine	Java	Collapsing a high-volume event stream into a short list of things a human can work on
analytics-control-plane	Kotlin	Provisioning dependent artifacts into a system with no transactions — and undoing it cleanly

---

Licence

MIT — see LICENSE.