multitenant-relay-router

The private-network service behind an edge relay: passwordless login, per-tenant credentials, and one request answered by many backends at once.

A mobile client wants a single view across several independently deployed backends — separate regions, separate customers, separate identity — and it must reach them without any of those backends being exposed to the internet. That produces three problems at once, and this project is about how they fit together:

1. Reachability. The router lives on a private network and dials out to a DMZ gateway, which is how inbound traffic reaches it without a single inbound firewall rule.
2. Identity. Users authenticate with a one-time code sent to their address — no password to store, leak or reset — and receive a token that identifies them to the router.
3. Authority. That token is deliberately never forwarded downstream. Backends authenticate the router's own service account and are told who the user is separately.

About this project. An original reference implementation, written to demonstrate the architecture of production systems I've worked on — built with a team, and not an extract from any employer's codebase. No proprietary code, configuration, credentials or customer detail appears in it. It runs on JDK 17 and Maven with no external infrastructure, and every command below is shown with real captured output.

It is also one half of a pair: the public edge it dials out to is the companion project edge-relay-gateway. This one runs standalone too.

---

Architecture

flowchart TB
    client([Mobile client])
    gw["edge-relay-gateway<br/><i>DMZ</i>"]

    subgraph router["multitenant-relay-router &mdash; private network"]
        conn["RelayConnectionManager<br/>dials out, reconnects"]
        rr{{"RequestRouter"}}
        otp["OtpService<br/>cooldown · expiry · single use"]
        probe["TenantAccessProbe<br/>parallel · cached"]
        tenantCall["TenantCall<br/>service-account token"]
    end

    alpha[("tenant alpha")]
    beta[("tenant beta")]
    gamma[("tenant gamma")]

    client --> gw
    conn -.->|"outbound connection"| gw
    gw --> conn
    conn --> rr
    rr -->|"login routes"| otp
    rr -->|"who may this user see?"| probe
    rr -->|"one or many"| tenantCall
    tenantCall --> alpha
    tenantCall --> beta
    tenantCall --> gamma

    style router fill:#eef7ee,stroke:#5a8f5a

Loading

---

Core concepts

1. The order of the checks is the design

RequestRouter.route is a sequence of cheap checks, each of which can only reject. The expensive work — reaching a tenant — happens strictly after a request has earned it.

Step	Question	On failure
1	Is this a login route?	— (they must work without a token)
2	Is the caller authenticated?	401, before any tenant is touched
3	Which tenants were requested?	400 if none named
4	Which of those may this user see?	403 if none survive
5	One tenant, or several?	proxy vs. fan-out

Step 4 is the one worth dwelling on. The routing key is a request, not an authorisation. A caller naming a tenant they cannot see has it dropped from the set — the request proceeds with what remains, and only fails if nothing does. That is why bob, who can see one tenant, gets a clean single-tenant answer when he asks for three, rather than an error.

2. The token boundary

This is the security decision at the centre of the project, and it is easy to get subtly wrong.

The router issues a token that says "this is alice". It would be very convenient to forward that token to each backend. It would also be wrong: the token is signed with the router's key, for the router's audience. A backend cannot meaningfully validate it, and a backend that accepts it anyway has outsourced its authentication to whoever can reach it.

So TenantCall splits the two things a downstream call needs to know:

• Who is calling? The router — proven with a per-tenant service-account token the backend can actually verify.
• On whose behalf? The end user — asserted in an X-User-Email header.

The trade-off is explicit: backends trust the router completely about user identity. That is only safe because the outbound request is built from scratch, with no inbound header copied onto it, so X-User-Email can only ever hold a value taken from a verified token. A caller cannot inject their own. RoutingAndAggregationTest.doesNotForwardTheCallersToken pins both halves down.

3. Passwordless login, and why each step is there

OtpService is short, and every line of it is doing security work:

• Cooldown — one code per address per window. Without it the login endpoint is a free mail cannon pointed at any address an attacker names, which is an abuse problem for the owner of that inbox as much as for this service.
• One live code per address — requesting a new code retires the old one, so the number of simultaneously-guessable codes stays at one instead of growing with every retry.
• Expiry — a code is a short-lived bearer credential.
• Single use — verification consumes it. A one-time code that survives its first use is just a short password.
• Identical failures — a wrong code and an expired code return the same message and status. Distinguishing them tells an attacker which addresses have a code outstanding.
• Normalised addresses — otherwise changing the case of an address bypasses the cooldown.
• SecureRandom — a code is a credential, and java.util.Random is seeded from the clock; a couple of observed outputs are enough to reconstruct its sequence.

Tokens are also recorded server-side rather than trusted purely as self-contained JWTs. That is what makes revocation possible: deleting the row ends the session now, instead of waiting out the token's own expiry.

4. Discovering access by asking everyone, in parallel

There is no central registry of who-can-see-what — each tenant is authoritative about its own users. So TenantAccessProbe asks all of them at once, with a bounded concurrency so a router with many tenants does not open a connection storm on every cold read.

Two decisions:

• A failing tenant means "no", not "error". If a tenant is unreachable, the probe reports no access there rather than failing outright. The alternative lets one sick tenant lock every user out of every other tenant — turning a partial outage into a total one.
• The cache TTL is short on purpose. This caches an authorisation decision, so a stale entry means revoked access still works. The TTL is exactly the width of that window, which is why it is minutes rather than hours, and why invalidate exists for anything that revokes access.

5. Fan-out that tolerates partial failure

When several tenants survive step 4, RequestRouter.fanOut calls them concurrently and merges the answers keyed by tenant. A tenant returning 502 contributes its failure as an entry rather than failing the batch — one broken backend must not cost the caller the answers from the healthy ones.

The aggregate reports 207 Multi-Status whenever the outcomes are not unanimous, so a client can tell "all fine" from "some of this is missing" without inspecting every entry.

6. A connection that repairs itself

RelayConnectionManager owns the single outbound connection and treats disconnection as normal rather than exceptional. Exponential backoff on connect, close detection, automatic reconnect, and one entry point (requester()) that callers use without knowing whether a reconnect is underway.

The failure it works hardest to avoid is a dead socket nobody noticed: the router looks healthy, the gateway looks healthy, and no traffic flows. Hence explicit onClose handling rather than waiting for the next request to discover the problem, and a compare-and-set guard so two concurrent callers cannot open two connections — the gateway would treat the second as a reconnect and dispose one of them.

---

Run it

Requires JDK 17+ and Maven. No database, no mail server, no identity provider.

Three stand-in tenants are seeded automatically, deliberately differing:

Tenant	Visible to	Behaviour
alpha	alice, bob	healthy
beta	alice	healthy
gamma	alice	access works, every data call fails

gamma exists so partial failure is something you can see rather than a paragraph in a README.

Standalone

mvn spring-boot:run -Dspring-boot.run.arguments=--relay.gateway.enabled=false

Drive it on http://localhost:9000/local/**. This builds the same envelope the relay would deliver and calls the same router, so every authentication check on the real path is on this path too.

Paired with the gateway

Start edge-relay-gateway (without its mock-peer profile), then:

mvn spring-boot:run

The router dials out to localhost:7000 and registers as private-router. Everything below is real output from that paired setup, driven through the gateway on port 8080.

The router is connected:

curl -s http://localhost:8080/gateway/peers

{"inFlightRequests":0,"connectedPeers":["private-router"]}

1 — Request a code. It is written to the router's log by the bundled LoggingOtpDelivery, which stands in for the SMTP sender:

curl -s -X POST http://localhost:8080/relay/auth/otp -H 'Content-Type: application/json' -d '{"email":"alice@example.test"}'

{"message":"A code has been sent to alice@example.test","accepted":true,"statusCode":200,"retryAfterSeconds":60}

┌──────────────────────────────────────────────┐
│  One-time code for alice@example.test
│  512390
└──────────────────────────────────────────────┘

2 — Without a token, nothing reaches a tenant:

curl -s -H 'X-Routing-Key: alpha' http://localhost:8080/relay/alerts

{"error":"A valid access token is required"}

3 — Verify the code to get a token:

curl -s -X POST http://localhost:8080/relay/auth/verify -H 'Content-Type: application/json' \
  -d '{"email":"alice@example.test","code":"512390"}'

4 — Which tenants can she see? Answered by probing all three in parallel:

curl -s -H "Authorization: Bearer $TOKEN" http://localhost:8080/relay/access/tenants

{"user":"alice@example.test","tenants":["alpha","beta","gamma"]}

5 — A single tenant is proxied through. Note authenticatedAs: the backend saw the router's service-account token, not alice's:

curl -s -H "Authorization: Bearer $TOKEN" -H 'X-Routing-Key: alpha' http://localhost:8080/relay/alerts

{"servedFor":"alice@example.test",
 "alerts":[{"severity":"high","summary":"Disk nearly full","id":"alpha-1"},
           {"severity":"low","summary":"Certificate expires soon","id":"alpha-2"}],
 "authenticatedAs":"Bearer svc-router-svc-alpha-80f7986f-3f8…",
 "tenant":"alpha"}

6 — Fan out across all three. gamma is broken; the other two still answer, and the status is 207 rather than a misleading 200:

curl -s -w '\nHTTP %{http_code}\n' -H "Authorization: Bearer $TOKEN" -H 'X-Routing-Key: alpha,beta,gamma' \
  http://localhost:8080/relay/alerts

{"alpha": {"tenant":"alpha","statusCode":200,"body":{ ...two alerts... }},
 "beta":  {"tenant":"beta","statusCode":200,"body":{ ...two alerts... }},
 "gamma": {"tenant":"gamma","statusCode":500,"body":{"error":"backend unavailable"}}}
HTTP 207

7 — Bob asks for all three and gets the one he is allowed. The other two are dropped, not refused:

curl -s -H "Authorization: Bearer $BOB_TOKEN" -H 'X-Routing-Key: alpha,beta,gamma' http://localhost:8080/relay/alerts

Tests

mvn test

19 tests. OtpServiceTest steps an injected Clock rather than sleeping — a cooldown test that waited out a real 60 seconds would never be run often enough to catch a regression. RoutingAndAggregationTest drives every routing decision end to end against the stand-in tenants.

---

Design notes

Tenants are rows, not config. Onboarding a tenant happens far more often than a release, and a config file makes every new tenant a redeploy. It also keeps per-tenant credentials out of anything that could end up in version control.

Ports where the outside world starts. OtpDelivery and TenantTokenProvider are interfaces with runnable stand-ins. That is what lets the whole system run with no mail server and no identity provider, and it marks precisely where a real SMTP sender and a real client-credentials exchange attach. Neither substitution changes a line of routing logic.

Reactive where it earns its keep. The fan-out and the access probe are genuinely concurrent, so they are written reactively. Blocking work — token acquisition, database reads — is pushed to boundedElastic rather than run on the event loop. The alternative, a hand-managed thread pool per concern, is more code doing the same thing less clearly.

Time is a dependency. Clock is injected rather than called statically, because cooldowns and expiry windows are the substance of the login flow and testing them against Instant.now() means either sleeping through real seconds or not testing them at all.

---

Simplified from production

• Token minting is a stub. CachingTenantTokenProvider returns an opaque string instead of performing an OAuth2 client-credentials exchange. The caching, which is the part that keeps an identity provider from becoming the busiest component in the system, is real.
• No per-tenant TLS. Real tenants need their own trust material and client certificates, configured per WebClient in TenantClients.
• Secrets belong in a secret manager. Tenant holds a service-account id; the matching secret should be fetched at call time, never stored alongside.
• Single instance. Several routers behind one gateway need shared session storage and a shared access cache; here both are in-process.
• LoggingOtpDelivery writes a live credential to the log. It exists so the flow is demonstrable offline. Do not deploy it.
• No rate limiting beyond the OTP cooldown. A real edge wants per-caller limits on everything.
• Observability. Fan-out latency per tenant, probe cache hit rate and reconnect counts should be metrics rather than log lines.

---

Part of a set

Four standalone projects, each isolating one problem from systems I've worked on in production. They live in separate repositories and each runs on its own.

Project	Language	The problem
edge-relay-gateway	Java	Serving public HTTP traffic for a private service the gateway is not allowed to connect to
multitenant-relay-router — you are here	Java	Passwordless auth, per-tenant credentials, and one request answered by many backends at once
event-correlation-engine	Java	Collapsing a high-volume event stream into a short list of things a human can work on
analytics-control-plane	Kotlin	Provisioning dependent artifacts into a system with no transactions — and undoing it cleanly

---

Licence

MIT — see LICENSE.