Skip to content

Repository files navigation

Content-Encoding Mismatch Lab

The bytes say gzip. The header says nothing.

This interactive lab models an OTLP/HTTP proxy that mutates a compressed body and its Content-Encoding metadata independently. The edge forwards every request, but the collector chooses its decoder from a contract that no longer describes the bytes.

The default scenario sends 240 trace batches per minute with 200 spans in each. A header-stripping proxy is rolled out to 55% of traffic:

Signal Edge view Collector view
Batches forwarded / minute 240
Batches decoded / minute 108
Apparent delivery 100%
Effective delivery 45%
Spans rejected / minute 0 26,400

Every result is computed by the compiled Vala model in model/encoding_contract.vala. The browser contains no duplicate simulation math.

Why this happens

OTLP/HTTP permits clients to gzip the request body. When they do, the stable OTLP specification requires the request to include Content-Encoding: gzip. A collector uses that metadata to select the decompression step before decoding the OTLP Protobuf payload.

Two proxy transformations break the contract:

header stripped
Content-Encoding: (absent)   bytes: 1f 8b 08 00…
collector selects Protobuf   result: invalid wire type

header stale
Content-Encoding: gzip       bytes: 0a 8f 9c 06…
collector selects gzip       result: gzip magic missing

Neither case is a transient outage. The OTLP specification classifies permanently undecodable input as bad data, for which the server returns HTTP 400 Bad Request and the client must not retry.

Primary references:

The repairs

The controls demonstrate two sound proxy contracts:

  1. Pass through — forward Content-Encoding: gzip and the gzipped bytes without changing either.
  2. Normalize — decompress the body and remove Content-Encoding, leaving ordinary Protobuf bytes for the collector.

The invariant is simple: the header and body must be transformed together.

What telemetry.sh reveals

A proxy request counter reports perfect delivery in both broken modes. The lab emits the signals needed to reconcile transport hops:

  • lab.otlp.apparent_delivery{hop="proxy"} counts successfully forwarded requests.
  • lab.otlp.effective_delivery{hop="collector"} counts decoded requests.
  • lab.otlp.decode_failures{error.type=...} separates missing-header and stale-header failures.
  • lab.otlp.rejected_spans{error.type=...} quantifies permanent loss.
  • lab.otlp.wire_bytes{representation=...} shows when proxy decompression silently expands traffic.
  • a shared trace ID connects the forwarding proxy span to the collector decode log, including Content-Encoding, first bytes, and selected decoder.

Try the investigation queries in the UI:

metrics | where name in ("lab.otlp.apparent_delivery", "lab.otlp.effective_delivery") | chart value by hop
logs | where error.type in ("gzip_header_missing", "gzip_magic_missing") | summarize count() by proxy.version
traces | where http.route == "/v1/traces" | project trace_id, proxy.content_encoding, collector.decoder, otel.status_code

This is the power of correlated telemetry: two individually plausible views become an obvious contradiction when joined.

Run locally

Install Vala 0.56, a C compiler, Python 3, and Node.js, then:

make check
make run

Open http://localhost:8080.

The live endpoints are:

GET /healthz
GET /api/simulate
GET /api/telemetry

Example repairs:

curl 'http://localhost:8080/api/simulate?mode=passthrough'
curl 'http://localhost:8080/api/simulate?mode=normalize'

Run with Docker

docker compose up --build

The multi-stage image compiles the Vala model and runs the full test suite, then copies only the model, Python service, and static UI into an unprivileged runtime image.

Repository layout

.
├── model/encoding_contract.vala  # header/body contract model
├── service/server.py             # static site, API, correlated evidence
├── tests/                        # model, service, and UI contracts
├── public/                       # dependency-free packet workbench
├── telemetry/signals.md          # evidence catalog
├── Dockerfile
└── compose.yaml

License

MIT

About

Interactive Vala lab showing how OTLP gzip bytes and Content-Encoding metadata drift apart across proxies.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages