The bytes say gzip. The header says nothing.
This interactive lab models an OTLP/HTTP proxy that mutates a compressed body
and its Content-Encoding metadata independently. The edge forwards every
request, but the collector chooses its decoder from a contract that no longer
describes the bytes.
The default scenario sends 240 trace batches per minute with 200 spans in each. A header-stripping proxy is rolled out to 55% of traffic:
| Signal | Edge view | Collector view |
|---|---|---|
| Batches forwarded / minute | 240 | — |
| Batches decoded / minute | — | 108 |
| Apparent delivery | 100% | — |
| Effective delivery | — | 45% |
| Spans rejected / minute | 0 | 26,400 |
Every result is computed by the compiled Vala model in
model/encoding_contract.vala. The browser
contains no duplicate simulation math.
OTLP/HTTP permits clients to gzip the request body. When they do, the stable
OTLP specification requires the request to include
Content-Encoding: gzip. A collector uses that metadata to select the
decompression step before decoding the OTLP Protobuf payload.
Two proxy transformations break the contract:
header stripped
Content-Encoding: (absent) bytes: 1f 8b 08 00…
collector selects Protobuf result: invalid wire type
header stale
Content-Encoding: gzip bytes: 0a 8f 9c 06…
collector selects gzip result: gzip magic missing
Neither case is a transient outage. The OTLP specification classifies
permanently undecodable input as bad data, for which the server returns
HTTP 400 Bad Request and the client must not retry.
Primary references:
The controls demonstrate two sound proxy contracts:
- Pass through — forward
Content-Encoding: gzipand the gzipped bytes without changing either. - Normalize — decompress the body and remove
Content-Encoding, leaving ordinary Protobuf bytes for the collector.
The invariant is simple: the header and body must be transformed together.
A proxy request counter reports perfect delivery in both broken modes. The lab emits the signals needed to reconcile transport hops:
lab.otlp.apparent_delivery{hop="proxy"}counts successfully forwarded requests.lab.otlp.effective_delivery{hop="collector"}counts decoded requests.lab.otlp.decode_failures{error.type=...}separates missing-header and stale-header failures.lab.otlp.rejected_spans{error.type=...}quantifies permanent loss.lab.otlp.wire_bytes{representation=...}shows when proxy decompression silently expands traffic.- a shared trace ID connects the forwarding proxy span to the collector decode
log, including
Content-Encoding, first bytes, and selected decoder.
Try the investigation queries in the UI:
metrics | where name in ("lab.otlp.apparent_delivery", "lab.otlp.effective_delivery") | chart value by hop
logs | where error.type in ("gzip_header_missing", "gzip_magic_missing") | summarize count() by proxy.version
traces | where http.route == "/v1/traces" | project trace_id, proxy.content_encoding, collector.decoder, otel.status_code
This is the power of correlated telemetry: two individually plausible views become an obvious contradiction when joined.
Install Vala 0.56, a C compiler, Python 3, and Node.js, then:
make check
make runOpen http://localhost:8080.
The live endpoints are:
GET /healthz
GET /api/simulate
GET /api/telemetry
Example repairs:
curl 'http://localhost:8080/api/simulate?mode=passthrough'
curl 'http://localhost:8080/api/simulate?mode=normalize'docker compose up --buildThe multi-stage image compiles the Vala model and runs the full test suite, then copies only the model, Python service, and static UI into an unprivileged runtime image.
.
├── model/encoding_contract.vala # header/body contract model
├── service/server.py # static site, API, correlated evidence
├── tests/ # model, service, and UI contracts
├── public/ # dependency-free packet workbench
├── telemetry/signals.md # evidence catalog
├── Dockerfile
└── compose.yaml
MIT