telltale

Find gaps between an OpenAPI contract and the telemetry a service actually ships.

telltale compares documented routes and error responses with Prometheus alert rules, OpenTelemetry Collector configuration, and Grafana dashboards. It reports coverage per route, signal, and documented failure mode.

Install and run

uvx foretop-telltale check .

An OpenAPI document is autodetected at the repository root. Pass --spec path/to/openapi.yaml when it lives elsewhere. Run uvx foretop-telltale check --help for every option.

What it checks

• OpenAPI 3 and Swagger documents named openapi or swagger with YAML or JSON extensions.
• Prometheus alerting rules associated with documented routes and status codes.
• OpenTelemetry Collector pipelines and processors that expose route-level signals.
• Grafana dashboard queries that provide route and failure-mode visibility.

Ambiguous matches stay ambiguous; absent or unreadable evidence is never reported as coverage.

Output and CI gating

--format supports table, markdown, json, sarif, annotations, and triage-json. The default scan is report-only. Add --base origin/main to exit 1 only when coverage that existed at the merge base has regressed; Telltale does not fail on an absolute coverage target. Exit code 2 means the check itself failed.

GitHub Action

- uses: foretop-dev/telltale@v0.2.0
  with:
    base: ${{ github.event.pull_request.base.sha }}

The Action emits inline annotations and can maintain one summary comment on pull requests. Grant pull-requests: write when comments are enabled and make the base revision available to the checkout when using the regression gate. Set base to an empty string for report-only use.

Privacy

Contract and telemetry analysis run locally. A normal check neither uploads repository content nor requires an account. --report and --gate are explicit hosted-mode options that require FORETOP_TOKEN; the CLI prints the metadata payload before reporting it. Source contents are never part of that payload.

Current limitations

• Coverage is inferred from committed configuration; it does not prove that telemetry arrived successfully at runtime.
• OpenAPI references are resolved within the local document; remote references are not fetched.
• Matching is deliberately conservative, so uncertain evidence is reported as ambiguous.

License

Apache-2.0. See LICENSE.

Questions and bug reports are welcome in GitHub Issues.