feat(observability): make the OTel tracing that is already wired up actually emit

[blarghmatey]

What are the relevant tickets?

Part of the OpenTelemetry APM coverage epic (tk-extend-otel-tracing-to-the-uninstrumented-apps-m-41729c). OVS is one of four in-house Django apps running in applications-production with zero tracing.

Description (What does it do?)

OVS has had mitol.observability.apps.ObservabilityConfig in INSTALLED_APPS since the observability library was adopted, and has produced exactly zero spans the whole time. Two halves were missing, and each one alone is silent:

1. No instrumentors were installed. configure_opentelemetry() discovers instrumentation through the opentelemetry_instrumentor entry point group. The venv had opentelemetry-api/sdk/exporter and nothing else, so the discovery loop found no entry points, instrumented nothing, and logged OpenTelemetry initialized successfully anyway.
2. No endpoint was readable. The library takes an endpoint from OTEL_EXPORTER_OTLP_ENDPOINT in the environment or from the OPENTELEMETRY_ENDPOINT Django setting. OVS declared neither, so configure_opentelemetry() returned at its early exit.

Neither half raises, warns, or shows up in a health check. A service with tracing "enabled" and no spans looks identical to a healthy one until somebody goes looking in Tempo and finds nothing.

psycopg2, not psycopg. opentelemetry-instrumentation-psycopg targets psycopg 3; this app is on psycopg2>=2.9.10,<3, so that package would install cleanly, expose an entry point and produce no database spans — the same silent-success shape as above. mit-learn and mitxonline use the psycopg variant because they are on psycopg 3; the difference is deliberate.

The library's postgres extra is also deliberately unused — it pins psycopg[c], and Django's postgresql backend prefers psycopg 3 whenever it is importable, so the extra would silently change this application's database driver. That is a migration, not a tracing change.

Why a Django setting rather than the standard env var?

OPENTELEMETRY_ENDPOINT carries the full /v1/traces path because the released library (2026.3.11) hands whatever it reads straight to OTLPSpanExporter(endpoint=...), which uses it verbatim. The spec-standard OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is not consulted by that version at all, and pointing OTEL_EXPORTER_OTLP_ENDPOINT at a base URL only starts working once mitodl/ol-django#551 ships. The Django setting is the one spelling that behaves identically before and after that change.

service.name needs no setting — the library reads OTEL_SERVICE_NAME from the environment directly.

Logging is unaffected: ObservabilityConfig was already installed, so structlog has been configuring this app's logging all along.

How can this be tested?

Tracing stays off everywhere the endpoint is unset, including local development, so the default path is a no-op.

• manage.py check with the endpoint set logs OpenTelemetry: OTLP exporter configured to http://localhost:4318/v1/traces.
• The entry point group now resolves to ['celery', 'django', 'psycopg2', 'redis', 'requests'] (previously empty).

To see spans end to end you need the ol-infrastructure change that sets the endpoint in QA — that is a separate PR and must land after this image deploys, since the env vars are inert without this.

Additional Context

The only new runtime cost is the instrumentation overhead per request plus a background BatchSpanProcessor export. Grafana Alloy tail-samples what actually reaches Tempo (all errors, everything over 5000ms, 15% of the rest).