chore: upgrade OpenSearch + Dashboards to opensearchstaging 3.8.0

[kylehounslow]

What

Move OpenSearch and OpenSearch Dashboards from opensearchproject 3.7.0 to opensearchstaging 3.8.0 in docker-compose and the Helm chart. Add two Dashboards explore flags.

Changes

• .env: OPENSEARCH_DOCKER_REPO=opensearchstaging; OPENSEARCH_VERSION and OPENSEARCH_DASHBOARDS_VERSION → 3.8.0
• docker-compose/opensearch-dashboards/opensearch_dashboards.template.yml: add explore.logsQueryBuilder.enabled and explore.logsDrilldown.enabled
• charts/observability-stack/values.yaml: OS/OSD image repo and tag → staging 3.8.0; same two explore flags in the Dashboards config block
• charts/observability-stack/Chart.yaml: appVersion → 3.8.0, chart version → 0.4.1

opensearchDashboards.enableIconSideNav: true was already set in both, so no change there.

Validation

• Compose: both services healthy on 3.8.0; / reports 3.8.0, OSD /api/status green, plugin:explore@3.8.0 green, no config-key rejections
• Helm: helm lint clean, helm template renders the staging 3.8.0 images, 73/73 helm unittest pass

Screenshots from the running compose stack on 3.8.0:

Icon side nav (enableIconSideNav):

Logs query builder (explore.logsQueryBuilder.enabled) — visual Where / Aggregation / Group by / Sort:

Logs drilldown app (explore.logsDrilldown.enabled) — per-index overview with sparklines and log-level breakdowns:

Unified alerting — OpenSearch and Prometheus/Cortex rules in one view:

Active alert firing end to end:

Additional Notes (from Claude)

• These are opensearchstaging pre-release images, not GA opensearchproject. This changes the default compose/chart images to a pre-release build; flag if a GA-default with opt-in staging is preferred.
• Local test reused a prior 3.7.0 data volume; a clean-volume boot is untested.
• otel-demo's bundled OPENSEARCH_IMAGE (3.4.0) is unchanged.
• Recreating the OpenSearch container leaves a running data-prepper bound to the old one (collector drops spans with connection refused on 21890); a docker compose restart data-prepper reconnects it. A clean compose up orders data-prepper after OpenSearch is healthy, so this only affects in-place upgrades.