Release: Phase 18a — Prometheus + Grafana (#61)

docker-compose.yml

@@ -125,5 +125,34 @@ services:
     ports:
       - "8080:8080"
 
+  prometheus:
+    # Observability stack — only with the `full` profile (it scrapes the containerised app).
+    profiles: ["full"]
+    image: prom/prometheus:v2.54.1
+    container_name: shopsphere-prometheus
+    depends_on:
+      app:
+        condition: service_started
+    volumes:
+      - ./observability/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
+    ports:
+      - "9090:9090"
+
+  grafana:
+    profiles: ["full"]
+    image: grafana/grafana:11.2.0
+    container_name: shopsphere-grafana
+    depends_on:
+      prometheus:
+        condition: service_started
+    environment:
+      # Default admin/admin. GF_SECURITY_ADMIN_PASSWORD is intentionally NOT set so Grafana keeps
+      # its built-in "change password on first login" prompt (the AC). Rotate before any real use.
+      GF_USERS_ALLOW_SIGN_UP: "false"
+    volumes:
+      - ./observability/grafana/provisioning:/etc/grafana/provisioning:ro
+    ports:
+      - "3000:3000"
+
 volumes:
   shopsphere-pgdata:

docs/adr/0018a-prometheus-and-grafana-over-existing-meters.md

@@ -0,0 +1,31 @@
+---
+status: accepted
+date: 2026-06-06
+cites: PragProg, APoSD, XP, PoEAA
+---
+
+# 0018a — Prometheus + Grafana over the meters we already have
+
+Phase 9 instrumented ShopSphere with four business meters — `orders_placed_total`, `payments_total`, `reservations_total`, and a `checkout_latency_seconds` timer — exposed only through `/actuator/metrics` (a JSON poke-and-hope endpoint). Phase 18a makes them *legible over time*: a Prometheus scrape endpoint, a Prometheus to store the series, and a provisioned Grafana dashboard. It adds **no new meters** and **no application code beyond a registry dependency**.
+
+## Scrape, don't push; reuse, don't re-instrument
+
+Adding `micrometer-registry-prometheus` and exposing `/actuator/prometheus` is the whole code change. The four meters were already there and correctly placed (Phase 9 put each counter *behind the dedupe gate* so redelivery never double-counts); a metrics backend is a read-only consumer of that work. **XP YAGNI** — the brief was explicitly "no new meters," and the dashboard is built from what exists. Resisting the urge to add request-rate or JVM panels keeps the dashboard about the *business* (orders, payments, reservations, checkout latency), which is what the four meters were chosen to express.
+
+Prometheus **pulls** from the app rather than the app pushing. **PragProg / PoEAA** — pull keeps the application ignorant of its monitoring: the app exposes a text endpoint and knows nothing about Prometheus, retention, or Grafana. The monitoring topology can change (more scrapers, federation, remote-write) without touching a line of application code. The scrape target is the app's compose service name (`app:8080`), so the wiring is declarative config in `prometheus.yml`, not code.
+
+## The dashboard targets real exported names, not guesses
+
+Micrometer's Prometheus naming has a well-known trap: a counter whose Micrometer name already ends in `_total` can export as either `…_total` or `…_total_total` depending on client version, and a timer fans out into `_count`/`_sum`/`_bucket`/`_max`. Rather than guess the PromQL, the integration test (`PrometheusEndpointIT`) records each meter and reads the actual exposition output — confirming the exported families are `orders_placed_total`, `payments_total`, `reservations_total` (no doubled suffix, on Micrometer 1.13 with the new Prometheus client) and `checkout_latency_seconds_bucket`. The committed dashboard's queries are written against those verified names. **APoSD** — the test is the place that *knows* the metric names; the dashboard borrows that knowledge instead of duplicating an assumption that silently rots.
+
+A second subtlety the test pinned down: `@SpringBootTest` disables metrics export by default (so test runs don't push to real backends), so the test needs `@AutoConfigureObservability` to see the endpoint. Production export is on by default and unaffected — recorded here because the asymmetry is surprising the first time it 404s a test.
+
+## Provisioned, not clicked
+
+Both the Prometheus datasource and the dashboard are **provisioned from committed files** (`observability/grafana/provisioning/`), so the dashboard is versioned with the code and survives a container wipe — no manual Grafana clicking to reproduce. **PragProg — automate the setup**: `docker compose --profile full up` brings up app + Prometheus + Grafana with the datasource and dashboard already wired. The observability stack lives under the `full` profile because it observes the containerised app; the default dev loop (Postgres + Kafka only) is unchanged, consistent with how Phase 10 gated the `app` service.
+
+Grafana's admin password is intentionally left at the `admin/admin` default so its built-in first-login change prompt stays in force — a seeded credential to rotate, the same honest posture as the Phase-17 seeded admin.
+
+## Consequences
+
+Metrics are now queryable over time and visible on a dashboard with zero new instrumentation and no external dependency — the whole stack is local containers. `mvn verify` stays green; `PrometheusEndpointIT` guards the endpoint and, implicitly, the metric names the dashboard depends on. The one manual step is the acceptance check that the four panels light up with non-zero data, which requires running the QA walkthrough against the `full` stack to generate traffic — a manual QA step, like the cloud-phase walkthroughs, not something a unit test asserts. Phase 18b builds on this Grafana with logs (Loki) as a second data source.

observability/grafana/provisioning/dashboards/dashboards.yml

@@ -0,0 +1,11 @@
+# Tells Grafana to load committed dashboard JSON from disk on startup (Phase 18a).
+apiVersion: 1
+
+providers:
+  - name: shopsphere
+    type: file
+    disableDeletion: false
+    allowUiUpdates: true
+    options:
+      path: /etc/grafana/provisioning/dashboards
+      foldersFromFilesStructure: false

observability/grafana/provisioning/dashboards/shopsphere-overview.json

@@ -0,0 +1,84 @@
+{
+  "uid": "shopsphere-overview",
+  "title": "ShopSphere — business metrics",
+  "tags": ["shopsphere"],
+  "timezone": "browser",
+  "schemaVersion": 39,
+  "version": 1,
+  "refresh": "10s",
+  "time": { "from": "now-1h", "to": "now" },
+  "panels": [
+    {
+      "id": 1,
+      "type": "timeseries",
+      "title": "Orders placed (by outcome)",
+      "description": "orders_placed_total, split by the checkout outcome tag.",
+      "datasource": { "type": "prometheus", "uid": "prometheus" },
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
+      "fieldConfig": { "defaults": { "custom": { "drawStyle": "line", "stacking": { "mode": "none" } } }, "overrides": [] },
+      "options": { "legend": { "displayMode": "list", "placement": "bottom" } },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "prometheus" },
+          "expr": "sum by (outcome) (orders_placed_total)",
+          "legendFormat": "{{outcome}}"
+        }
+      ]
+    },
+    {
+      "id": 2,
+      "type": "timeseries",
+      "title": "Payments (by outcome)",
+      "description": "payments_total, split by the payment outcome tag.",
+      "datasource": { "type": "prometheus", "uid": "prometheus" },
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
+      "fieldConfig": { "defaults": { "custom": { "drawStyle": "line", "stacking": { "mode": "none" } } }, "overrides": [] },
+      "options": { "legend": { "displayMode": "list", "placement": "bottom" } },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "prometheus" },
+          "expr": "sum by (outcome) (payments_total)",
+          "legendFormat": "{{outcome}}"
+        }
+      ]
+    },
+    {
+      "id": 3,
+      "type": "timeseries",
+      "title": "Stock reservations (by status)",
+      "description": "reservations_total, split by GRANTED/DENIED status.",
+      "datasource": { "type": "prometheus", "uid": "prometheus" },
+      "gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
+      "fieldConfig": { "defaults": { "custom": { "drawStyle": "line", "stacking": { "mode": "none" } } }, "overrides": [] },
+      "options": { "legend": { "displayMode": "list", "placement": "bottom" } },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "prometheus" },
+          "expr": "sum by (status) (reservations_total)",
+          "legendFormat": "{{status}}"
+        }
+      ]
+    },
+    {
+      "id": 4,
+      "type": "timeseries",
+      "title": "Checkout latency (p95, 5m)",
+      "description": "95th percentile of checkout_latency_seconds from the histogram buckets.",
+      "datasource": { "type": "prometheus", "uid": "prometheus" },
+      "gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
+      "fieldConfig": { "defaults": { "unit": "s", "custom": { "drawStyle": "line" } }, "overrides": [] },
+      "options": { "legend": { "displayMode": "list", "placement": "bottom" } },
+      "targets": [
+        {
+          "refId": "A",
+          "datasource": { "type": "prometheus", "uid": "prometheus" },
+          "expr": "histogram_quantile(0.95, sum by (le) (rate(checkout_latency_seconds_bucket[5m])))",
+          "legendFormat": "p95"
+        }
+      ]
+    }
+  ]
+}

observability/grafana/provisioning/datasources/prometheus.yml

@@ -0,0 +1,11 @@
+# Auto-provisions Prometheus as Grafana's default datasource on startup (Phase 18a).
+apiVersion: 1
+
+datasources:
+  - name: Prometheus
+    uid: prometheus
+    type: prometheus
+    access: proxy
+    url: http://prometheus:9090
+    isDefault: true
+    editable: false

observability/prometheus/prometheus.yml

@@ -0,0 +1,11 @@
+# Prometheus scrape config for ShopSphere (Phase 18a).
+# Scrapes the app's Micrometer Prometheus endpoint. The app is reachable by its compose
+# service name `app` on the internal network; metrics_path points at the actuator endpoint.
+global:
+  scrape_interval: 15s
+
+scrape_configs:
+  - job_name: shopsphere
+    metrics_path: /actuator/prometheus
+    static_configs:
+      - targets: ["app:8080"]

pom.xml

@@ -44,6 +44,11 @@
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-actuator</artifactId>
         </dependency>
+        <dependency>
+            <groupId>io.micrometer</groupId>
+            <artifactId>micrometer-registry-prometheus</artifactId>
+            <scope>runtime</scope>
+        </dependency>
 
         <dependency>
             <groupId>org.postgresql</groupId>

src/main/resources/application.yml

@@ -50,7 +50,7 @@ management:
   endpoints:
     web:
       exposure:
-        include: health,info,metrics,modulith
+        include: health,info,metrics,modulith,prometheus
   endpoint:
     health:
       show-details: always

src/test/java/com/shopsphere/PrometheusEndpointIT.java

@@ -0,0 +1,65 @@
+package com.shopsphere;
+
+import io.micrometer.core.instrument.MeterRegistry;
+import io.micrometer.core.instrument.Timer;
+import org.junit.jupiter.api.Test;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.boot.test.autoconfigure.actuate.observability.AutoConfigureObservability;
+import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
+import org.springframework.boot.test.context.SpringBootTest;
+import org.springframework.test.context.DynamicPropertyRegistry;
+import org.springframework.test.context.DynamicPropertySource;
+import org.springframework.test.web.servlet.MockMvc;
+import org.springframework.test.web.servlet.MvcResult;
+
+import java.time.Duration;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
+import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
+
+/**
+ * Phase 18a — the app exposes a Prometheus scrape endpoint. Asserts {@code /actuator/prometheus}
+ * serves the exposition format and that the four existing business meters appear once recorded
+ * (Micrometer registers a meter lazily on first use). No new meters are introduced.
+ * <p>
+ * {@link AutoConfigureObservability} is required because {@code @SpringBootTest} otherwise disables
+ * metrics export in tests; production export is on by default, so this only re-enables it here.
+ */
+@SpringBootTest
+@AutoConfigureMockMvc
+@AutoConfigureObservability
+class PrometheusEndpointIT {
+
+    @DynamicPropertySource
+    static void containers(DynamicPropertyRegistry registry) {
+        SharedContainers.registerProperties(registry);
+    }
+
+    @Autowired
+    MockMvc mockMvc;
+
+    @Autowired
+    MeterRegistry meters;
+
+    @Test
+    void prometheusEndpointServesTheBusinessMeters() throws Exception {
+        // Touch each meter with its production name + tag key so it is registered for the scrape.
+        meters.counter("orders_placed_total", "outcome", "placed").increment();
+        meters.counter("payments_total", "outcome", "succeeded").increment();
+        meters.counter("reservations_total", "status", "granted").increment();
+        Timer.builder("checkout_latency_seconds").register(meters).record(Duration.ofMillis(5));
+
+        MvcResult result = mockMvc.perform(get("/actuator/prometheus"))
+                .andExpect(status().isOk())
+                .andReturn();
+        String body = result.getResponse().getContentAsString();
+
+        assertThat(body).contains("jvm_memory_used_bytes");
+        // Micrometer suffixes counters with _total; the timer becomes _seconds_count/_sum/_bucket.
+        assertThat(body).contains("orders_placed_total");
+        assertThat(body).contains("payments_total");
+        assertThat(body).contains("reservations_total");
+        assertThat(body).contains("checkout_latency_seconds");
+    }
+}