Add Grafana Dashboards for KSML Monitoring

[KasparMetsa]

Summary

Create production-ready Grafana dashboards for monitoring KSML applications using the existing Prometheus metrics integration.

Background

• KSML already exposes JMX metrics via Prometheus (port 9999)
• kafka-streams-dashboards provides 14+ proven Kafka Streams dashboards
• Need to adapt these for KSML's metric naming format (lowercase labels)

Scope

Focus on Kafka Streams specific metrics:

• Message throughput and latency
• State store performance
• Task and thread metrics
• Processing errors
• Python function metrics (KSML-specific)

Out of scope:

• Pod resource metrics (CPU/Memory/Disk)

Deliverables

1. Docker Compose setup with Prometheus + Grafana + KSML
   • Located in docs/monitoring/
   • Zero-config setup for users
2. Adapted dashboards (~6-8 core dashboards)
   • Kafka Streams topology overview
   • Throughput and latency
   • State store performance
   • Error handling
   • KSML-specific metrics
3. Documentation
   • Getting started guide
   • Dashboard user guide
   • Custom metrics guide

Considerations

• KSML abstracts Kafka Streams complexity -> use user-friendly naming in dashboards
• Keep standalone/Axual-independent for open source users
• Can serve as foundation for later Axual Metrics Exposer integration

Success Criteria

• Users can run docker-compose up and see monitoring dashboards
• Works with existing KSML examples
• Looks great in demos