Lightweight Observability and Monitoring Stack

This project sets up a complete lightweight observability and monitoring setup using Node.js, Prometheus, Grafana, and Node Exporter via Docker Compose.

Components

• Node.js App: A simple Express app exposing a /metrics endpoint using prom-client for Prometheus metrics.
• Prometheus: Scrapes metrics from the app and Node Exporter, evaluates alert rules.
• Node Exporter: Exports system metrics (CPU, memory, etc.) from the host.
• Grafana: Visualizes metrics with a custom dashboard.
• Alert Dispatcher: Bash script to fetch and log active alerts from Prometheus API.

Architecture

The setup includes:

• Metrics collection from app and system
• Alerting on high CPU usage (>70%) and app unavailability
• Visualization dashboard for CPU, memory, and request latency
• Local logging of alerts

Prerequisites

• Docker
• Docker Compose

Quick Start

1. Clone or navigate to the project directory.

2. Start all services:

   docker-compose up -d

3. Access the services:

   • App: http://localhost:3000
   • Prometheus: http://localhost:9090
   • Grafana: http://localhost:3001 (admin/admin)
   • Node Exporter: http://localhost:9100

4. View metrics at http://localhost:3000/metrics

5. Run alert dispatcher (optional):

   ./alert_dispatcher.sh

   Check alerts.log for logged alerts.

6. Stop services:

   docker-compose down

Files Overview

• docker-compose.yml: Service definitions
• prometheus.yml: Prometheus configuration
• alert.rules.yml: Alert rules
• grafana-dashboard.json: Grafana dashboard configuration
• app/: Node.js application
  • server.js: Main app with metrics
  • package.json: Dependencies
  • Dockerfile: App container
• alert_dispatcher.sh: Alert logging script
• dashboard_screenshot_placeholder.txt: Sample dashboard description

Alert Rules

• HighCPUUsage: Triggers when CPU > 70% for 5 minutes
• AppDown: Triggers when app is unreachable for 1 minute

Dashboard

The Grafana dashboard includes panels for:

• CPU Usage (%)
• Memory Usage (%)
• Request Latency (95th percentile)

Import grafana-dashboard.json into Grafana if needed.

Troubleshooting

• Ensure ports 3000-9100 are free.
• Check Docker logs: docker-compose logs <service>
• For alerts, ensure Prometheus is scraping correctly.

Sample Screenshot Placeholder

See dashboard_screenshot_placeholder.txt for a description of the dashboard with triggered alerts.