Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Metrics is a time series reporting framework for aggregators and metrics collect
* Built in [reporters](#Reporters):
* [Graphite (statsd)](#Graphite)
* [DataDog](#DataDog)
* [Prometheus (experimental)](#prometheus-experimental)
* [String](#String)
* [Console](#Console)
* [InMemory (for testing)](#InMemory)
Expand All @@ -29,6 +30,7 @@ Metrics is a time series reporting framework for aggregators and metrics collect
* [Reporters](#reporters)
* [Graphite](#Graphite)
* [DataDog](#DataDog)
* [Prometheus (experimental)](#prometheus-experimental)
* [String](#String)
* [Console](#Console)
* [InMemory](#InMemory)
Expand Down Expand Up @@ -275,6 +277,98 @@ const metrics = new Metrics({ reporters: [memoryReporter], errback: error => { /
When a metric is reported, an object with `key`, `value` and `tags` properties is pushed to the array.<br/>
Then, the array can be used in order to validate the report.

#### Prometheus (experimental)
PrometheusReporter is an experimental reporter that generates metrics in the [Prometheus text exposition format](https://prometheus.io/docs/instrumenting/exposition_formats/) without any external dependencies:
```js
const { Metrics, PrometheusReporter } = require('metrics-reporter');
const express = require('express');

const prefix = 'myapp_'; // Optional - prefix for all metric names
const softLimit = 5000; // Optional - Default `5000` - Reset metrics after scrape when exceeded
const hardLimit = 10000; // Optional - Default `10000` - Force reset to prevent OOM
const warnAt = 4000; // Optional - Default `4000` - Log warning when approaching soft limit
const buckets = [10, 50, 100, 250]; // Optional - Custom histogram buckets (ms)
const logCallback = (logEvent) => { // Optional - function to handle log events
console.log(`[${logEvent.level.toUpperCase()}] ${logEvent.message}`, logEvent.params);
};

const prometheusReporter = new PrometheusReporter({
prefix,
softLimit,
hardLimit,
warnAt,
buckets,
logCallback,
});

const metrics = new Metrics({ reporters: [prometheusReporter] });

// Expose metrics endpoint for Prometheus to scrape
const app = express();
app.get('/metrics', (req, res) => {
res
.type('text/plain')
.send(prometheusReporter.getMetrics());
});

app.listen(3000);
```

##### Design Principles
The PrometheusReporter implements a **double-threshold strategy** to prevent memory issues common with high-cardinality metrics:

1. **Soft Limit** (default 5000): When the number of unique metrics exceeds this limit, they are reset **after** the next scrape, ensuring Prometheus receives the data before clearing.

2. **Hard Limit** (default 10000): An emergency threshold that immediately resets all metrics to prevent out-of-memory errors, even if Prometheus hasn't scraped yet.

3. **Warning Threshold** (default 4000): Logs a warning when approaching the soft limit, helping identify cardinality issues before they become critical.

This approach provides **memory safety** while maintaining **data integrity**, unlike traditional Prometheus clients that can suffer from unbounded memory growth.

##### Metric Type Mapping
- `increment()` → Counter (with `_total` suffix)
- `value()` → Gauge
- `report()` → Histogram (with buckets, sum, and count)

##### Configuration Recommendations
Configuration should be according to your use-case, use these as guidelines to an initial configuration and tweak as needed:
- **Low traffic**: Use defaults (soft: 5000, hard: 10000)
- **High traffic**: Increase limits (soft: 20000, hard: 50000)
- **Microservices**: Lower limits (soft: 1000, hard: 2000)
- **Development**: Very low limits for testing (soft: 100, hard: 200)

##### Log Events
The PrometheusReporter provides structured logging through an optional `logCallback` function. This allows you to handle log events programmatically instead of relying on console output.

```js
const prometheusReporter = new PrometheusReporter({
logCallback: (logEvent) => {
// Handle log events with your preferred logging library
logger.log(logEvent.level, logEvent.message, {
code: logEvent.code,
params: logEvent.params,
reporter: logEvent.reporter,
timestamp: logEvent.timestamp
});
}
});
```

**Log Event Structure:**
- `level`: 'error' | 'warn' | 'info' | 'debug'
- `code`: Event code for programmatic handling
- `message`: Human-readable message
- `params`: Additional parameters relevant to the event
- `timestamp`: Unix timestamp in milliseconds
- `reporter`: Always 'PrometheusReporter'

**Event Codes:**
- `APPROACHING_SOFT_LIMIT`: Warning when approaching the soft limit threshold
- `SOFT_LIMIT_EXCEEDED`: Info when soft limit exceeded during getMetrics()
- `HARD_LIMIT_REACHED`: Error when hard limit forces immediate reset

We recommend monitoring these events, as threshold violations can cause metric data loss.

### Building new reporters
Metrics support creating new reports according to an application needs.

Expand Down
15 changes: 15 additions & 0 deletions docker/docker-compose-prometheus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
version: '3'
services:
prometheus:
image: prom/prometheus:latest
restart: always
ports:
- '9090:9090'
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
- '--web.enable-lifecycle'
9 changes: 9 additions & 0 deletions docker/prometheus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
global:
scrape_interval: 15s

scrape_configs:
- job_name: 'metrics-reporter-example'
static_configs:
- targets: ['host.docker.internal:3000']
scrape_interval: 5s
metrics_path: /metrics
52 changes: 52 additions & 0 deletions examples/prometheus.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/**
* Simple Prometheus Reporter Example
*
* This example shows how to expose Prometheus metrics via HTTP for scraping.
* To run this example:
* 1. node examples/prometheus.js
* 2. Visit http://localhost:3000/metrics to see Prometheus metrics
*/
const http = require('http');
const { Metrics, PrometheusReporter } = require('..');

// Create the Prometheus reporter
const prometheusReporter = new PrometheusReporter({
prefix: 'myapp_',
});

// Initialize metrics
const metrics = new Metrics({
reporters: [prometheusReporter],
});

// Generate some example metrics
function generateMetrics() {
// Counter: HTTP requests
metrics.space('http_requests', { method: 'GET', status: '200' }).increment();
metrics.space('http_requests', { method: 'POST', status: '201' }).increment(2);
// Gauge: Active connections
metrics.space('active_connections').value(Math.floor(Math.random() * 50) + 10);
}

// Generate initial metrics
generateMetrics();

// Continue generating metrics every 5 seconds
setInterval(generateMetrics, 5000);

// Create HTTP server with metrics endpoint
const server = http.createServer((req, res) => {
if (req.url === '/metrics') {
res.writeHead(200, { 'Content-Type': 'text/plain; version=0.0.4' });
res.end(prometheusReporter.getMetrics());
} else {
res.writeHead(404, { 'Content-Type': 'text/plain' });
res.end('Try /metrics');
}
});

const PORT = 3000;
server.listen(PORT, () => {
console.log(`Server running on http://localhost:${PORT}`);

Check warning on line 50 in examples/prometheus.js

View workflow job for this annotation

GitHub Actions / build

Unexpected console statement
console.log(`Metrics available at http://localhost:${PORT}/metrics`);

Check warning on line 51 in examples/prometheus.js

View workflow job for this annotation

GitHub Actions / build

Unexpected console statement
});
13 changes: 8 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"datadog",
"DogStatsD"
],
"version": "2.1.0",
"version": "2.2.0",
"repository": {
"type": "git",
"url": "https://github.com/ysa23/metrics-reporter"
Expand All @@ -25,10 +25,13 @@
"tsc:watch": "tsc -w",
"example:graphite": "node ./examples/graphite.js",
"example:datadog": "node ./examples/datadog.js",
"docker:datadog:up": "cd ./docker && docker-compose -f docker-compose-datadog.yml up -d",
"docker:datadog:down": "cd ./docker && docker-compose -f docker-compose-datadog.yml down",
"docker:graphite:up": "cd ./docker && docker-compose -f docker-compose-graphite.yml up -d",
"docker:graphite:down": "cd ./docker && docker-compose -f docker-compose-graphite.yml down"
"example:prometheus": "node ./examples/prometheus.js",
"docker:datadog:up": "cd ./docker && docker compose -f docker-compose-datadog.yml up -d",
"docker:datadog:down": "cd ./docker && docker compose -f docker-compose-datadog.yml down",
"docker:graphite:up": "cd ./docker && docker compose -f docker-compose-graphite.yml up -d",
"docker:graphite:down": "cd ./docker && docker compose -f docker-compose-graphite.yml down",
"docker:prometheus:up": "cd ./docker && docker compose -f docker-compose-prometheus.yml up -d",
"docker:prometheus:down": "cd ./docker && docker compose -f docker-compose-prometheus.yml down"
},
"devDependencies": {
"eslint": "^6.8.0",
Expand Down
2 changes: 1 addition & 1 deletion src/metrics.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import {IReporter} from "./types/reporter";

declare interface MetricsOptions {
reporters: IReporter[];
tags: Tags;
tags?: Tags;
errback?: ErrorCallback;
}

Expand Down
1 change: 1 addition & 0 deletions src/reporters/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ export * from "./console-reporter";
export * from "./datadog-reporter";
export * from "./graphite-reporter";
export * from "./in-memory-reporter";
export * from "./prometheus-reporter";
export * from "./string-reporter";
2 changes: 2 additions & 0 deletions src/reporters/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@ const { ConsoleReporter } = require('./console-reporter');
const { DataDogReporter } = require('./datadog-reporter');
const { GraphiteReporter } = require('./graphite-reporter');
const { InMemoryReporter } = require('./in-memory-reporter');
const { PrometheusReporter } = require('./prometheus-reporter');
const { StringReporter } = require('./string-reporter');

module.exports = {
ConsoleReporter,
DataDogReporter,
GraphiteReporter,
InMemoryReporter,
PrometheusReporter,
StringReporter,
};
23 changes: 23 additions & 0 deletions src/reporters/prometheus-reporter.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import {IReporter} from "../types/reporter";
import {Tags} from "../types/tags";
import {LogCallback} from "../types/log-event";

declare interface PrometheusReporterOptions {
prefix?: string;
softLimit?: number;
hardLimit?: number;
warnAt?: number;
buckets?: number[];
logCallback?: LogCallback;
}

export declare class PrometheusReporter implements IReporter {
constructor(options?: PrometheusReporterOptions);

report(key: string, value: number, tags?: Tags): void;
value(key: string, value: number, tags?: Tags): void;
increment(key: string, value?: number, tags?: Tags): void;

getMetrics(): string;
close(): void;
}
Loading
Loading