Observability & Monitoring

Use the prom-client library to define Counter, Histogram, and Gauge metrics, then register middleware that records request duration and counts per route. Expose a /metrics endpoint returning prometheus.register.metrics() for Prometheus to scrape.

Monitoring tracks known unknowns like whether a system is up, while observability handles unknown unknowns like why the system is slow for users in a specific region. Good observability lets you ask questions you did not anticipate.

Initialize the OpenTelemetry NodeSDK with auto-instrumentations and a Jaeger exporter, then create manual spans with tracer.startActiveSpan for key operations. Set attributes and record exceptions on spans to capture context across service calls.

Never log passwords, tokens, API keys, credit card numbers, SSNs, or other PII. Also avoid full request bodies that may contain secrets and binary data, while still logging authentication events, errors, and business events with context.

Error budget equals 100% minus the SLO target. For a 99.9% availability SLO, the error budget is 0.1%, which equals about 43 minutes of downtime per month, and you alert when consumption approaches the budget limit.