observability-and-instrumentation

Instruments production code with structured logs, metrics, traces, and symptom-based alerts.

Updated May 21, 2026
One-click install
npx skills add https://github.com/nicorevo/AI-SDLC-Template --skill observability-and-instrumentation-nicorevo
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: observability-and-instrumentation
Source: https://github.com/nicorevo/AI-SDLC-Template/tree/main/.opencode/skills/observability-and-instrumentation
Command: npx skills add https://github.com/nicorevo/AI-SDLC-Template --skill observability-and-instrumentation-nicorevo

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Production features often ship without telemetry, so when incidents occur engineers cannot tell what happened from the available data. This Skill guides you to instrument code alongside the feature so production behavior is visible and diagnosable from day one. ## Core Features & Use Cases - Structured Logging: Enforces JSON log events with stable event names, consistent log levels, mandatory correlation/request IDs, and strict rules against logging secrets or PII. - Metrics and Tracing: Applies RED (Rate, Errors, Duration) and USE (Utilization, Saturation, Errors) methods with histogram-based latency, cardinality-safe labels, and OpenTelemetry distributed tracing with context propagation. - Symptom-Based Alerting: Defines actionable alerts tied to user-facing symptoms with runbook links, thresholds, and two severity tiers, plus a verification step that test-fires alerts and validates telemetry end-to-end. - Use Case: When adding a payment retry flow with external provider calls, use this Skill to define the on-call questions first, then emit structured payment_failed events, RED metrics on the provider dependency, and an alert on elevated error rate. ## Quick Start Use the observability-and-instrumentation skill to add structured logging, RED metrics, and OpenTelemetry tracing to my new checkout endpoint.

Frequently Asked Questions about observability-and-instrumentation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add structured logging to a production service?▼

Structured logging means emitting JSON log events with stable event names and machine-readable fields instead of string-interpolated prose. Attach a correlation ID at the system boundary and include it on every log line, and never log secrets, tokens, or full PII.

What metrics should I track for an HTTP endpoint?▼

Use the RED method: Rate (requests per second), Errors (failure rate), and Duration (latency histogram). Track percentiles like p95 and p99 rather than averages, and keep label values from small fixed sets such as route templates and status classes.

How do I set up distributed tracing with OpenTelemetry?▼

Initialize the OpenTelemetry NodeSDK with auto-instrumentations before importing anything else, which covers HTTP, gRPC, and common database clients. Add manual spans around meaningful internal work and propagate context across async boundaries so traces do not break.

Why should I avoid user IDs as metric labels?▼

Every unique label combination creates a separate time series, so unbounded values like user IDs, raw URLs, or error messages cause cardinality explosions that can take down the metrics backend. High-cardinality lookups belong in logs and traces instead.

When should I use symptom-based vs cause-based alerts?▼

Alert on symptoms users feel, such as error rate above 1% or p99 latency above 2 seconds, rather than causes like CPU usage. Cause-based alerts fire when nothing is wrong and miss unpredicted failures, while symptom alerts fire exactly when users are hurt.

When is this observability skill not the right tool?▼

It is not for diagnosing an active failure, which belongs to debugging workflows, nor for profiling measured slowness, which is performance optimization. It covers the instrumentation written alongside features that makes later diagnosis fast.