infra-observability-stack-review

Design and audit Kubernetes observability stacks using LGTM with OpenTelemetry Collector.

14|1|Updated May 5, 2026
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill infra-observability-stack-review
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: infra-observability-stack-review
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/infra_dataops_group_skills/infra_observability_stack_review
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill infra-observability-stack-review

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Обеспечивает аудит и проектирование observability-стека для data platform так, чтобы метрики, логи и трассировки работали согласованно, имели управляемую кардинальность и поддерживали SLO вместо “пороговых” алертов.

Core Features & Use Cases

  • Three-pillar architecture (metrics/logs/traces): проектирует сквозной контур наблюдаемости с единым сборщиком через OpenTelemetry Collector и унифицированной визуализацией в Grafana.
  • LGTM stack setup (Loki + Grafana + Tempo + Prometheus/Mimir): описывает связку для long-term metrics, лог-агрегации и distributed tracing.
  • Cardinality management + structured logs: задаёт подходы к предотвращению взрыва series и рекомендацию по JSON-логированию с полями вроде dag_id/task_id/level.
  • SLO & error-budget tracking: помогает определить SLO, индикаторы и burn-rate алерты (например, через Pyrra), а также контролировать pipeline freshness/throughput/error rate.
  • Observability as Code (Grafana provisioning): автоматизирует публикацию дашбордов и алертов через Kubernetes-конфиги.
  • Use Case: при запуске новой платформы данных или при разборе production-инцидента, когда “ничего не видно”, но нужно быстро закрыть gaps по метрикам, логам и трассам.

Quick Start

Use this skill to audit your current monitoring and design an LGTM + OpenTelemetry + SLO-based observability plan that prevents high-cardinality issues and enables pipeline freshness, throughput, and error-rate dashboards and alerts.

Frequently Asked Questions about infra-observability-stack-review

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an observability stack for a Kubernetes data platform using LGTM and OpenTelemetry?▼

To design an LGTM observability stack, integrate metrics, logs, and traces via a unified OpenTelemetry Collector, route long-term metrics to Prometheus or Mimir, aggregate logs in Loki, and correlate traces in Tempo.

What is the best way to control Prometheus cardinality when logging structured JSON data?▼

Control Prometheus cardinality by enforcing structured JSON logging with specific fields like dag_id and task_id, and apply cardinality management rules at the OpenTelemetry Collector to prevent series explosions.

How do I set up SLO and error-budget burn-rate alerting instead of threshold alerts?▼

Set up SLO and error-budget burn-rate alerting by defining service level indicators for pipeline freshness, throughput, and error rates, then configure burn-rate alerts using tools like Pyrra.

Can I automate Grafana dashboard and alert provisioning for observability as code?▼

Yes, you can automate Grafana dashboard and alert provisioning by defining observability-as-code configurations and deploying them through Kubernetes manifests to ensure consistent visualization.

Does OpenTelemetry Collector work with Loki and Tempo for incident debugging?▼

OpenTelemetry Collector works with Loki and Tempo by routing logs and traces to their respective backends, enabling consistent incident debugging and cross-correlation within the Grafana LGTM ecosystem.

When should I audit my observability stack for metrics and trace gaps?▼

Audit your observability stack when launching a new data platform or during a production incident where visibility is lacking, to quickly identify and close gaps across metrics, logs, and traces.