model-monitor

Monitors LLM token usage, Pi-hole DNS stats, and Grafana Alloy metrics for Hermes infrastructure.

Updated May 29, 2026
One-click install
npx skills add https://github.com/Reimonsk8/hermes-9router-model-balancing --skill model-monitor-reimonsk8
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: model-monitor
Source: https://github.com/Reimonsk8/hermes-9router-model-balancing/tree/main/scripts/skills/devops/model-monitor
Command: npx skills add https://github.com/Reimonsk8/hermes-9router-model-balancing --skill model-monitor-reimonsk8

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Tracking LLM token consumption across multiple models and providers is error-prone when relying on hardcoded model lists, and infrastructure stats like Pi-hole ad-blocking become inaccessible when web APIs fail. This Skill provides dynamic monitoring of Hermes LLM usage alongside server and network observability. ## Core Features & Use Cases - Dynamic Model Discovery: Parses ~/.hermes/config.yaml to retrieve the active model chain instead of relying on hardcoded lists, and sorts usage by consumption percentage with unconfigured models last. - Token Usage Reporting: Runs monitor scripts and reports a calculated total of tokens used across all models for clear consumption visibility. - Pi-hole Stats via FTL Database: Queries /etc/pihole/pihole-FTL.db directly with SQLite when the Pi-hole API returns 404, correctly counting GRAVITY (status 1) blocks while excluding GRAVITY_CNAME (status 17). - Grafana Alloy Deployment: Guides metrics and log scraping setup on standalone Linux (including Alpine) with node_exporter-style metrics and journal logs. - Use Case: When a user asks for a model usage report, run the monitor script, sum the used tokens, and include the total in the summary so consumption state is immediately visible. ## Quick Start Ask the assistant to show current LLM token usage across all configured models including a total consumption summary.

Frequently Asked Questions about model-monitor

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor LLM token usage across multiple models?▼

Run the monitor script after parsing ~/.hermes/config.yaml to discover the active model chain dynamically. Sort models by consumption percentage with unconfigured models last, and always include a summed total of used tokens in the report.

How to get Pi-hole stats when the API returns 404?▼

Query the FTL SQLite database directly at /etc/pihole/pihole-FTL.db using SQL against the query_storage table. Count rows with status = 1 (GRAVITY) for the true ad-block count, excluding status 17 (GRAVITY_CNAME).

Why is my Pi-hole ad-block count higher than the dashboard shows?▼

You are likely summing GRAVITY_CNAME (status 17) with GRAVITY (status 1). Status 17 is a CNAME chain lookup, not a true block, so only status 1 should be counted as ads blocked.

Can Grafana Alloy run on Alpine Linux for metrics and logs?▼

Yes, Grafana Alloy deploys on standalone Linux including Alpine as the successor to Grafana Agent. It scrapes node_exporter-style metrics and journal logs as JSON, and requires rc-service alloy restart after config changes.

Why are new Prometheus metrics files not being scraped by Alloy?▼

The file path must be explicitly added to the local.file_match targets in config.alloy, and the Alloy service must be restarted for changes to take effect. Missing either step leaves new metrics files unscraped.