skills.rest
swarm-ai-research
swarm-ai-researchOFFICIAL
Explore all skills by this author →
Website𝕏@ResearchSwarmAIGitHub

Skill Evals

Compare latest skill-evals results against prior runs to surface regressions.

Updated Jun 3, 2026
Search Tags:#automation#metrics#evaluation#quality#governance#regression#evals
One-click install
npx skills add https://github.com/swarm-ai-research/aeon --skill skill-evals-swarm-ai-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Skill Evals
Source: https://github.com/swarm-ai-research/aeon/tree/main/skills/skill-evals
Command: npx skills add https://github.com/swarm-ai-research/aeon --skill skill-evals-swarm-ai-research

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This evaluation framework continuously monitors outputs from registered skills, detects regressions since the last run, and channels issues and fixes into a structured workflow to maintain quality over time.

Core Features & Use Cases

  • Automated cross-run comparisons: surface deltas between current and prior evals.
  • Issue filing and queue management: create open issues for regressions and track remediation.
  • Publish governance artifacts: generate articles and logs documenting verdicts, metrics, and actions.

Quick Start

Run the evaluation workflow to capture the latest results.

Frequently Asked Questions about Skill Evals

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect skill regressions and automate quality evaluation?▼

Skill regressions are detected by comparing the latest skill-evals results against the prior run to surface deltas. The framework checks output presence, word counts, required patterns, numeric bounds, and memory-backed quality scores.

How does automated issue filing work for evaluation regressions?▼

Automated issue filing creates open issues for detected regressions and tracks remediation through an executable action queue. This enforces a defined verdict workflow to maintain quality governance over time.

What quality metrics are checked during skill evaluation?▼

Quality metrics checked include output presence, word counts, required and forbidden patterns, numeric bounds, and memory-backed quality scores. These checks scope all tracked skills defined in the configuration.

Can I generate governance artifacts and logs from evaluation runs?▼

Governance artifacts are generated as published articles and logs documenting verdicts, metrics, and actions. These artifacts are produced automatically when the evaluation workflow completes its run.

What is the best way to compare current and prior evaluation runs?▼

Cross-run comparisons surface deltas between current and prior evals to identify output quality issues. Running the evaluation workflow captures the latest results and automatically flags any regressions.

Do I need dependencies to run automated skill evaluations?▼

No external dependencies are required to run automated skill evaluations. The framework operates independently to scope tracked skills, execute checks, and generate governance logs without additional packages.

Related Skills

Explore More Data & Analytics Skills→
VjayRamVjayRam

dspy-expert

Define DSPy signatures and modules with deterministic evaluation across splits.

Community
Advanced
csuzngjhcsuzngjh
1

watch-evolution

Runs a background daemon to diagnose and fix queued code issues.

Community
Advanced
williamzujkowskiwilliamzujkowski
17

authorization-security

Implement RBAC and ABAC authorization with policy enforcement and audit trails.

Community
Advanced
chriswesttchriswestt

canary

Monitor deployed apps for console errors, performance regressions, and page failures.

Community
Advanced