evaluate-run

Grade a completed agent run against conventions using a compressed transcript digest and diff.

21|1|Updated Jul 22, 2026
One-click install
npx skills add https://github.com/uiverify/uiverify --skill evaluate-run-uiverify
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluate-run
Source: https://github.com/uiverify/uiverify/tree/main/packages/skills/skills/evaluate-run
Command: npx skills add https://github.com/uiverify/uiverify --skill evaluate-run-uiverify

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? A green CI run does not prove an agent's work was good: it can pass while bending conventions, writing tautological tests, or repeating mistakes you already corrected. This Skill judges a finished factory run cold and turns the findings into durable rules and memories so the system improves over time. ## Core Features & Use Cases - Cold adversarial evaluation: Spawns a fresh evaluator subagent that judges from a compressed session digest plus the real git diff, never from the agent's own self-narration. - Seven-dimension rubric: Scores loop adherence, convention compliance, test integrity, ask-vs-proceed judgment, correction handling, verification honesty, and efficiency with file:line evidence. - Durable lesson routing: Routes RULE findings to /add-rule (conventions doc plus lint guard) and PROCESS findings to persistent memory, acting only on user approval. - Use Case: After a factory run converges on a green PR, ask for an evaluation to discover it mocked the database in a test that required real dependencies, then capture that as a permanent lint guard. ## Quick Start Ask the agent to evaluate the current factory run and grade how it went against your conventions.

Frequently Asked Questions about evaluate-run

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate a completed AI coding agent run?▼

Run the evaluation on the finished worktree or an explicit transcript.jsonl file. The Skill compresses the session into a digest of prompts, corrections, and tool timeline, then a fresh evaluator subagent scores it against a rubric using the real git diff as ground truth.

What does the evaluation rubric check in an agent run?▼

The rubric scores seven dimensions: loop adherence, convention compliance, test integrity, ask-versus-proceed judgment, correction handling, verification honesty, and efficiency. Each dimension gets a pass, warn, or fail score backed by file:line or digest evidence.

Can I evaluate the current chat session while it is still running?▼

Evaluating the current chat works, but the digest will not include the in-flight turn. For a clean read, evaluate a run that has come to rest, meaning it has converged or stopped.

Why evaluate a run that already passed CI?▼

CI only proves the code compiles and tests pass. The evaluation targets what CI cannot see: convention violations, tautological tests, poor judgment on when to ask questions, and corrections that were never captured durably.

What happens to the issues the evaluator finds?▼

Each finding is tagged RULE or PROCESS with a proposed durable fix. RULE findings go to /add-rule for a conventions doc entry and lint guard, while PROCESS findings become persistent memories, all applied only after your approval.