sddl-eval-report

Rebuild sdd-lite evaluation reports and compare candidate runs against named baselines.

Updated Apr 2, 2026
One-click install
npx skills add https://github.com/nico0695/ai-tools --skill sddl-eval-report-nico0695
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sddl-eval-report
Source: https://github.com/nico0695/ai-tools/tree/main/sdd/sdd-lite-evals/skills/sddl-eval-report
Command: npx skills add https://github.com/nico0695/ai-tools --skill sddl-eval-report-nico0695

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Reviewing sdd-lite evaluation trends, regressions, and token usage requires aggregating scattered run evidence without launching another expensive model-backed test. This Skill rebuilds campaign reports and local history from existing evidence, or compares a candidate run against an explicitly named baseline. ## Core Features & Use Cases - Campaign Report Rebuild: Regenerate evaluation reports for a workspace and campaign using the sddl_eval.py report command. - Explicit Baseline Comparison: Compare a candidate run against a named baseline run, flagging comparisons as confounded when provider, case, project hash, or model differ. - Verdict and Cost Summaries: Summarize verdict deltas, new/persisting/resolved findings, and token or cost changes while keeping raw transcripts out of the human summary. - Use Case: After a week of sdd-lite evaluation runs, rebuild the campaign report to spot regressions and compare yesterday's candidate run against last week's baseline to explain token usage differences. ## Quick Start Ask the assistant to rebuild the sdd-lite evaluation report for your workspace and campaign, or to compare a specific candidate run against a named baseline run.

Frequently Asked Questions about sddl-eval-report

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I rebuild an sdd-lite evaluation report?▼

Run the sddl_eval.py report command with your workspace id and campaign name. This aggregates existing evidence into a campaign report without launching another model-backed test.

How do I compare two sdd-lite evaluation runs?▼

Pass both --candidate-run and --baseline-run identifiers to the report command. The Skill never auto-selects the latest run, so you must name both runs explicitly.

What makes an evaluation run comparison confounded?▼

A comparison is labeled confounded when provider, case, project hash, or model differ between the candidate and baseline runs. The Skill explains the specific differences so verdict deltas are not misattributed.

Does rebuilding a report run new model-backed tests?▼

No. The report command only aggregates existing evidence from prior runs. It summarizes verdict deltas, findings, and token or cost changes without executing new evaluations.

What appears in the evaluation report summary?▼

The summary covers verdict deltas, new, persisting, and resolved findings, token and cost changes when observable, and evidence paths. Raw transcripts are excluded from the human-readable summary.