evaluate-results

Evaluates Chronicle experiment runs against pre-registered hypotheses using real metrics.

1|1|Updated May 9, 2026
One-click install
npx skills add https://github.com/methodic-research/skills --skill evaluate-results-methodic-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluate-results
Source: https://github.com/methodic-research/skills/tree/main/research-plugin/skills/evaluate-results
Command: npx skills add https://github.com/methodic-research/skills --skill evaluate-results-methodic-research

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? After an experiment's variations and runs finish, someone must judge what actually worked against what was pre-registered — pulling real metrics from W&B, execution logs, and attached reports rather than guessing from run statuses. This Skill performs that judgment in-context and optionally persists findings, lessons, and reports to the Chronicle record. ## Core Features & Use Cases - Metrics-grounded judgment: Enumerates variations and runs, pulls real numbers via chronicle.wandb_* mediation tools, execution_log assets, and attached reports, then classifies each variation as working, partial, or not_working against its pre-registered hypothesis. - Read-only first-class mode: "Just tell me what the results say" ends at the presented read; persisting findings, lessons, or reports happens only on explicit request. - Record-checked evaluation: Optionally validates claims via chronicle.evaluation_support before persisting, surfacing verdicts, assumptions, and matched failure modes. - Use Case: After an lr-sweep experiment completes, ask the agent to evaluate the runs — it produces a per-variation table of what worked, what didn't, and what's unexplained, then records findings and a review-gated takeaways report if requested. ## Quick Start Ask the agent to evaluate the results of experiment X and tell you what worked and what didn't against each variation's pre-registered hypothesis.

Frequently Asked Questions about evaluate-results

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate experiment results against pre-registered hypotheses?▼

Invoke this skill with an experiment ID; it enumerates variations and runs, pulls real metrics from W&B and execution logs, and classifies each variation as working, partial, or not_working against its pre-registered hypothesis.

Can I read experiment results without writing anything back?▼

Yes, read-only evaluation is the default. The persist parameter stays at none unless you explicitly request findings, lessons, or a durable report to be recorded.

Does this skill work without the chronicle plugin installed?▼

No, it requires the chronicle plugin's MCP tools and the chronicle-distill and chronicle-write-report skills. If they are unavailable, it stops and tells you to install both plugins from the methodic marketplace.

What happens when a variation has no metrics or no W&B run?▼

The skill records the absence as a finding itself, such as "no linked W&B run", and continues with the remaining variations. It never invents numbers to fill the table.

How are negative or crashed runs handled in the evaluation?▼

Judgment is based on metrics, not run status: a clean run can still refute its hypothesis, and a crash can still be informative. Negative results are called out explicitly as a core part of the read.

What if the evaluation_support check is unavailable on the server?▼

It is non-fatal. The skill states that the read was not checked against the record and continues, never presenting an unchecked read as a checked one.