experiment-report

Renders single-page HTML experiment reports from manifests, logs, W&B data, and Prometheus evidence.

Updated Jun 6, 2026
One-click install
npx skills add https://github.com/Kirrito-k423/AutoResearch --skill experiment-report-kirrito-k423
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: experiment-report
Source: https://github.com/Kirrito-k423/AutoResearch/tree/main/.agents/skills/08-experiment-report
Command: npx skills add https://github.com/Kirrito-k423/AutoResearch --skill experiment-report-kirrito-k423

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? It turns scattered experiment artifacts—manifests, logs, W&B runs, and Prometheus metrics—into a single readable Chinese HTML report, so teams can review results, diagnose missing metrics, and explain zero scores without manually piecing together evidence. ## Core Features & Use Cases - Three-view report rendering: Combines log event timelines, W&B curves/tables, and Prometheus resource curves into one HTML page. - Formal Verl case diagnostics: Shows trainer_val_only status, validation matrix completeness, sequence-length effects, sync/async comparison, and accuracy/consistency results. - Missing-data diagnosis: Explains zero scores (e.g., missing \boxed{} output or extraction failure) and distinguishes pushed vs. missing Prometheus metrics instead of falsely concluding no data exists. - Use Case: After running a smoke experiment on a remote NPU server, generate a localized Chinese report that links W&B runs, lists deliverables, and flags incomplete validation matrix rows for review. ## Quick Start Ask the assistant to render the experiment report for run id 2026-06-06-smoke-001 and open the resulting HTML page.

Frequently Asked Questions about experiment-report

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate an HTML experiment report from W&B and Prometheus data?▼

Run the render_report.py script with a run id, for example python3 .agents/skills/08-experiment-report/scripts/render_report.py --run-id 2026-06-06-smoke-001 --open. It reads the manifest, logs, W&B data, and Prometheus evidence to build a single-page HTML report.

How to diagnose zero scores in a Verl validation matrix report?▼

Check the rows/*/validation/0.jsonl files referenced in the report for output, gts, and acc fields. A strict reward may record 0 when the model output lacks a \\boxed{} answer or answer extraction fails.

Does the report work when Prometheus or W&B services are unavailable?▼

Yes. The report states its data sources up front and resolves deliverable paths from the local data bundle first. When live services are unavailable, it reports pushed versus missing metrics instead of concluding the experiment has no data.

Why does my validation matrix look like GRPO training in the report?▼

When trainer_val_only=True, the report explicitly labels the run as a validation matrix that does not update model parameters. This prevents misreading validation-only runs as training runs.

What are the limitations of the experiment report skill?▼

It only renders and diagnoses reports; it does not collect metrics, run test cases, modify configurations, or change training script behavior. Diagnostics shown in the report are not new experimental conclusions.