skills.rest
yyccbb
yyccbbCOMMUNITY
Explore all skills by this author →
GitHub

result-analysis

Extracts reproduction metrics with auditable scripts and compares them against paper tables and figures.

Updated Aug 14, 2026
Search Tags:#machine learning#audit trail#matplotlib#tensorboard#metric extraction#result comparison#paper reproduction
One-click install
npx skills add https://github.com/yyccbb/PaperReproduction-Skills --skill result-analysis-yyccbb
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: result-analysis
Source: https://github.com/yyccbb/PaperReproduction-Skills/tree/main/skills/result-analysis
Command: npx skills add https://github.com/yyccbb/PaperReproduction-Skills --skill result-analysis-yyccbb

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires matplotlib, pandas, numpy, pymupdf, tbparse.

What problem does it solve? After running a machine learning paper reproduction, you need a trustworthy paper-vs-reproduction comparison where every number can be re-derived by a human auditor, without hand-transcribing values from logs or judging success prematurely. ## Core Features & Use Cases - Scripted Metric Extraction: Writes one rerunnable extraction script per experiment that reads results.json files, TensorBoard event files, console logs, or checkpoints, and outputs CSV/JSON data files. - Paper Presentation Regeneration: Rebuilds the paper's tables as paired-column markdown tables with deltas, and regenerates plots with matplotlib matching the paper's axes, alongside cropped images of the original figures from the PDF. - Evidence-Bound Reporting: Analyzes only runs marked COMPLETE by the upstream run-experiment stage, reports failed experiments as NOT ANALYZED with their upstream reason, and never edits code, installs packages, or issues a reproduction verdict. - Use Case: After stages 1-5 of a paper-reproduction pipeline finish, run this stage to produce result-analysis.md comparing each experiment's reproduced metrics against the paper's Table 1 or Figure 1 side by side. ## Quick Start Analyze the completed reproduction runs in this repo against the paper and write the result analysis report.

Frequently Asked Questions about result-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare reproduction results against a paper's tables?▼

Locate the paper's table in the PDF, extract the reproduced metrics from run artifacts with a saved script, and build a paired-column markdown table showing paper value, reproduced value, and delta per row. Every number must trace back to a rerunnable script and data file.

How to extract metrics from TensorBoard event files and training logs?▼

Prefer machine-readable artifacts like results.json first, then parse TensorBoard event files with tbparse, then parse console.log with explicit patterns, and only recompute from checkpoints as a last resort using the repo's own eval code. Write the extraction as a saved script that outputs CSV or JSON.

Can I read data points off a paper's figure for comparison?▼

No. Digitizing values off a plotted curve is treated as fabrication. Paper numbers may only come from table cells or values stated in text; for figures, crop the paper's figure from the PDF and present it side by side with your regenerated plot instead.

What happens when an experiment failed or never ran upstream?▼

The experiment is reported as NOT ANALYZED with the upstream verdict quoted verbatim, and no partial numbers are scraped from dead runs. The gap routes back to the earlier pipeline stage rather than being fixed or re-run during analysis.

Does the analysis stage modify the codebase or environment?▼

No. The codebase and conda environment are read-only: no commits, no package installs, no re-runs. All outputs live under .paper-reproduction/analysis/ plus the report file, and missing packages are recorded as blockers for the environment-setup stage.

Related Skills

Explore More Education & Research Skills→
gyy0592gyy0592
7

gen-report-concise

Generate concise experiment reports from training run artifacts and logs.

Community
Advanced
AlanxtlAlanxtl
1

reproduce-papers

Plan and execute reproducible computer science paper reproductions with artifact inventory and gap tracking.

Community
Advanced
AmirTlinovAmirTlinov

compas

Validate code quality and enforce standards across repositories with configurable checks.

Community
Advanced
wildwasserwildwasser
35

ml-experiment

Track machine learning experiments with structured logging and reproducibility.

Community
Advanced