What problem does it solve? When running multiple related LLM research projects (e.g., baseline vs. improved variants), results live in separate .research/<slug>/ directories, making rigorous cross-project comparison manual and error-prone. This Skill aggregates succeeded run metrics and produces a statistically sound comparison report. ## Core Features & Use Cases - Metric Aggregation: Collects primary metric values from 06_RUNS/*/metrics.json across two or more projects, excluding failed runs, into a reproducible raw.json. - Statistical Testing: Runs paired bootstrap (B=10000) or Welch's t-test per project pair, with Cohen's d, Cliff's delta effect sizes, and Holm-Bonferroni multiple-comparison correction. - Report & Figures: Generates REPORT.md with ranking tables and pairwise results, plus boxplot and effect-size PDF figures via matplotlib/seaborn. - Use Case: Compare an MMLU baseline project against a chain-of-thought variant to determine whether the accuracy improvement is statistically significant before writing it up in a paper. ## Quick Start Use the research.cross.compare skill to compare the projects llm-eval-mmlu-baseline and llm-eval-mmlu-cot on their primary metric with statistical tests.
