Skills
.Work. You
Rest

plan-mode-official-leaderboard-abctest

Community

Run official A/B/C benchmarks with stats.

Data & Analytics#benchmark#t-test#statistical-analysis#model-comparison#leaderboard#abctest#evaluation-harness
Authorzapabob
Version1.0.0
Installs0

System Documentation

What problem does it solve?

This Plan-mode skill executes official leaderboard-compliant A/B/C tests to compare Phi-3.5-mini-instruct, Borea-phi3.5-instinct-jp, and AEGIS-Phi3.5mini-jpv2.4 on GSM8K, MATH, and ARC-Challenge, providing statistically validated results.

Core Features & Use Cases

  • Official benchmark evaluation across GSM8K, MATH, and ARC-Challenge for three models.
  • Statistical analysis including t-tests, effect sizes (Cohen's d), confidence intervals, and multiple-comparison corrections.
  • Result aggregation and leaderboard-style reporting for reproducible comparisons.
  • SO8T workflow integration and automated reporting.

Quick Start

Configure the test with your models and benchmarks, then run the OfficialABCTestPlan to generate a statistically validated leaderboard.

Dependency Matrix

Required Modules

None required

Components

Standard package

💻 Claude Code Installation

Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.

Please help me install this Skill:
Name: plan-mode-official-leaderboard-abctest
Download link: https://github.com/zapabob/SO8T/archive/main.zip#plan-mode-official-leaderboard-abctest

Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
View Source Repository

Agent Skills Search Helper

Install a tiny helper to your Agent, search and equip skill from 223,000+ vetted skills library on demand.