Design Experiment

What problem does it solve?

Manually planning complex LLM fine-tuning and evaluation experiments is time-consuming, error-prone, and difficult to document. This Skill automates the entire planning process, ensuring consistency, verifying resources, and providing clear documentation, so you can focus on the science.

Core Features & Use Cases

• Automated Experiment Planning: Guides you through defining variables, resources, and evaluation strategies for LLM fine-tuning.
• Resource Verification: Automatically checks for the existence of models, datasets, and evaluation scripts on your system.
• Compute Estimation: Provides accurate estimates for training time, evaluation time, and disk space, often based on prior runs.
• Self-Documenting Plan: Generates a comprehensive experiment_summary.md with all configurations and status tracking, ready for review.
• Use Case: Before starting a new research project comparing different LoRA ranks for a specific LLM, use this skill to generate a detailed plan, verify all necessary files are in place, and get accurate time/cost estimates, all documented automatically.

Quick Start

To start designing your experiment (this skill is interactive):

claude design-experiment

Follow the prompts to define your experiment's parameters.

A detailed experiment_summary.md will be created upon completion,

outlining your entire research plan.