auto-research

What problem does it solve?

This Skill automates the process of iteratively improving code by running systematic experiments, keeping successful changes, and discarding failures, allowing for continuous optimization without constant human oversight.

Core Features & Use Cases

• Autonomous Experimentation: Runs experiments on codebases to optimize a specific metric.
• Iterative Improvement: Automatically keeps changes that improve the metric and reverts those that don't.
• Resource Management: Can utilize local or remote GPUs for parallel execution.
• Use Case: Imagine you have a machine learning model that needs its hyperparameters tuned. You can trigger this Skill to autonomously run hundreds of experiments overnight, exploring different learning rates, batch sizes, and architectures, and waking up to a significantly improved model.

Quick Start

Use the auto-research skill to autonomously experiment on the codebase to improve the val_bpb metric.