experiment

Runs iterative agent-driven loops that optimize a measurable code metric with keep-or-discard decisions.

111|73|Updated Mar 20, 2026
One-click install
npx skills add https://github.com/autopus-ai/autopus-adk --skill experiment-autopus-ai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: experiment
Source: https://github.com/autopus-ai/autopus-adk/tree/main/.omp/skills/experiment
Command: npx skills add https://github.com/autopus-ai/autopus-adk --skill experiment-autopus-ai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Manual performance tuning is slow and error-prone: developers guess at optimizations, lose track of what worked, and accumulate bloated changes. This Skill automates metric-driven experimentation so every code change is measured, and only genuine improvements are kept. ## Core Features & Use Cases - Automated Experiment Loop: Repeatedly measures a baseline metric, asks an executor agent for one focused change, re-measures, and commits or resets based on the result. - Circuit Breaker & Simplicity Gate: Stops the loop after N consecutive non-improvements and discards changes whose metric gain does not justify the added code complexity. - Crash & Scope Handling: Resets trivially broken builds without penalizing the approach, enforces allowed file scope, and shuts down gracefully on SIGINT with a summary. - Use Case: Optimize a Go benchmark by pointing the loop at pkg/foo/foo.go with a go test -bench metric command, letting it iterate up to 30 times and report the best result via auto experiment summary. ## Quick Start Initialize an experiment session with a metric command and target files, then let the agent run the optimization loop and show the summary of kept improvements.

Frequently Asked Questions about experiment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run an automated code optimization loop?▼

Initialize a session with `auto experiment init --session-id <id>` on a clean worktree, define a metric command that prints a parseable value, then let the executor agent iterate. Each iteration measures the metric and keeps or discards the change automatically.

How do I optimize a Go benchmark automatically?▼

Point the loop at a benchmark metric command such as `go test -bench=. ./...` piped through grep and awk to emit a JSON metric value, set the target file, and choose direction minimize. The loop repeats until max iterations or the circuit breaker trips.

What happens when an experiment change breaks the build?▼

Trivial crashes like compile errors or test panics trigger an immediate reset to the last kept commit and are recorded as status crash without counting against the circuit breaker. Three or more consecutive crashes on the same approach count as a circuit breaker failure.

Can the experiment loop modify files outside my target scope?▼

No. After each executor commit the loop validates changes against the allowed paths using git scope checking. Out-of-scope modifications are recorded as scope-violation, reset to the last keep hash, and do not count as circuit breaker failures.

When does the experiment loop stop early?▼

The loop stops when the circuit breaker trips after N consecutive non-improvements (default 10), when crashes prevent metric measurement for 5 or more consecutive iterations, or when the user presses Ctrl+C, which triggers a graceful shutdown with a summary.