Optimize

Invoke metric mode with a metric name, a shell command that produces the measurement, and the file globs the agent may modify. The loop hypothesizes changes, measures each experiment, keeps improvements, and reverts regressions until gains plateau or a target value is reached.

Use eval mode by passing a target path to a skill directory, prompt file, or agent definition. The system auto-generates 3-6 binary eval criteria and 3-5 test inputs for your approval, then runs the optimization loop scored by LLM-as-judge across multiple runs per experiment.

Metric mode optimizes quantifiable targets like latency or bundle size using a shell command that emits a number, running about 12 experiments per hour. Eval mode handles qualitative targets like prompt quality using LLM-as-judge scoring, which is slower at roughly 6-8 experiments per hour.

Yes, use the --locked argument with a glob pattern to mark files the agent must not modify, and --constraints to add rules like requiring all tests to pass. Guard rail violations trigger an automatic revert regardless of score improvement.

Hill-climbing can get stuck in local optima, so if the score plateaus you may need to reset with different initial conditions. Regression tolerance should not be set to zero, since minor secondary regressions are acceptable when the primary metric improves significantly.