autoresearch

Runs autonomous keep-or-revert experiment loops to optimize measurable metrics in code, prompts, and prose.

Updated Jul 28, 2026
One-click install
npx skills add https://github.com/christian-byrne/comfy-skills --skill autoresearch-christian-byrne
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/christian-byrne/comfy-skills/tree/main/skills/autoresearch
Command: npx skills add https://github.com/christian-byrne/comfy-skills --skill autoresearch-christian-byrne

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Manual optimization of code performance, prompts, or prose requires constant human attention and guesswork. This Skill automates the entire experiment cycle — editing, measuring, keeping improvements, and reverting regressions — so optimization runs autonomously for hours without supervision. ## Core Features & Use Cases - Autonomous Keep/Revert Loop: Edits in-scope files, commits each experiment, verifies against a metric command, and automatically reverts regressions via git. - Domain Routing: Ships reference guides for code performance, test speed, prompt optimization, ML training, code quality, agent harnesses, visual performance, and subjective prose via adversarial debate judging. - Fast vs Epoch-Based Loops: Distinguishes metrics measurable in seconds from metrics that accumulate over days of real-world agent runs, with protocols for each. - Use Case: Point it at a slow benchmark suite overnight — it runs dozens of atomic experiments, keeps the ones that reduce execution time, and hands you a clean branch of only winning commits. ## Quick Start Ask the agent to run autoresearch on your project with a goal, the files it may edit, a metric with direction, and a shell command that prints that metric.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run an autonomous optimization loop on my code?▼

Provide a goal, the files the agent may edit, a metric with direction, and a shell command that prints that metric. The loop then makes one atomic change per iteration, commits it, measures the metric, and keeps or reverts based on the result.

What metrics can autoresearch optimize?▼

Any metric extractable from a shell command: execution time, allocations, test duration, bundle size, lint warnings, coverage, validation loss, or eval success rate. For subjective prose, it manufactures a synthetic metric through adversarial debate and blind judge panels.

When should I use the epoch-based loop instead of the fast loop?▼

Use the epoch-based loop when the metric cannot be computed from a single shell command and depends on real-world agent runs accumulating over days. If a verify command produces a fresh score in under 60 seconds, the fast loop applies.

Does the optimization loop ever stop on its own?▼

By design it never stops until interrupted, since it is meant to run unattended overnight. The exception is prompt optimization, which exits automatically after scoring 95% or higher on three consecutive kept experiments.

What happens when the loop gets stuck with repeated failures?▼

After five consecutive discards it switches to a fundamentally different approach. After ten, it triggers the nuclear rewrite protocol: deleting the in-scope files and regenerating them from specification only, without reading the contaminated prior code.

Can prompt optimization loops cheat the evaluation?▼

Yes — known patterns include embedding expected outputs in the prompt, keyword stuffing for the judge, and refusing hard scenarios. The skill mandates anti-cheating rules in the mutation prompt and a diff-based detection heuristic that reverts suspicious edits.