autoresearch

Runs a metric-driven evolve-score-verify loop that optimizes prompts, code, or creative artifacts against a numeric target.

4|1|Updated Jul 31, 2025
One-click install
npx skills add https://github.com/alvis/.agents --skill autoresearch-alvis
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autoresearch
Source: https://github.com/alvis/.agents/tree/main/plugins/essential/skills/autoresearch
Command: npx skills add https://github.com/alvis/.agents --skill autoresearch-alvis

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Optimizing prompts, code, configs, or creative text by hand is slow and unmeasurable — you tweak, eyeball the result, and never know if you actually improved. This Skill turns a fuzzy optimization goal into an approved research brief with a concrete metric, baseline, and target, then runs an automated generate-score-verify-evolve loop until the target is met, the round budget is spent, or progress plateaus. ## Core Features & Use Cases - Metric-driven research loop: Define a metric, evaluator backend (programmatic command, LLM judge panel, or human scoring), baseline, and target, then evolve candidates across rounds with genetic mutation, recombination, and wildcard exploration. - Adversarial verification: Every round winner is attacked by a refuter checking constraint violations, metric gaming, harness bugs, and prompt injection before its score stands. - Auditable, resumable runs: Every round persists candidates, scores.yaml, verify.yaml, and round logs to disk, producing a final dossier, leaderboard, and best-artifact copy; runs resume via --resume. - Use Case: Optimize a prompt until accuracy reaches 0.90, get p95 latency under 200ms, or evolve taglines until a judge panel scores 8.5 — all with a full provenance trail. ## Quick Start Ask the agent to run autoresearch with the goal of optimizing your prompt until its accuracy on the eval set reaches 0.90.

Frequently Asked Questions about autoresearch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I optimize a prompt against a numeric metric automatically?▼

Provide an optimization goal and an eval command that prints exactly one number on stdout. The skill interviews you to build a research brief with metric, baseline, target, and budget, then runs generate-score-verify-evolve rounds until the target, budget, or plateau stops the run.

What eval backends does the autoresearch loop support?▼

Three backends: programmatic (a command printing one parseable number), judges (a panel of at least 3 independent LLM judges using an anchored rubric with median consensus), and human (you score candidates each round through a structured input tool).

Can I resume an interrupted autoresearch run?▼

Yes. Pass --resume=<run-dir> and the skill reconstructs round, survivors, best candidate, fanout, and leaderboard from the persisted rounds/ directory. Every round writes candidates, scores.yaml, verify.yaml, and a round log, so any run is resumable from disk alone.

How does the loop prevent candidates from gaming the metric?▼

The eval command, script, rubric, and fixtures are auto-listed as immutable paths; any candidate touching them is disqualified. An adversarial refuter also attacks each round winner for hardcoded eval outputs, test-set overfitting, harness bugs, and prompt injection before its score stands.

When should I use deep-research instead of autoresearch?▼

Use deep-research for literature reviews, web research, and fact-finding reports. Autoresearch is only for goals with a nameable, computable metric where candidate solutions can be evolved and scored against a numeric target.

What happens when the optimization run stops?▼

The loop stops on target met, budget exhausted, or plateau, then asks you to accept, extend the budget, or pivot the brief. On acceptance it synthesizes a dossier, rewrites the leaderboard, copies the winning artifact to best/, and a reviewer verifies every score traces to disk.