meta-harness

Define a clean candidate interface for the harness component, build a deterministic scorer with quality and cost axes, then run the propose-score-frontier loop. Proposer agents write candidate implementations each round and a Pareto frontier tracks quality versus cost tradeoffs.

It occurs when candidates are scored by replaying cached outputs, so a new candidate cannot change the recorded quality result and only the cost axis moves. The search then wins by emptying context while quality appears constant. Fix it by grading something the candidate controls, like retrieval relevance or compression fidelity.

Use it when the base model is fixed, such as an off-the-shelf API, and gains must come from the code around the model. It is complementary to RL: it hardens the eval and reward at low cost and produces a leaner harness as a starting point for later training.

You supply five pieces: a candidate interface, a $0 deterministic scorer with a rubric, an eval corpus with a held-out test split, a proposer prior steering toward mechanism-level changes, and a frontier plus run log. Templates for all five ship in the assets directory.

The usual causes are a scorer whose quality number cannot vary with the candidate, a leaky eval split, or a soft metric the proposer exploits. Add a hard quality floor, verify the scorer changes when candidates change, and validate proxy rankings against the true metric before trusting results.

Yes, the loop can be driven by CronCreate for scheduled cloud runs, /loop with an interval for in-session cadence, or ScheduleWakeup for self-paced continuation. The Workflow mode also supports resume via resumeFromRunId for long searches.