What problem does it solve? Choosing which hub orchestrator model and reasoning-effort setting to ship is a high-stakes decision that ad-hoc testing cannot answer. This Skill runs the existing OBaI E2E regression gate once per candidate model:effort combo under a provably identical source tree, then ranks the reviewed outcomes on quality, dollar cost, and latency so the comparison is evidence-based rather than anecdotal. ## Core Features & Use Cases - Controlled multi-combo execution: Pins each hub model/effort pair via environment injection, scrubs inherited overrides, snapshots a source digest per run, and stops the session on the first failed combo. - Blind review and cross-combo audit: Reviews each run independently, then audits drafts across combos to surface disagreements before finalizing immutable reviewed results. - Fairness-gated ranking report: Refuses to rank runs whose suite fingerprint, cases snapshot, or source digest differ, applies a guardrail-failure disqualifier and a podium rule, and prices every span from a YAML rate table. - Use Case: When deciding whether to switch the hub default from gpt-5.6-terra:max to gpt-5.6-sol:high, run a smoke pass across candidates, then a core benchmark on the top two plus the incumbent, and read the scoreboard before proposing any settings change. ## Quick Start Ask the assistant to benchmark gpt-5.6-sol:high against gpt-5.6-terra:high on the core tier, starting with the offline dry run to confirm the plan and disclosed cost before authorizing paid execution.
