orc-retro

Aggregates ORC behavior traces into calibration reports filed as upstream GitHub PRs.

6|1|Updated Jul 4, 2026
One-click install
npx skills add https://github.com/azure-id/orc --skill orc-retro-azure-id
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: orc-retro
Source: https://github.com/azure-id/orc/tree/main/templates/skills/orc-retro
Command: npx skills add https://github.com/azure-id/orc --skill orc-retro-azure-id

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? ORC runs leave permanent behavior traces on disk, but raw trace files do not answer whether the scoring rubric is calibrated, whether workers ran at their assigned model tiers, or where the pipeline leaks. This Skill mines those traces and turns them into an actionable calibration report delivered upstream to the ORC repository. ## Core Features & Use Cases - Trace aggregation: Parses the closed verb set (SCORE, OUTCOME, VERIFY, GATE, FINDING, VERDICT) across run traces, preferring structured .jsonl sidecars, and aggregates per-band retries, requeues, downgrades, and gate bounce rates. - Read-only, report-only operation: Never edits the rubric, skills, config, or project code; it dispatches mining to a cheap subagent and validates the return against a strict contract. - Upstream delivery: Files the AI-readable report (YAML frontmatter plus human sections) as a PR, with issue fallback, to the configured retro_repo via the gh CLI or a GitHub MCP; a P0 preflight stops the run entirely if no delivery channel exists. - Use Case: After a week of ORC runs, ask how well the scoring is calibrated. The Skill mines all traces, reports per-band outcomes and two silent model downgrades, and opens a PR titled orc-retro: 130726 — 3 runs, 2 recommendations against the ORC repo. ## Quick Start Run /orc-retro to analyze the logged ORC traces and file a calibration report to the ORC repository.

Frequently Asked Questions about orc-retro

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze ORC behavior traces from past runs?▼

Run /orc-retro after ORC runs have written traces to the configured log_dir. The Skill collects the .txt traces and .jsonl sidecars, dispatches a mining subagent, and returns per-band statistics, downgrades, and recommendations with evidence line references.

How does orc-retro deliver its calibration report?▼

The report is filed upstream to the configured retro_repo (default azure-id/orc) as a pull request, with issue creation as fallback. Delivery uses the gh CLI when authenticated, or a GitHub MCP server; if neither channel exists, the retro stops before mining anything.

Does orc-retro modify my rubric, config, or project code?▼

No. The Skill is strictly read-only and report-only against the local system. It never edits effort-and-mode.md, skills, config, or code; recommendations are phrased for a human or the ORC repo maintainer to apply.

Why does orc-retro stop before mining any traces?▼

The P0 preflight requires a delivery channel: an authenticated gh CLI or a GitHub MCP server. If neither exists, the run stops immediately and tells you to run gh auth login or connect a GitHub MCP, since an undeliverable report is pointless.

What happens when there are too few runs to draw conclusions?▼

Every aggregate states its sample size n, and any recommendation based on fewer than three runs is labeled weak signal rather than stated as a conclusion. With zero traces, the Skill reports that no runs have been traced and stops without inventing findings.

How are foreign non-Claude dispatches counted in the retro?▼

Foreign dispatches appear as EXTRA lines with no SPAWN or RETURN, so the Skill runs orc extra stats --json instead of parsing them. It reports outcome mix, substitution and reroute counts per profile per band, and never treats a missing RETURN as a leak.