retro-analysis

Computes guardrail effectiveness and detects recurring failure patterns from pipeline event logs.

Updated May 18, 2026
One-click install
npx skills add https://github.com/FredoAi/fredo --skill retro-analysis-fredoai
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: retro-analysis
Source: https://github.com/FredoAi/fredo/tree/main/.opencode/skills/retro-analysis
Command: npx skills add https://github.com/FredoAi/fredo --skill retro-analysis-fredoai

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Self-improving agent pipelines accumulate guardrails and lessons, but without structured retrospective analysis there is no way to know whether a guardrail actually reduced its target failure, whether a rule was ever baked into a playbook, or when a recurring failure class deserves escalation to a deterministic check. ## Core Features & Use Cases - Guardrail Effectiveness Computation: Compares failure event rates before and after a guardrail's activation date using per-issue JSONL event logs and state-machine metrics, classifying results as Confirmed, Partial, Pending, or Ineffective. - Cross-Spec Pattern Detection: Groups failures across multiple specs by class to surface unguarded recurring patterns and prioritize new guardrail proposals. - Grounded Verification & ACE Curation: Verifies guardrail rules exist verbatim in agent playbooks, then applies an archive/escalate/strengthen decision tree following the ACE Generation-Reflection-Curation lifecycle. - Use Case: After an audit verdict, the Self-Improver agent runs these recipes to persist a new guardrail record in references.md, verify prior guardrails still work, and escalate any rule that was ignored despite being documented. ## Quick Start Ask the Self-Improver agent to run the retro-analysis recipes against the current pipeline event logs and report guardrail effectiveness with any unguarded failure patterns.

Frequently Asked Questions about retro-analysis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I measure whether a guardrail reduced failures in an agent pipeline?▼

Count failure events matching the guardrail's target class before and after its activation date in the per-issue JSONL event logs, then compare rates. Zero occurrences after activation across at least two issues classifies the guardrail as Confirmed.

How to detect recurring failure patterns across multiple specs?▼

Group failure events from all per-issue event logs by failure class derived from failed audit verdicts, rework transitions, and blocked outcomes. Any class with two or more occurrences and no matching active guardrail is flagged as an unguarded pattern.

What data sources does the retro-analysis skill read?▼

It reads per-issue JSONL event logs under .opencode/state/issues, state-machine metrics from pipeline-state.rs, script-errors.jsonl, and spec issue comments via gh issue view. It never queries fredo.db product telemetry.

When should a guardrail be escalated to a deterministic hook?▼

Escalate when a guardrail is classified Ineffective and grounded verification shows the rule exists in the agent's playbook but the failure recurred anyway. This means the agent saw the rule and ignored it, so a script or CI check is needed.

Can guardrail records contain code snippets?▼

No. Guardrail records must describe the rule in prose only, with no code snippets, API signatures, or symbol references. This keeps the ledger readable by any agent and avoids stale code embedded in documentation.