skills.rest
adriancodes
adriancodesCOMMUNITY
Explore all skills by this author →
GitHub

ecc-rag-pipeline-reviewer

Reviews RAG retrieval pipelines for context relevance, citations, tenant isolation, and evaluation gaps.

Updated Apr 18, 2025
Search Tags:#rag#evaluation#retrieval#citations#vector store#ragas#pipeline review
One-click install
npx skills add https://github.com/adriancodes/dotfiles --skill ecc-rag-pipeline-reviewer-adriancodes
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ecc-rag-pipeline-reviewer
Source: https://github.com/adriancodes/dotfiles/tree/main/dot_agents/skills/ecc-rag-pipeline-reviewer
Command: npx skills add https://github.com/adriancodes/dotfiles --skill ecc-rag-pipeline-reviewer-adriancodes

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Retrieval-augmented generation pipelines often ship without verified context relevance, citation grounding, tenant isolation, or evaluation baselines, leading to ungrounded answers and silent regressions. This Skill provides a structured, read-only review of a RAG pipeline and returns an evidence-backed decision report. ## Core Features & Use Cases - Retrieval Inspection: Traces vector store, embedding model, chunking strategy, top-k selection, tenant filters, deduplication, reranking, and token-budget truncation in supplied code and outputs. - Evaluation Verification: Checks for RAGAS-or-equivalent metrics (faithfulness, context_recall, context_precision), versioned baselines, acceptance thresholds, query slices, and regression gates. - Citation and Fallback Review: Flags claims attributed to unverified text and missing insufficient-context fallback behavior. - Use Case: Given a ChromaDB + Ollama pipeline with no evaluation results, the Skill reports the evidence gap, explains that missing reranking alone is not a defect, and recommends establishing baseline metrics before production readiness. ## Quick Start Review my RAG pipeline code and retrieval outputs for context relevance, citation grounding, tenant isolation, and evaluation coverage, then return an approve or block decision report.

Frequently Asked Questions about ecc-rag-pipeline-reviewer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I review a RAG pipeline for retrieval quality?▼

Trace the retrieval path from vector store and embedding model through chunking, top-k selection, tenant filters, deduplication, and reranking. Then verify evaluation coverage with faithfulness, context_recall, and context_precision metrics against a versioned baseline before approving the pipeline.

What RAGAS metrics should a RAG pipeline evaluation include?▼

The minimum metric set is faithfulness, context_recall, and context_precision. There is no universal near-1.0 threshold; the project must define its own baseline scores, acceptance thresholds, query slices, and allowed regression deltas appropriate to its risk level.

Is a missing reranker a defect in a retrieval pipeline?▼

No, a reranker is optional and its absence alone does not establish a defect. Compare retrieval metrics across representative query slices first, then use error evidence to decide whether filtering, deduplication, hybrid retrieval, or reranking is warranted.

Can this review execute evaluations or call vector stores?▼

No, the review is strictly read-only over supplied source code and evidence. It does not run evaluations, install packages, call embedding or model APIs, or query a vector store; missing results are reported as evidence gaps.

Why does a RAG system need an insufficient-context fallback?▼

Without a fallback, the system answers even when retrieval returns poor or empty results, producing ungrounded responses. The pipeline should signal insufficient grounding, such as requesting more documents, instead of answering anyway.

Related Skills

Explore More Software Engineering Skills→
daffy0208daffy0208
34

RAG Implementer

Implement Retrieval-Augmented Generation pipelines with vector stores and hybrid search.

Community
Advanced
Ankh-StudioAnkh-Studio

regression

Run automated regression tests comparing evaluations against baseline metrics.

Official
Advanced
dhntdhnt

performance-review

Review code diffs for hot-path complexity, allocations, and N+1 patterns.

Official
Intermediate
ArMaTeCArMaTeC

rag-architect

Designs RAG systems with chunking, embeddings, hybrid search, reranking, and retrieval evaluation.

Community
Advanced