hallucination-investigator

Diagnose root causes of fabricated AI outputs across context, retrieval, prompting, and tool use.

Updated Jul 12, 2026
One-click install
npx skills add https://github.com/sumitake/agent-collab --skill hallucination-investigator-sumitake
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: hallucination-investigator
Source: https://github.com/sumitake/agent-collab/tree/main/plugins/agent-collab/skills/hallucination-investigator
Command: npx skills add https://github.com/sumitake/agent-collab --skill hallucination-investigator-sumitake

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? When an AI system produces a wrong or fabricated answer, teams often guess at fixes or rewrite prompts blindly. This Skill reconstructs exactly what evidence the system had when it failed, pinpoints where the failure originated, and recommends the smallest fix that addresses the actual cause. ## Core Features & Use Cases - Failure Reconstruction: Rebuilds the exact input, available context or retrieved documents, and the bad output to establish an evidence trail. - Root Cause Classification: Distinguishes no-evidence failures from evidence-ignored failures, and rules out retrieval misses, stale data, tool errors, and prompt framing before blaming model overreach. - Targeted Remediation: Recommends the highest-leverage fix for the identified origin point plus a verification case that would catch a recurrence. - Use Case: A RAG chatbot cites a policy that does not exist. Use this Skill to determine whether the retrieval pipeline missed the right chunk, the prompt rewarded confident completion, or the model inferred beyond the retrieved evidence. ## Quick Start Ask the agent to investigate why the system produced a specific wrong answer, providing the failing input, the output, and any available context or logs.

Frequently Asked Questions about hallucination-investigator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I investigate why an AI model hallucinated an answer?▼

Reconstruct the failing case fully: the exact input, the context or retrieved documents available, and the output produced. Then determine whether the failure came from missing evidence, ignored evidence, retrieval problems, prompt framing, or tool misuse before choosing a fix.

How to tell the difference between a hallucination and a retrieval failure?▼

Check whether the correct evidence was present in the context the system actually received. If the evidence was absent or stale, it is a retrieval or data problem; if the evidence was present but ignored or exceeded, it is a genuine model overreach.

Can prompt wording cause an AI to fabricate answers?▼

Yes. Instructions or examples that reward confident, complete-sounding answers can suppress honest uncertainty responses. The investigation examines prompt framing as one of several candidate root causes before recommending changes.

When should I not blame the model for a wrong answer?▼

Do not attribute failure to the model when retrieval missed relevant content, source data was stale or wrong, a tool result was misread, or a tool failure was silently treated as a null result. These causes require different fixes than prompt or model changes.

What does a hallucination investigation report include?▼

The report includes a reconstruction of the failure, the most likely root cause with alternatives ruled out, the highest-leverage fix, detection or guardrail ideas, at least one targeted verification case, and the residual risk remaining after the fix.