claim-verify

Verifies each claim in a results draft against its source dataset through reproduction and stress tests.

Updated Sep 9, 2026
One-click install
npx skills add https://github.com/xlinh2301/EditCTC --skill claim-verify-xlinh2301
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: claim-verify
Source: https://github.com/xlinh2301/EditCTC/tree/main/.agents/skills/claim-verify
Command: npx skills add https://github.com/xlinh2301/EditCTC --skill claim-verify-xlinh2301

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Results drafts often contain numbers that reproduce but conclusions that do not hold up — causal claims undermined by confounds, striking rates built on tiny subgroups, correlations driven by single outliers. This Skill red-teams every discrete claim in a draft against the underlying dataset before publication, so fragile or wrong claims are hedged, scoped, or retracted instead of shipped. ## Core Features & Use Cases - Claim extraction and reproduction: Splits a prose draft into discrete checkable claims and recomputes each stated statistic directly from the source dataset with Python check snippets. - Adversarial stress-testing: Attacks each reproduced claim with the threats most likely to kill it — outlier sensitivity, Simpson's reversals, confound stratification, subgroup size, and alternative specifications. - Verdict-driven revision: Classifies each claim as verified, fragile, or refuted, then rewrites the draft so fragile claims are hedged or scoped and refuted claims are corrected, with a ledger recording every verdict. - Use Case: Before submitting an A/B test report claiming a treatment improves recovery by 16 points, run this loop to discover the effect reverses within age strata, and revise the draft to a descriptive claim with the causal language retracted. ## Quick Start Verify every claim in results_draft.md against trial_data.csv and produce a revised draft with each claim marked verified, hedged, or retracted.

Frequently Asked Questions about claim-verify

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify claims in a results report against the source data?▼

Extract each discrete checkable claim from the draft, recompute its exact statistic from the dataset with a Python snippet, then stress-test it against outliers, confounds, and subgroup size. Claims that reproduce and survive are verified; others are hedged or retracted.

What is the difference between a reproduced claim and a verified claim?▼

A reproduced claim merely has a number that recomputes correctly from the data. A verified claim also survives the threat most likely to kill it, such as a Simpson's reversal or an outlier driving the effect, so its interpretation is supported.

Does claim verification require pandas or numpy?▼

No. Check snippets are stdlib-first, using csv and statistics. If pandas or numpy would help, the snippet probes with try/except ImportError and degrades to a stdlib path, or offers a consented install.

When should I not use a claim verification loop?▼

Do not use it for open-ended discovery of new findings over a dataset, which is a data-analysis task, or for diagnosing a single known anomaly or pipeline failure. It is a gate over an existing draft of claims.

What happens to claims that fail verification?▼

Refuted claims are corrected with the right number or removed. Fragile claims are hedged, scoped, or retracted to what the data supports, and every verdict is recorded in a tab-separated ledger with its evidence.