ara-rigor-reviewer

Reviews Agent-Native Research Artifacts across six epistemic dimensions and writes a severity-ranked level2_report.json.

13.0k|930|Updated Nov 3, 2025
One-click install
npx skills add https://github.com/Orchestra-Research/AI-research-SKILLs --skill ara-rigor-reviewer-orchestra-research
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ara-rigor-reviewer
Source: https://github.com/Orchestra-Research/AI-research-SKILLs/tree/main/22-agent-native-research-artifact/rigor-reviewer
Command: npx skills add https://github.com/Orchestra-Research/AI-research-SKILLs --skill ara-rigor-reviewer-orchestra-research

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Research artifacts need an objective epistemic critique before publication, but manually checking whether evidence truly supports claims, whether falsification criteria are meaningful, and whether the research process is honestly documented is slow and inconsistent. ## Core Features & Use Cases - Six-Dimension Scoring: Scores evidence relevance, falsifiability, scope calibration, argument coherence, exploration integrity, and methodological rigor on a 1-5 scale with defined anchors. - Severity-Ranked Findings: Produces constructive findings (critical, major, minor, suggestion) with verbatim evidence spans, reasoning, and actionable fixes. - Calibrated Grade: Computes a Strong Accept-to-Reject recommendation from dimension means and writes a structured level2_report.json to the artifact root. - Use Case: After Level 1 structural validation passes on an ARA directory, run this review to get an objective peer-review-style critique with questions for authors before releasing the artifact. ## Quick Start Run the ARA Seal Level 2 semantic epistemic review on the artifact directory at ./my-research-artifact and write level2_report.json to its root.

Frequently Asked Questions about ara-rigor-reviewer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I review an Agent-Native Research Artifact for epistemic rigor?▼

Run the Level 2 review after Level 1 structural validation passes. It reads PAPER.md, logic files, the exploration tree, and evidence in a fixed order, scores six dimensions from 1-5, and writes level2_report.json with findings and an overall grade.

What dimensions does the ARA Level 2 review evaluate?▼

It evaluates six dimensions: evidence relevance, falsifiability quality, scope calibration, argument coherence, exploration integrity, and methodological rigor. Each dimension has defined scoring anchors and produces strengths, weaknesses, and suggestions.

Does the Level 2 review check structural validation or references?▼

No. Level 2 assumes Level 1 structural validation already passed and does not re-check reference resolution, field presence, YAML parsing, or cross-link consistency. It focuses entirely on whether the content is epistemically sound.

How is the overall ARA review grade calculated?▼

The grade is computed from the mean of the six dimension scores. Strong Accept requires a mean of at least 4.5 with no dimension below 3, while Reject applies when the mean is below 2.0 or any dimension scores 1.

Can the reviewer fetch external sources or run experiments?▼

No. The review is artifact-only: it does not execute code, fetch URLs, or consult external sources. It takes the ARA's reported evidence at face value and evaluates it using only Read, Write, Glob, and Grep operations.