paper-evaluator

Evaluates research papers against international reporting guidelines and produces Markdown assessment reports.

Updated Feb 12, 2026
One-click install
npx skills add https://github.com/qmakescl/QSkills --skill paper-evaluator-qmakescl
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: paper-evaluator
Source: https://github.com/qmakescl/QSkills/tree/main/skills/paper-evaluator
Command: npx skills add https://github.com/qmakescl/QSkills --skill paper-evaluator-qmakescl

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pdfplumber, pypdf, python-docx, and includes references (resource) components.

What problem does it solve? Manually checking a research paper against reporting guidelines like PRISMA, CONSORT, or STROBE is slow and error-prone. This Skill automates the full appraisal pipeline: it identifies the study design from the abstract, applies the correct international reporting and risk-of-bias guidelines, and outputs a structured Markdown evaluation report. ## Core Features & Use Cases - Automatic study design identification: Parses the abstract with a keyword decision tree to classify the paper as SR/meta-analysis, RCT, cohort, case-control, cross-sectional, diagnostic accuracy, or case report, then confirms with the user. - Guideline-based item appraisal: Lazy-loads only the relevant guideline files (PRISMA 2020, CONSORT 2010, STROBE, STARD, CARE, AMSTAR 2, RoB 2.0, ROBINS-I, QUADAS-2, NOS) and rates each item as Yes / Partial Yes / No / N/A with evidence and improvement suggestions. - GRADE certainty grading: For systematic reviews and meta-analyses, rates the certainty of evidence per outcome across the five downgrade domains and three upgrade criteria. - Use Case: Before submitting a manuscript, ask the agent to evaluate your RCT draft; it identifies the design, checks all 25 CONSORT items plus RoB 2.0 domains, and returns a compliance report with prioritized revision recommendations. ## Quick Start Evaluate the attached paper 'my-manuscript.pdf' against the appropriate reporting guidelines and generate a full assessment report.

Frequently Asked Questions about paper-evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if my paper complies with PRISMA or CONSORT guidelines?▼

Provide the paper file and the skill identifies the study design from the abstract, loads the matching guideline checklist, and rates each item as Yes, Partial Yes, No, or N/A with evidence and improvement suggestions. It outputs a Markdown compliance report with adherence percentages.

What reporting guidelines does this paper evaluation support?▼

It supports PRISMA 2020, PRISMA-P, PRISMA-ScR, CONSORT 2010, STROBE, STARD 2015, and CARE 2013 for reporting, plus AMSTAR 2, RoB 2.0, ROBINS-I, ROBIS, QUADAS-2, and Newcastle-Ottawa Scale for quality and risk-of-bias assessment.

Can it evaluate PDF and Word manuscripts?▼

Yes, it extracts text from PDF using pdfplumber or pypdf and from DOCX using python-docx. Scanned PDFs without a text layer require OCR via pytesseract before evaluation.

Does it perform GRADE certainty of evidence grading?▼

GRADE grading runs only for systematic reviews and meta-analyses. It rates each outcome starting from High for RCT-based or Low for observational evidence, then applies the five downgrade domains and three upgrade criteria to reach a final rating.

What happens if the study design cannot be identified from the abstract?▼

The skill falls back to analyzing the title, introduction, and methods sections. If the design remains unclear, it presents the detected clues and asks the user to select the correct design from a list before proceeding.