scientific-critical-thinking

Evaluate study design appropriateness, internal validity (randomization, blinding, confounding control), external validity (sample representativeness), and measurement quality. The Skill provides structured checklists covering each validity type with specific questions to ask.

GRADE rates evidence as high, moderate, low, or very low quality. RCTs start as high and observational studies as low, then are downgraded for risk of bias, inconsistency, indirectness, imprecision, or publication bias, and upgraded for large effects or dose-response relationships.

Check for preregistration and compare registered outcomes to published ones, look for suspicious p-value clustering just below .05, and watch for undisclosed subgroup analyses. The Skill's bias taxonomy covers detection and mitigation strategies for these analysis biases.

Use Cochrane ROB for randomized trials, ROBINS-I or Newcastle-Ottawa for observational studies, QUADAS-2 for diagnostic accuracy studies, and AMSTAR-2 for systematic reviews. The references detail when each tool applies.

It supports both. Beyond critique, it includes a research design guidance workflow covering question refinement, power analysis, bias minimization, measurement strategy, prespecified analysis plans, and preregistration for planning rigorous new studies.

Checklists guide judgment but cannot replace domain expertise; a well-conducted observational study can outweigh a flawed RCT. The Skill emphasizes proportionate critique, acknowledging uncertainty, and distinguishing fatal flaws from minor limitations.