Research Guides: Physical Therapy: Critical Appraisal

Step 1: Understanding the Source

To appraise evidence quality, you first need to understand the source of the evidence.

What type of source is this? (See the University of Minnesota Library's page on Primary, Secondary, and Tertiary Sources in the Health Sciences.)
What is the purpose of the source (or the research the source is presenting)?
How current is the source? When was it published? Is the evidence presented still applicable? Have clinical practices changes? Has consensus understanding of a disease, condition, or treatment changed?
Who is the author of the source? What are the author's credentials and qualifications? Was the source peer reviewed or edited?
Who is the publisher of the source? A scholarly journal, newspaper, magazine, association, organization, or some other entity?

Step 2: Assessing the Study Methodology

Understanding how a study was conducted (the methodology) can help you determine the quality of the evidence being presented. Here are some tips for determining study methodology:

Scan the research article to find the study or publication type (e.g., randomized controlled trial, case report, observational study, review article, etc.). Note that a review article is not necessarily a "systematic review."
Read the methods section of the article to determine how the study was conducted. The following resources provide guidance for understanding study methodology.

Understanding Research Study Designs (University of Minnesota Library)
Study Designs (Centre for Evidence Based Medicine)
Quantitative Study Designs (Deakin University Library

Step 3: Doing the Critical Appraisal

Below are critical appraisal tools and checklists, as well as reporting guidelines for different types of studies.

AGREE II Instrument
Used to assess practice guidelines

For help using the AGREE II Instrument, see AGREE II Training Tools.

AMSTAR II
Used to appraise systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both
Appendix 1: Checklists for Finding, Appraising, and Implementing Evidence (Found in Appendix 1 of How to Read a Paper: The Basics of Evidence Based Medicine, 2010, 4th edition, by Trisha Greenhalgh)
A collection of checklists for a number of purposes related to EBM, including finding, interpreting, and evaluating research evidence
CASP International Network (CASPin) Checklists
CASP is the Critical Appraisal Skills Programme. They offer checklists for systematic reviews, randomized controlled trials, qualitative research studies, economic evaluation studies, cohort studies, case control studies, and diagnostic test studies
CEBM (Oxford's Centre for Evidence Based Medicine - Critical Appraisal Worksheets)
CEBM offers sheets for systematic reviews, diagnostic accuracy studies, prognosis studies, randomized controlled trials, qualitative studies, and individual patient data systematic reviews
GRADE
For use in systematic reviews and other evidence syntheses (e.g., clinical guidelines) where a recommendation impacting practice will be made
Joanna Briggs Institute Critical Appraisal Tools
Offers critical appraisal checklists for case control studies, case reports, cohort studies, diagnostic test accuracy studies, economic evaluations, prevalence studies, qualitative research, randomized controlled trials, systematic reviews, and other types of studies
Mixed Methods Appraisal Tool (MMAT) 2018
Used for "appraising and/or describing studies included in systematic mixed studies reviews (reviews including original qualitative, quantitative and mixed methods studies)" (from the website)
The Patient Education Materials Assessment Tool (PEMAT) and User's Guide
Used to evaluate and compare the understandability and actionability of patient education materials
PEDro Scale (Physiotherapy Evidence Database)
Helps users identify trials that are likely to be internally valid and have sufficient statistical information to guide clinical decision-making
Risk of Bias (RoB) Tools
The RoB 2 tool is designed for assessing risk of bias in randomized trials, while the ROBINS-I tool is meant for assessing non-randomized studies of interventions.
CanChild / McMaster EBP Research Group - Evidence Review Forms
Evidence review forms from the McMaster University Occupational Therapy Evidence-Based Practice for appraising quantitative and qualitative evidence.

Searchable Registries of Appraisal Tools & Reporting Guidelines

Equator Network: Enhancing the QUAlity and Transparency Of health Research
Comprehensive searchable database of reporting guidelines for main study types and also links to other resources relevant to research reporting
The Registry of Methods and Tools for Evidence-Informed Decision Making
The Registry of Methods and Tools for Evidence-Informed Decision Making ("the Registry") is a collection of resources to support evidence-informed decision making in practice, programs and policy. This curated, searchable resource offers a selection of methods and tools for each step in the evidence-informed decision-making process. It includes tools related to implementation science, assessing the applicability and transferability of evidence.

Determining Level of Evidence

There are a number of evidence hierarchies that could be used to "rank" evidence. Which hierarchy is applied often depends on disciplinary norms. Students should refer to materials and guidance from their professors about which hierarchy is appropriate to use.

Oxford Centre for Evidence Based Medicine - Levels of Evidence
Documents that provide guidance on ranking evidence
Joanna Briggs Levels of Evidence [PDF]
The JBI Levels of Evidence and Grades of Recommendation are meant to be used alongside the supporting document (PDF) outlining their use.