Research Guides: Health Care Administration and Management: Critical Appraisal Resources

Tools & Cheklists

In order to select a tool for critical appraisal (also known as quality assessment or "risk of bias" assessment), it is necessary to understand what methodology was used in the study. (For help understanding study design, see this section of the guide.)

The list below sets of contains critical appraisal tools and checklists, with information about what types of studies those tools are meant for. Additionally, there are links to reporting guidelines for different types of students, which can also be useful for quality assessment.

If you're new to critical appraisal, check out this helpful video overview of some of the common tools:

Checklists & Tools

AGREE II Instrument
The AGREE II an instrument is valid and reliable tool that can be applied to any practice guideline in any disease area and can be used by health care providers, guideline developers, researchers, decision/policy makers, and educators.
For help using the AGREE II instrument, see the AGREE II Training Tools
AMSTAR 2
AMSTAR 2 is the revised version of the popular AMSTAR tool (a tool for critically appraising systematic reviews of RCTs). AMSTAR 2 can be used to critically appraise systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both.
Appendix 1: Checklists for Finding, Appraising, and Implementing Evidence
A collection of checklists for a number of purposes related to EBM, including finding, interpreting, and evaluating research evidence.
Found in Appendix 1 of Greenhalgh, Trisha. (2010). How to Read a Paper : The Basics of Evidence Based Medicine, 4th edition.
CASP International Network (CASPin) Checklists
The CASP International Network (CASPin) is an international collaboration which supports the teaching and learning of critical appraisal skills, and in particular helps other people set up sustainable training programmes.
Appraisal tools are available for:

Systematic reviews
Randomised controlled trials
Qualitative research studies
Economic evaluation studies
Cohort studies
Case control studies
Diagnostic test studies
CEBM (Oxford's Centre for Evidence Based Medicine) - Critical Appraisal Worksheets
CEBM offers Critical Appraisal Sheets for:
Systematic Reviews
Diagnostic Accuracy Studies
Prognosis Studies
Randomised Controlled Trials (RCT)
Qualitative Studies
Individual Patient Data Systematic Reviews
GRADE
The GRADE working group has developed a common, sensible and transparent approach to grading quality of a body of evidence and strength of recommendations that can be drawn from randomized and non-randomized trials. GRADE is meant for use in systematic reviews and other evidence syntheses (e.g., clinical guidelines) where a recommendation impacting practice will be made.
Joanna Briggs Institute Critical Appraisal Tools
JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers. There are checklists available for:
Analytical Cross Sectional Studies
Case Control Studies
Case Reports
Case Series
Cohort Studies
Diagnostic Test Accuracy Studies
Economic Evaluations
Prevalence Studies
Qualitative Research
Quasi-Experimental Studies
Randomized Controlled Trials
Systematic Reviews
Text and Opinion
The Patient Education Materials Assessment Tool (PEMAT) and User’s Guide
The Patient Education Materials Assessment Tool (PEMAT) is a systematic method to evaluate and compare the understandability and actionability of patient education materials. It is designed as a guide to help determine whether patients will be able to understand and act on information. Separate tools are available for use with print and audiovisual materials.
MMAT (Mixed Methods Appraisal Tool) 2018
"The MMAT is a critical appraisal tool that is designed for the appraisal stage of systematic mixed studies reviews, i.e., reviews that include qualitative, quantitative and mixed methods studies. It permits to appraise the methodological quality of five categories to studies: qualitative research, randomized controlled trials, non randomized studies, quantitative descriptive studies, and mixed methods studies."
PEDro Scale (Physiotherapy Evidence Database)
The PEDro scale was developed to help users rapidly identify trials that are likely to be internally valid and have sufficient statistical information to guide clinical decision-making.
Risk of Bias (RoB) Tools
The RoB 2 tool is designed for assessing risk of bias in randomized trials, while the ROBINS-I tool is meant for assessing non-randomized studies of interventions.
CanChild / McMaster EBP Research Group - Evidence Review Forms
Evidence review forms from the McMaster University Occupational Therapy Evidence-Based Practice for appraising quantitative and qualitative evidence.

Reporting Guidelines

CONSORT (CONsolidated Standards Of Reporting Trials)
The CONSORT Statement is an evidence-based, minimum set of standards for reporting of randomized trials. It offers a standard way for authors to prepare reports of trial findings, facilitating their complete and transparent reporting, and aiding their critical appraisal and interpretation.
TREND (Transparent Reporting of Evaluations with Nonrandomized Designs)
The TREND statement has a 22-item checklist specifically developed to guide standardized reporting of non-randomized controlled trials.
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)
RISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA primarily focuses on the reporting of reviews evaluating the effects of interventions, but can also be used as a basis for reporting systematic reviews with objectives other than evaluating interventions.
There are also extensions available for scoping reviews, as well as other aspects or types of systematic reviews.
SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence)
The SQUIRE guidelines provide a framework for reporting new knowledge about how to improve healthcare (i.e., quality improvement). These guidelines are intended for reports that describe system level work to improve the quality, safety, and value of healthcare, and used methods to establish that observed outcomes were due to the intervention(s).

Searchable Registries of Appraisal Tools & Reporting Guidelines

Equator Network: Enhancing the QUAlity and Transparency Of health Research
Comprehensive searchable database of reporting guidelines for main study types and also links to other resources relevant to research reporting.
The Registry of Methods and Tools for Evidence-Informed Decision Making
The Registry of Methods and Tools for Evidence-Informed Decision Making ("the Registry") is a collection of resources to support evidence-informed decision making in practice, programs and policy. This curated, searchable resource offers a selection of methods and tools for each step in the evidence-informed decision-making process. Includes tools related to implementation science, assessing the applicability and transferability of evidence.

For a list of additional tools, as well as some commentary on their use, see:

Ma, L.-L., Wang, Y.-Y., Yang, Z.-H., Huang, D., Weng, H., & Zeng, X.-T. (2020). Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: What are they and which is better? Military Medical Research, 7(1), 7. https://doi.org/10.1186/s40779-020-00238-8