PRISMA or Preferred Reporting Items for Systematic Reviews and Meta-Analyses is an evidence-based protocol for reporting information in systematic reviews and meta-analyses.
To appraise the quality of evidence, it is essential understand the nature of the evidence source. Begin the appraisal process by considering these general characteristics:
Understanding how a study was conducted (the methodology) is fundamental for determining the level of evidence that was generated by the study, as well as assessing the quality of the evidence it generated. While some papers state explicitly in the title what kind of method was used, it is often not so straightforward. When looking at report of a study, there are a few techniques you can use to help classify the study design.
In some bibliographic databases, there is information found in the Subject field, or the Publication Type field of the record that can provide information about a study's methodology. Try to locate the record for the article of interest in CINAHL, PubMed or PsycINFO and look for information describing the study (e.g., is it tagged as a "randomized controlled trial," a "case report," and "observational study", a "review" article, etc).
While there may be some information in the abstract that indicates a study's design, it is often necessary to read the full methods section in order to truly understand how the study was conducted. For help understanding the major types of research methodologies within the health sciences, see:
Once the study methodology is understood, a tool or checklist can be selected to appraise the quality of the evidence that was generated by that study.
In order to select a tool for critical appraisal (also known as quality assessment or "risk of bias" assessment), it is necessary to understand what methodology was used in the study. (For help understanding study design, see this section of the guide.)
The list below sets of contains critical appraisal tools and checklists, with information about what types of studies those tools are meant for. Additionally, there are links to reporting guidelines for different types of students, which can also be useful for quality assessment.
If you're new to critical appraisal, check out this helpful video overview of some of the common tools:
The AGREE II an instrument is valid and reliable tool that can be applied to any practice guideline in any disease area and can be used by health care providers, guideline developers, researchers, decision/policy makers, and educators.
For help using the AGREE II instrument, see the AGREE II Training Tools
A collection of checklists for a number of purposes related to EBM, including finding, interpreting, and evaluating research evidence.
Found in Appendix 1 of Greenhalgh, Trisha. (2010). How to Read a Paper : The Basics of Evidence Based Medicine, 4th edition.
The CASP International Network (CASPin) is an international collaboration which supports the teaching and learning of critical appraisal skills, and in particular helps other people set up sustainable training programmes.
Appraisal tools are available for:
Randomised controlled trials
Qualitative research studies
Economic evaluation studies
Case control studies
Diagnostic test studies
CEBM offers Critical Appraisal Sheets for:
Diagnostic Accuracy Studies
Randomised Controlled Trials (RCT)
Individual Patient Data Systematic Reviews
JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers. There are checklists available for:
Analytical Cross Sectional Studies
Case Control Studies
Diagnostic Test Accuracy Studies
Randomized Controlled Trials
Text and Opinion
RISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA primarily focuses on the reporting of reviews evaluating the effects of interventions, but can also be used as a basis for reporting systematic reviews with objectives other than evaluating interventions.
There are also extensions available for scoping reviews, as well as other aspects or types of systematics reviews.
For a list of additional tools, as well as some commentary on their use, see:
Ma, L.-L., Wang, Y.-Y., Yang, Z.-H., Huang, D., Weng, H., & Zeng, X.-T. (2020). Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: What are they and which is better? Military Medical Research, 7(1), 7. https://doi.org/10.1186/s40779-020-00238-8
Determining the level of evidence for a particular study or information source depends on understanding, the nature of the research question that is being investigated and the methodology that was used to collect the evidence. See these these resources for help understanding study methodologies.
There are a number of evidence hierarchies that could be used to 'rank' evidence. Which hierarchy is applied often depends on disciplinary norms - students should refer to materials and guidance from their professors about which hierarchy is appropriate to use.