Complementary & Alternative Medicine

To appraise the quality of evidence, it is essential understand the nature of the evidence source. Begin the appraisal process by considering these general characteristics:

Purpose

• What is the source? (A book chapter? A journal article? A newspaper article?)
  • Is the source primary, secondary or tertiary? (See University of Minnesota Library - Primary, Secondary, and Tertiary Sources in the Health Sciences)
• If the source is a journal article, what kind of article is it? (A report of original research? A review article? An opinion or commentary?)
• If the source is reporting original research, what was the purpose of the research?

Currency

• What is the date of publication?
• Would the evidence presented in the source still be applicable today? (Consider: has technology changed? Have recommended best clinical practices changed? Has consensus understanding of a disease, condition, or treatment changed?)

Authority/Accuracy

• Who is the author? What are the author's credentials and qualifications and to write on the topic?
• Was the source published by a credible entity? (a scholarly journal? a popular periodical, e.g, newspaper or magazine?  an association? an organization?)
• Did the source go through a peer review or editorial process before being published? (See this section of the guide for more information about locating peer reviewed articles)

Understanding how a study was conducted (the methodology) is fundamental for determining the level of evidence that was generated by the study, as well as assessing the quality of the evidence it generated.  While some papers state explicitly in the title what kind of method was used, it is often not so straightforward.  When looking at report of a study, there are a few techniques you can use to help classify the study design.

1. Notice Metadata in Database Records

In some bibliographic databases, there is information found in the Subject field, or the Publication Type field of the record that can provide information about a study's methodology.  Try to locate the record for the article of interest in CINAHL, PubMed or PsycINFO and look for information describing the study (e.g., is it tagged as a "randomized controlled trial,"  a "case report," and "observational study", a "review" article, etc).

• A word of caution: A  "review" article is not necessarily a "systematic review."  Even if the title or abstract says "systematic review," carefully evaluate what type of review it is (a systematic review of interventions? a mixed methods SR? a scoping review? a narrative review?).

2. Read the Methods Section

While there may be some information in the abstract that indicates a study's design, it is often necessary to read the full methods section in order to truly understand how the study was conducted.  For help understanding the major types of research methodologies within the health sciences, see:

• Understanding Research Study Designs (University of Minnesota Library)
• Study Designs (Centre for Evidence Based Medicine)
• Jeremey Howick's Introduction to Study Designs (Flow Chart) [PDF]
• Quantitative Study Designs (Deakin University Library)
• Grimes, D. A., & Schulz, K. F. (2002). An overview of clinical research: the lay of the land. Lancet (London, England), 359(9300), 57–61. https://doi.org/10.1016/S0140-6736(02)07283-5
• "Understanding the Evidence" article series, published in Urologic Nursing
  1. Deconstructing the Research Article (May/Jun2022; 42(3): 138-140)
  2. Background, Significance, and Literature Review (Jul-Aug2022; 42(4): 203-205)
  3. Purpose Statement, Research Questions, and Hypotheses (Sep/Oct2022; 42(5): 249-257)
  4. Quantitative Research Designs (Nov/Dec2022; 42(6): 303-311)
  5. Qualitative Research Designs (Jan/Feb2023; 43(1): 41-45)
  6. Non-Experimental Research Designs (Mar/Apr2023; 43(2): 99-102)

Once the study methodology is understood, a tool or checklist can be selected to appraise the quality of the evidence that was generated by that study.  

In order to select a tool for critical appraisal (also known as quality assessment), it is necessary to understand what methodology was used in the study.  (For help understanding study design, see this section of the guide.)

The list below sets of contains critical appraisal tools and checklists, with information about what types of studies those tools are meant for.  Additionally, there are links to reporting guidelines for different types of students, which can also be useful for quality assessment.  

If you're new to critical appraisal, check out this helpful video overview of some of the common tools:

Checklists & Tools

Reporting Guidelines

Searchable Registries of Appraisal Tools & Reporting Guidelines

For a list of additional tools, as well as some commentary on their use, see:

Ma, L.-L., Wang, Y.-Y., Yang, Z.-H., Huang, D., Weng, H., & Zeng, X.-T. (2020). Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: What are they and which is better? Military Medical Research, 7(1), 7. https://doi.org/10.1186/s40779-020-00238-8

Determining the level of evidence for a particular study or information source depends on understanding, the nature of the research question that is being investigated and the methodology that was used to collect the evidence.  See these these resources for help understanding study methodologies.  

There are a number of evidence hierarchies that could be used to 'rank' evidence. Which hierarchy is applied often depends on disciplinary norms - students should refer to materials and guidance from their professors about which hierarchy is appropriate to use.