Skip to Main Content

Systematic Reviews: How-To

4. Search the Evidence

The goal of systematic review searches is to identify all relevant studies on a topic. Therefore, systematic review searches are typically quite extensive. It is necessary, however, to strike a balance between striving for comprehensiveness and maintaining relevance when developing a search strategy. Additionally, the searches should be well-documented and reproducible. Click through the tabs below to review core tenets from both the Cochrane Handbook for Systematic Reviews and the National Academies Press / IOM. 

Issues to consider when creating a systematic review search:   

  • If possible, work with a librarian experienced in conducting comprehensive literature searches.
  • All major concepts are included in the strategy; typically, we do not search on the outcomes.
  • Use of all appropriate controlled vocabulary terms across each database.
  • Appropriate use of explosion, subheadings, and floating subheadings.
  • Use of natural language (text words) in addition to controlled vocabulary terms.
  • Use of appropriate synonyms, acronyms, etc.
  • Truncation and spelling variation as appropriate.
  • Appropriate use of limits and filters.
  • Boolean operators used appropriately.
  • Check indexing of exemplar articles.
  • Search strategy adapted as needed for multiple databases (three minimum).
  • Inclusion of grey literature sources - see below for more information. 

Search Standards

  • Review authors should work closely, from the start of the protocol, with an experienced medical librarian.
  • Searches should aim for high sensitivity, which may result in relatively low precision.
  • Search strategies should avoid using too many different search concepts but a wide variety of search terms should be combined with OR within each included concept.
  • Both free-text and subject headings (e.g. Medical Subject Headings (MeSH) and Emtree) should be used.

Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Rader T, Shokraneh F, Thomas J, Wieland LS. Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). 

  • 3.1.1 Work with a librarian trained in performing systematic reviews (SRs) to plan the search strategy
  • 3.1.2 Design the search strategy to address each key research question
  • 3.1.3 [summarized] independent peer review of search
  • 3.1.4, 3.15 Search bibliographic databases + indexes
  • 3.1.6 Search literature cited by eligible studies
  • 3.1.7 Update the search at intervals appropriate to the pace of generation of new information for the research question being addressed

Institute of Medicine 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press. https://doi.org/10.17226/13059.

PRISMA-S is a 16-item checklist that covers multiple aspects of the search process for systematic reviews. It is intended to guide reporting, not conduct, of the search. The checklist should be read in conjunction with the Explanation and Elaboration (Part 3), which provides more detail about each item.

Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB; PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev. 2021 Jan 26;10(1):39. doi: 10.1186/s13643-020-01542-z. PMID: 33499930; PMCID: PMC7839230.

Grey Literature: Definition and How to Search

"Grey literature stands for manifold document types produced on all levels of government, academics, business and industry in print and electronic formats that are protected by intellectual property rights, of sufficient quality to be collected and preserved by libraries and institutional repositories, but not controlled by commercial publishers; i.e. where publishing is not the primary activity of the producing body."

- The Twelfth International Conference on Grey Literature in Prague in 2010. 

Examples of grey literature include: conference abstracts, presentations, proceedings; regulatory data; unpublished trial data; government publications; reports (such as white papers, working papers, internal documentation); dissertations/theses; patents;  and policies & procedures.

Inclusion of grey literature into a systematic review is recommended in order to help minimize publication bias. The inclusion of grey literature in systematic reviews is widely recognized as important and international organizations have incorporated this information in their guidelines and manuals for working on reviews and meta-analyses.

  • AHRQ: Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions
    This is a chapter from AHRQ's "Methods Guide for Effectiveness and Comparative Effectiveness Reviews."
  • Cochrane Handbook: Unpublished and Onging Studies
    From the Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]
  • Cochrane Handbook: Including unpublished studies in systematic reviews
    From the Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]
  • The Joanna Briggs Institute Reviewer's Manual
    Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer's Manual. The Joanna Briggs Institute, 2017. Available from https://reviewersmanual.joannabriggs.org/
  • PRISMA for systematic review protocols (PRISMA-P)
    Item 9: "Describe all intended information sources (such as electronic databases, contact with study authors, trial registers or other grey literature sources) with planned dates of coverage."

Searching the grey literature can be a daunting task. You should search those resources that make the most sense for your research question. At a minimum, consider searching unpublished trial data through trial registries as well as conference abstracts. Also check out the papers and reports of relevant stakeholder organizations.

The links below may shed some additional light on the process.

Sources of Grey Literature

The approval process for new drugs and devices involves submission of data that might not be published elsewhere. These documents can help identify publication bias even when complete methodological details of unpublished trials are unavailable. This information is not available prior to a drug’s approval and may be heavily redacted; however, when available, reviewers can compare results of published and unpublished trials.

If further information is requested, these are additional resources to consider:

Online trial registries may include results of completed but unpublished clinical trials. These resources can be helpful in identifying otherwise unreachable trials and provide additional details of published trials. Many individual drug companies also have posted trial registries on their websites, although their quality varies. Even without results, knowledge that the trial exists can be helpful for reviewers, because they can contact the principal investigator for more information. 

If further information is requested, these are additional resources to consider:

 

Newton Gresham Library | (936) 294-1614 | (866) NGL-INFO | Ask a Question | Share a Suggestion

Sam Houston State University | Huntsville, Texas 77341 | (936) 294-1111 | (866) BEARKAT
© Copyright Sam Houston State University | All rights reserved. | A Member of The Texas State University System