Searching for Systematic Reviews & Evidence Synthesis: Define your search question

With all literature searching but particularly with searching for literature for inclusion in systematic reviews it is essential that you have a focused and specific research question before you start. If your search question is too broad you will be overwhelmed with results and never be able to sift through all of them. If you haven't identified other ways that authors may have referred to the concepts in your search question then you may miss relevant papers. If you haven't broken your search question down into separate concepts you will find it more difficult to search the databases effectively and to appropriately combine your search terms.

Use a literature searching planning template to break your search question down into concepts or facets and then brainstorm for synonyms, abbreviations etc. Using a framework can also help with this process. The PICO framework encourages you to identify the population, intervention, comparison and outcomes which should be present in relevant articles that you wish to retrieve. Alternatively, PEO asks who the population is, what the exposure is and what are the outcomes? The literature searching planning template also offers a helpful reminder on how search terms should be combined in the databases using OR and AND. The template is available to download on this page.

Other frameworks are also available to assist in defining your search question if PICO or PEO does not seem appropriate (see the box on this page). Remember these are frameworks only; you don't have to use them or fill in something for all of the areas. Sometimes your actual search on a database will only be for some of the concepts identified with the others used to inform inclusion and exclusion criteria. Using PICO for example you may only wish to search for a specific patient group and intervention and not specify a comparison or specific outcomes. If you retrieve too much literature to cope with initially then you could add these extra concepts in to focus your question further. Even where you do not use these concepts within your search strategy they will form part of the reasoning for which studies you include or exclude from your systematic review.

From an unfocused question: What is the best treatment for ADHD?

To a focused question: Benefits and harms of methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)

The research question now specifies a particular intervention and a particular population group as well as focusing the outcomes on the beneficial and harmful effects of the intervention drug.

This focused research question is drawn from a Cochrane Systematic Review and in the selection criteria they state that they also limited the studies which were included to randomised controlled trials (RCTs). This limitation can be included in the search strategy along with the specific intervention and patient group (see Using Filters for more information). Outcomes assessed included ADHD symptoms, serious adverse events, non-serious adverse events, general behaviour and quality of life and these outcomes can also be included within a search, as well as being used as criteria to include/exclude studies when sifting through the results.

You can see the final published search strategy for this focused question in the Appendix of the systematic review (strategy optimised for the different databases the authors searched).

Storebø, O., Ramstad, E., Krogh, H., Nilausen, T., Skoog, M., Holmskov, M., Rosendal, S., Groth, C., Magnusson, F.L., Moreira-Maia, C.R., Gillies, D., Buch Rasmussen, K., Gauci, D., Zwi, M., Kirubakaran, R., Forsbøl, B., Simonsen, E., Gluud, C. (2015), 'Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)', Cochrane Database of Systematic Reviews, Issue 11. Art. No.: CD009885. DOI: 10.1002/14651858.CD009885.pub2

When we talk about developing a focused research question we mean that you should work to develop a research question/title for your systematic review which is clear and focused, and guides and centres your research. Preliminary searching may be required to gain an idea of the amount and quality of research evidence available on a particular topic before your final research question and scope of your systematic review can be finalised.

One common concern that systematic reviewers raise with us is that "there are too many results". Whilst sometimes this may be because the search strategy is not specific enough or does not adequately reflect the research question it is often because the search question itself is too broad and the evidence base is large. The evidence that exists and which you retrieve from your searches is the evidence that you should be including and appraising in your review. If there is too much evidence for you to appraise you can narrow and focus your research question, e.g. can you limit to a particular care setting, or age-range or to a high quality research design such as RCTs, or add in a comparison? Use the templates suggested on this page to help with this.

It is worth remembering that Cochrane and other high quality systematic reviewers will spend significant amounts of time (often months rather than weeks) both developing their search strategy and then in appraising the evidence that is retrieved. If you feel there is too much evidence being retrieved ask yourself whether the results are relevant to your research question as it stands (if not you will have to review the search strategy and see why large numbers of irrelevant results are being retrieved) and if they are relevant and you feel they are too many to deal with then you will need to narrow and focus the question (and adapt the search strategy appropriately). 

Consider at an early stage of your systematic review what inclusion and exclusion criteria you will be using to assess the relevance and quality of the research which you retrieve as part of the search, and to decide whether it will be included in the actual systematic review. Using a framework such as PICO (discussed elsewhere on this page) can be useful as you may identify concepts which you may not include within the search strategy but which instead will form the basis of your inclusion and exclusion criteria. This can help you formulate the search strategy so that you retrieve a targeted set of results.

"The search process is based on the eligibility criteria that reviewers establish before they begin the process of identifying, locating, and retrieving the research needed to address the problem of evidence-based practice. The eligibility criteria specify which studies will be included and which will be excluded from the systematic review— though the criteria may be subject to change as the systematic review progresses through the early stages of the process, some of the criteria are fundamental to collecting a rigorous and defensible set of data for the review."

Meline, T. (2006), 'Selecting studies for systematic review: inclusion and exclusion criteria', Contemporary Issues in Communication Science & Disorders 33, 21-27.

1.Focus your question
2.Identify key search topics and terms
3.Develop inclusion criteria
4.Identify appropriate databases for your topic

The Literature Searching Planning Template provides a worked example of using the template to break down a search question into its concepts. It also provides guidance on how to combine synonyms/terms within the same concept and then how to combine all the concepts together. This template can be used as a standalone tool or in addition to one of the other search tools/frameworks e.g. PICO, PEO, or PESTEL. 

A blank template is also provided for you to use with your own search question.

Other frameworks/planning tools that you could consider using to break your search question down into concepts include:

• PICOST (As PICO but with additional areas to consider): Population, Intervention, Comparison, Outcome, Study Design, Time-frame
• PEO: Population and/or Problem, Exposures, Outcome
• PCC: Population, Concept, and Context [often used in Scoping Reviews]
• PESTEL: Political, Economic, Social, Technological, Environmental and Legal Factors
• SPICE: Setting, Population or Perspective, Intervention, Comparison, Evaluation
• ECLIPS [management and service related issues]: Expectations, Client Group, Location, Impact, Professionals Involved, Service
• MIP [medical ethics review]: Methodology, Issues, Participants
• SPIDER: Sample, Phenomenon of interest, Design, Evaluation, Research type 

The Campbell Collaboration (systematic reviews in the social sciences etc) states in Searching for studies: A guide to information retrieval for Campbell Systematic Reviews that: 

Generally speaking, a search strategy to identify intervention studies will typically have three sets of terms: 1) the condition of interest, i.e., the population; 2) the intervention(s) evaluated; and 3) the outcomes (optional)."

For a rapid review of 38 different frameworks for formulating questions see:
Booth A, Noyes J, Flemming K, et al. Formulating questions to explore complex interventions within qualitative evidence synthesis, MJ Global Health 2019;4:e001107. 
http://dx.doi.org/10.1136/bmjgh-2018-001107  (the frameworks identified are included in the online supplementary file 1.)