With all literature searching but particularly with searching for literature for inclusion in systematic reviews it is essential that you have a focused and specific research question before you start. If your search question is too broad you will be overwhelmed with results and never be able to sift through all of them. If you haven't identified other ways that authors may have referred to the concepts in your search question then you may miss relevant papers. If you haven't broken your search question down into separate concepts you will find it more difficult to search the databases effectively and to appropriately combine your search terms.
Use a template such as the PICO template (particularly useful for reviews looking at interventions) or the Literature Searching Planning template to break your search question down into concepts and then brainstorm for synonyms, abbreviations etc. PICO encourages you to identify the population, intervention, comparison and outcomes which should be present in relevant articles that you wish to retrieve. The Literature Searching Planning template also offers a helpful reminder on how search terms should be combined in the databases using OR and AND. More information on these templates are available on this page.
Other frameworks are also available to assist in defining your search question if PICO does not seem appropriate (see the box on this page). Remember these are frameworks only; you don't have to use them or fill in something for all of the areas. Using PICO for example you may only wish to search for a specific patient group and intervention and not specify a comparison or specific outcomes. If you retrieve too much literature to cope with initially then you could add these extra concepts in to focus your question further. Even where you do not use these concepts within your search strategy they will form part of the reasoning for which studies you include or exclude from your systematic review.
From an unfocused question: What is the best treatment for ADHD?
To a focused question: Benefits and harms of methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)
The research question now specifies a particular intervention and a particular population group as well as focusing the outcomes on the beneficial and harmful effects of the intervention drug.
This focused research question is drawn from a Cochrane Systematic Review and in the selection criteria they state that they also limited the studies which were included to randomised controlled trials (RCTs). This limitation can be included in the search strategy along with the specific intervention and patient group (see Using Filters for more information). Outcomes assessed included ADHD symptoms, serious adverse events, non-serious adverse events, general behaviour and quality of life and these outcomes can also be included within a search, as well as being used as criteria to include/exclude studies when sifting through the results.
You can see the final published search strategy for this focused question in the Appendix of the systematic review (strategy optimised for the different databases the authors searched).
Storebø, O., Ramstad, E., Krogh, H., Nilausen, T., Skoog, M., Holmskov, M., Rosendal, S., Groth, C., Magnusson, F.L., Moreira-Maia, C.R., Gillies, D., Buch Rasmussen, K., Gauci, D., Zwi, M., Kirubakaran, R., Forsbøl, B., Simonsen, E., Gluud, C. (2015), 'Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)', Cochrane Database of Systematic Reviews, Issue 11. Art. No.: CD009885. DOI: 10.1002/14651858.CD009885.pub2
When we talk about developing a focused research question we mean that you should work to develop a research question/title for your systematic review which is clear and focused, and guides and centres your research. Preliminary searching may be required to gain an idea of the amount and quality of research evidence available on a particular topic before your final research question and scope of your systematic review can be finalised.
One common concern that systematic reviewers raise with us is that "there are too many results". Whilst sometimes this may be because the search strategy is not specific enough or does not adequately match the research question it is often because the search question itself is too broad and the evidence base is large. The evidence that exists and which you retrieve from your searches is the evidence that you should be including and appraising in your review. If there is too much evidence for you to appraise you can narrow and focus your research question, e.g. can you limit to a particular care setting, or age-range or to a high quality research design such as RCTs, or add in a comparison? Use the templates suggested on this page to help with this.
It is worth remembering that Cochrane and other high quality systematic reviewers will spend significant amounts of time (often months rather than weeks) both developing their search strategy and then in appraising the evidence that is retrieved. If you feel there is too much evidence being retrieved ask yourself whether the results are relevant to your research question as it stands (if not you will have to review the search strategy and see why large numbers of irrelevant results are being retrieved) and if they are relevant and you feel they are too many to deal with then you will need to narrow and focus the question (and adapt the search strategy appropriately).
Consider at an early stage of your systematic review what inclusion and exclusion criteria you will be using to assess the relevance and quality of the research which you retrieve as part of the search, and to decide whether it will be included in the actual systematic review. Using a framework such as PICO (discussed elsewhere on this page) can be useful as you may identify concepts which you may not include within the search strategy but which instead will form the basis of your inclusion and exclusion criteria. This can help you formulate the search strategy so that you retrieve a targeted set of results.
"The search process is based on the eligibility criteria that reviewers establish before they begin the process of identifying, locating, and retrieving the research needed to address the problem of evidence-based practice. The eligibility criteria specify which studies will be included and which will be excluded from the systematic review— though the criteria may be subject to change as the systematic review progresses through the early stages of the process, some of the criteria are fundamental to collecting a rigorous and defensible set of data for the review."
Meline, T. (2006), 'Selecting studies for systematic review: inclusion and exclusion criteria', Contemporary Issues in Communication Science & Disorders 33, 21-27.
1.Focus your question
2.Identify key search topics and terms
3.Develop inclusion criteria
4.Identify appropriate databases for your topic
The Literature Searching Planning Template provides a worked example of using the template to break down a search question into its concepts. It also provides guidance on how to combine synonyms/terms within the same concept and then how to combine all the concepts together. This template can be used as a standalone tool or in addition to one of the other search tools/frameworks e.g. PICO or PESTEL.
A blank template is also provided for you to use with your own search question.
Worked example using the PICO framework to break down a focussed search question into its separate concepts and to identify synonyms etc, followed by a PICO exercise.
A blank template is also provided for you to use with your own search question.
Other frameworks/planning tools that you could consider using to break your search question down into concepts include:
The Campbell Collaboration (systematic reviews in the social sciences etc) states in Searching for studies: A guide to information retrieval for Campbell Systematic Reviews that:
Generally speaking, a search strategy to identify intervention studies will typically have three sets of terms: 1) the condition of interest, i.e., the population; 2) the intervention(s) evaluated; and 3) the outcomes (optional)."
For a rapid review of 38 different frameworks for formulating questions see:
Booth A, Noyes J, Flemming K, et al. Formulating questions to explore complex interventions within qualitative evidence synthesis, MJ Global Health 2019;4:e001107.