Systematic reviews are carried out by a large number of staff and students at King's College London and King's Health Partners across the disciplines. This guide aims to assist you in understanding more about how to effectively and systematically search for literature to include in your systematic review. The main focus is on searching for content to include in systematic reviews carried out in health and clinical and life sciences, but some specific links and guidance are also available for searching for systematic reviews of social interventions and other qualitative research areas in health and the social sciences.
Performing a high quality electronic search of information resources ensures the accuracy and completeness of the evidence base used in your review. It is essential to get this searching element right otherwise your results will potentially be biased/missing crucial evidence. To be successful you will need to be looking for the evidence in the right places, matching your topic to the resources you are searching and, as far as possible leaving no stone unturned. Spending time on the search part of the systematic review is very important.
"Systematic reviews attempt to bring the same level of rigour to reviewing research evidence as should be used in producing that research evidence in the first place and should be based on a peer-reviewed protocol so that they can be replicated if necessary ...
High quality systematic reviews seek to:
Source: Hemingway, P. and Brereton, N. (2009) What is a systematic review?, What is...? series [online]. URL: http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/Systreview.pdf [accessed 01.12.15; no longer accessible at stated URL 17.11.16].
*All our emphasis; the content of this guide focuses on how to search for and identify the evidence to be included in systematic reviews.
The Cochrane Collaboration sets out eight stages of doing a systematic review:
It is important to consider whether you are undertaking a full systematic review or are instead being asked to complete a systematic literature review (which may be more limited in scope). A full systematic review aims to comprehensively identify, evaluate and integrate the findings of all relevant studies on a particular research question. A systematic literature review is more selective but implies a rigorous and structured search strategy, without necessarily attempting to include all available research on a particular topic.
Whilst much of the information included in this guide will be relevant for those undertaking systematic literature reviews (as opposed to a systematic review) you may wish to discuss with your supervisor the scope of the review you are being asked to complete and whether you need to be as comprehensive as a full systematic review would demand. For example, are you expected to include both published and unpublished material (conference papers, RCT trial databases, PhD theses)? Are you being guided to only search in a restricted number of databases? Have you been told to only include a specific number of results? Or advised to limit by date or language purely to restrict numbers of results? Often time is a major limitation to the systematic literature review (e.g. undertaken as part of an MSc course) and so limits have to be placed, particularly on the number of articles to be appraised.
You will still be able to undertake a high quality systematic literature review if any of the above apply but it is worth bearing in mind when you start your review in case some of the guidance included in this library guide is not necessary for you to follow. When writing up your review you could consider whether any of these decisions could be considered a limitation to the research you have conducted and perhaps what further research should include to improve the quality and check the validity of the results.
Systematic reviews may examine quantitative or qualitative evidence. In the past systematic reviews were predominantly medical and often with a narrowly defined focus. Increasingly systematic reviews are attempting to deal with much broader topics, including topics allied to medicine but also topics outside of medicine. It is becoming more common in certain disciplines to see two or more types of evidence included and appraised and this is often called a mixed-method systematic review.
The guidance for systematic review methodology promoted by the Cochrane Library is focused very much on quantitative methods and may not be suitable for those undertaking a qualitative systematic review where a meta-ethnography is the aim as opposed to a meta-analysis.
There is much discussion as to whether a qualitative systematic review should aim to include a comprehensive literature search in the same way as is required for quantitative systematic reviews. It may be that "while it is certainly important for the search process to be free from bias, it is more important that the search process be systematic, explicit and reproducible rather than comprehensive. Thoroughness in this context should apply to the rigour of the search process not its comprehensiveness" (Booth, 2001).
Booth (2001) suggests that literature searching for qualitative systematic reviews should exhibit the following characteristics:
a) Identifying major "schools of thought" in a particular area whilst being alert to the identification of variants, minority views and dissenters. It is particularly important to identify negative or disconfirming cases.
b) Searching within a broad range of disciplines so as to bring different views (e.g. clinician, consumer, manager, health economist, statistician, research commissioner, etc.) to bear on the topic in hand.
c) Using complementary electronic and manual search techniques to ensure that materials are not missed either through the inadequacies of indexing or through selective coverage of databases.
Booth, A. (2001), 'Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research?' In Qualitative Evidence-based conference: Taking a critical stance, Coventry University. http://www.leeds.ac.uk/educol/documents/00001724.htm
The Cochrane Database of Systematic Reviews (CDSR) is the leading resource for systematic reviews in health care.
The Cochrane Collaboration are often described as the gold standard producer of systematic reviews. They provide guidance on how a systematic review (of an intervention or DTA) should be carried out, including a detailed section on the searching element.
The Campbell Collaboration is an international research network that produces systematic reviews of the effects of social interventions in Crime & Justice, Disability, Education, International Development, Knowledge Translation and Implementation, Nutrition, and Social Welfare. They promote positive social and economic change through the production and use of systematic reviews and other evidence synthesis for evidence-based policy and practice.
The Campbell Collaboration's Information Retrieval Methods Group has published a guide to information retrieval for systematic reviews: "Searching for studies: A guide to information retrieval for Campbell Systematic Reviews". This is based on the searching chapter within the Cochrane Handbook but adapted to suit the different subject area.