Skip to main content

Library Guides

Searching for Systematic Reviews: Home

This guide brings together information and guidance on effective searching for journal articles and grey literature for those undertaking a systematic review.

Introduction to searching for systematic reviews

Systematic reviews are carried out by a large number of staff and students at King's College London and King's Health Partners across the disciplines. This guide aims to assist you in understanding more about how to effectively and systematically search for literature to include in your systematic review. The main focus is on searching for content to include in systematic reviews carried out in health and clinical and life sciences, but some specific links and guidance are also available for searching for systematic reviews of social interventions and other qualitative research areas in health and the social sciences. 

Performing a high quality electronic search of information resources ensures the accuracy and completeness of the evidence base used in your review. It is essential to get this searching element right otherwise your results will potentially be biased/missing crucial evidence. To be successful you will need to be looking for the evidence in the right places, matching your topic to the resources you are searching and, as far as possible leaving no stone unturned. Spending time on the search part of the systematic review is very important.

What are systematic reviews?

"Systematic reviews attempt to bring the same level of rigour to reviewing research evidence as should be used in producing that research evidence in the first place and should be based on a peer-reviewed protocol so that they can be replicated if necessary ...

High quality systematic reviews seek to:

  • Identify all relevant published and unpublished evidence*
  • Select studies or reports for inclusion
  • Assess the quality of each study or report
  • Synthesise the findings from individual studies or reports in an unbiased way
  • Interpret the findings and present a balanced and impartial summary of the findings with due consideration of any flaws in the evidence."

Source: Hemingway, P. and Brereton, N. (2009) What is a systematic review?, What is...? series [online]. URL: http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/Systreview.pdf [accessed 01.12.15; no longer accessible at stated URL 17.11.16].

*All our emphasis; the content of this guide focuses on how to search for and identify the evidence to be included in systematic reviews.

Stages of a Systematic Review

The Cochrane Collaboration sets out eight stages of doing a systematic review:

1.  Defining the review question and developing criteria for including studies
2.  Searching for studies
3.  Selecting studies and collecting data
4.  Assessing risk of bias in included studies
5.  Analysing data and undertaking meta-analyses
6.  Addressing reporting biases
7.  Presenting results and “summary of findings” tables
8.  Interpreting results and drawing conclusions
 
This guide focusses on the first two stages, with some guidance on the third stage.

When is a systematic review not a systematic review?

It is important to consider whether you are undertaking a full systematic review or are instead being asked to complete a systematic literature review (which may be more limited in scope).

Whilst much of the information included in this guide will be relevant for those undertaking systematic literature reviews (as opposed to a systematic review) you may wish to discuss with your supervisor the scope of the review you are being asked to complete and whether you need to be as comprehensive as a full systematic review would demand. For example are you expected to include both published and unpublished material (conference papers, RCT trial databases, PhD theses)? Are you being guided to only search in a restricted specific number of databases? Have you been told to only include a specific number of results? Or advised to limit by date or language purely to restrict numbers of results? Often time is a major limitation to the systematic literature review (e.g. undertaken as part of an MSc course) and so limits have to be placed, particularly to limit the number of articles which have to be appraised.

You will still be able to undertake a high quality systematic literature review if any of the above apply but it is worth bearing in mind when you start your review in case some of the guidance included in this library guide is not necessary for you to follow. When writing up your review you could consider whether any of these decisions could be considered a limitation to the research you have conducted and perhaps what further research should include to improve the quality and check the validity of the results.

Cochrane Collaboration

The Cochrane Database of Systematic Reviews (CDSR) is the leading resource for systematic reviews in health care.

The Cochrane Collaboration are often described as the gold standard producer of systematic reviews. They provide guidance on how a systematic review (of an intervention or DTA) should be carried out, including a detailed section on the searching element.

Campbell Collaboration

The Campbell Collaboration is an international research network that produces systematic reviews of the effects of social interventions in Crime & Justice, Education, International Development, and Social Welfare. 

The Campbell Collaboration's Information Retrieval Methods Group has published a guide to information retrieval for systematic reviews: "Searching for studies: A guide to information retrieval for Campbell Systematic Reviews". This is based on the searching chapter within the Cochrane Handbook but adapted to suit the different subject area.

KEATS modules

Are you having trouble booking onto a face-to-face session? We have created a comprehensive e-learning course on Searching for Systematic Review that includes everything we cover in the face-to-face session, but you can learn anywhere and anytime.

To find the course, click on this link: KLaSS - King's Learning and Skills Services Home and navigate to the Advanced Searching for Systematic Review section.

Training Sessions

Courses on literature review searching and on advanced search techniques for systematic reviews and on EndNote/RefWorks to manage your references are run during term time and can be booked for free via Skills Forge (King's College London staff and students) or the NHS session booking form. Bookings are taken on a first come, first served basis.

Please note that undergraduates wishing to attend a course on searching for systematic reviews should contact their supervisor/tutor in order to make contact with the library to arrange group training.

Assistance on searching techniques is also provided to all systematic reviewers (all students and staff) via the Library Enquiry Desk in the first instance (visit your Library, or email, telephone or instant chat).

Different types of Systematic Reviews - Qualitative evidence

Systematic reviews may examine quantitative or qualitative evidence. In the past systematic reviews were predominantly medical and often with a narrow defined focus. Increasingly systematic reviews are attempting to deal with much broader topics and also topics outside of medicine; both those disciplines allied to medicine and beyond. It is becoming more common in certain disciplines to see two or more types of evidence included and appraised and this is often called a mixed-method systematic review.

The guidance for systematic review methodology promoted by the Cochrane Library is focused very much on quantitative methods amd may not be suitable for those undertaking a qualitative systematic reviews where a meta-ethnography is the aim as opposed to a meta-analysis.

There is much discussion as to whether a qualitative systematic review should aim to include a comprehensive literature search in the same way as is required for quantitative systematic reviews. It may be that "while it is certainly important for the search process to be free from bias, it is more important that the search process be systematic, explicit and reproducible rather than comprehensive. Thoroughness in this context should apply to the rigour of the search process not its comprehensiveness", (Booth, 2001).

Booth (2001) suggests that:

Literature searching for qualitative systematic reviews should exhibit the following characteristics:

a) Identifying major "schools of thought" in a particular area whilst being alert to the identification of variants, minority views and dissenters. It is particularly important to identify negative or disconfirming cases (Paterson et al, 1998)

b) Searching within a broad range of disciplines so as to bring different views (e.g. clinician, consumer, manager, health economist, statistician, research commissioner etceteras) to bear on the topic in hand.

c) Using complementary electronic and manual search techniques to ensure that materials are not missed either through the inadequacies of indexing or through selective coverage of databases.

Booth A: Cochrane or cock-eyed? How should we conduct systematic reviews of qualitative research? In Qualitative Evidence-based conference: Taking a critical stance. Coventry University ; 2001. http://www.leeds.ac.uk/educol/documents/00001724.htm