Searching for Systematic Reviews & Evidence Synthesis: Further information & Glossary

Bias

Systematic error or deviation in results or inferences from the truth. Bias will undermine the validity of a study’s results.

Campbell Collaboration 

Campbell Collaboration is a non-profit sister initiative of Cochrane that seeks to help policymakers, practitioners and the public make well informed decisions about public policy. It does this through raising the quality of evidence used in policymaking and establishing high standards of methods used in primary research and systematic reviews, for example on the effectiveness of social and behavioural interventions in education, crime and justice, and social welfare.

Cochrane Library 

A source of high-quality, independent evidence used to inform healthcare decision-making, focusing on systematic reviews, randomised and quasi-randomised controlled trials, and clinical guidelines. The Cochrane Central Register of Controlled Trials (CENTRAL) is a key database for health literature searches. The Cochrane Handbook for Systematic Reviews of Interventions is the official guide to the process of preparing and maintaining reviews that study the effects of healthcare interventions.  

CONSORT

CONSORT (Consolidated Standards of Reporting Trials) is a set of recommendations for reporting RCTs, using a checklist and flow diagram.

Critical appraisal 

The process of systematically assessing the outcome of scientific research to judge its trustworthiness, value and relevance. Checklists specific to the type of study being appraised ensure key information is considered, e.g. from CASP or Joanna Briggs Institute.

Evaluation framework

AACODS (Authority, Accuracy, Coverage, Objectivity, Date, Significance) is a checklist designed specifically for the evaluation and critical appraisal of grey literature.

Evidence gap map

A tool for collecting and summarising existing evidence. Evidence gap maps aim to inform policy decision-making and prioritise future research.

Evidence synthesis tools 

Evidence synthesis software are online tools designed to facilitate, speed up or automate labour-intensive aspects of the review process, e.g. Covidence, Rayyan, RevMan. Further information can be found on the managing references & software tools tab of the library guide.

Forest plot

Graphic representation of the size of the overall effect of all included studies from a meta-analysis.

Grey literature 

Research and information produced outside more conventional forms of publishing, which can be a valuable source of up-to-date or specialised knowledge. Grey literature can provide a corrective to publication bias. It includes reports and policy papers, theses and dissertations, conference proceedings, trial registers, clinical trial data and preprints. It also refers to techniques of research such as citation chaining and handsearching. 

Information can be found on the grey literature tab of the library guide.

Inclusion and exclusion criteria 

Defining which people or studies are eligible for review; they are typically developed after formulating the research question. Inclusion criteria are the specific characteristics a study must meet to be included in the review's analysis, and help researchers identify which studies are most relevant to their research question; exclusion criteria are the characteristics that would disqualify a study from being included and ensure a focused analysis by eliminating studies that do not fit the criteria. The process of applying these criteria is made systematic and represented visually by completing a PRISMA flowchart.

Joanna Briggs Institute 

A global organisation that promotes and supports evidence-based decisions to improve health and health service delivery. The JBI Manual for Evidence Synthesis provides comprehensive guidance for conducting systematic reviews. JBI also provides a series of open access checklists for critical appraisal.

KORDS 

KORDS (King’s Open Research Data System) is a research data repository, providing long-term storage and access for datasets at project-end and supporting publications. It also operates as a repository for protocols.

Meta-analysis 

Data collected from numerous studies that have already been conducted, comparing and combining results to draw overall conclusions.   

Narrative review 

Another name for a conventional literature review, which aims to identify and summarise what has previously been published on a topic. Compared with systematic reviews, which focus on a narrow question using a prespecified methodology, narrative reviews can include a wide variety of study types and provide an overall summary, interpretation and appraisal of existing literature.

Pre-published search strategies 

Cochrane reviews often include a ‘search strategies’ section which contains full details of the searches run in different databases. These can be used to supplement the search strategy for a new literature review, along with an appropriate acknowledgement.

PRESS 

PRESS (Peer Review of Electronic Search Strategies) is a structured tool for peer review of electronic literature search strategies for systematic review; using a structured tool such as PRESS enhances the quality and comprehensiveness of a search.

PRISMA 

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) provides guidance for literature review reporting using a detailed checklist to appraise the quality of a paper's structure, methodology, reporting and so on. PRISMA can be used to report on different aspects or types of review, e.g. protocols or scoping reviews. PRISMA also supplies a flow diagram to help researchers document the different stages of a systematic review.

PROSPERO 

An international registry for systematic reviews with health-related outcomes that are intended to be published. It helps to avoid replication by providing a database of existing reviews.

Protocol 

Guidelines established prior to a study to determine how it will be conducted, including rationale, objectives, research design, methodology, statistical considerations, pre-specified criteria for efficacy, and so on. Protocols can be submitted to PROSPERO. KCL staff and research students can also submit protocols to KORDS. Information on protocols can be found on the recording search strategy and results tab of the library guide.

Publication bias 

When outcomes determine publication, i.e. a bias towards publishing positive outcomes which distorts clinical results and means positive results are overrepresented in the research base. Researchers may deliberately skew their results knowing they are more likely to be published if positive. English-language studies and conditions affecting western populations also tend to be over-represented in the literature.  

Rapid review 

An assessment of what is already known about a specific policy or practice. It uses rigorous systematic review methods to search and appraise existing research, but to speed up the process limits some elements, e.g. by conducting fewer and broader database searches or restricting searches of grey literature.  

Realist review

A type of qualitative systematic review that explains how and why an intervention works in ‘real world’ contexts, rather than simply assessing its effectiveness.

Research framework 

Frameworks widely used to help identify the key concepts of a research topic, to inform the structure of a review, establish search terms and inclusion/exclusion criteria. PICO (population, intervention, comparison, outcome) and PEO (population, exposure, outcome) are among the most commonly-used frameworks. A list of frameworks can be found on the define your search question tab of the library guide.

Scoping review 

Scoping reviews are used to survey a body of literature, identify the types of available evidence, identify and analyse knowledge gaps, and clarify key concepts/definitions. They may also be helpful precursors to systematic reviews and can be used to confirm the relevance of inclusion criteria and potential research questions. 

Screening

The methodical evaluation of studies retrieved through the literature searching process to determine whether they meet the predefined inclusion criteria set out in the review protocol. It aims to identify and select studies relevant to the research question, while excluding those that do not. Screening is typically done by two or more reviewers to minimise bias and ensure consistency.

Search filters

A means of adding a set of terms to a search strategy which limits that search to a particular study type, such as RCTs, or ensures the sole inclusion of a specific population group or study focus, such as quality improvement. These pre-tested search filters are typically produced for specific databases and can be copied and pasted line by line into a search strategy. See the using filters tab of the library guide. 

Systematic review 

Aims to reliably answer a specific research question and produce findings to inform decision-making. This is done by finding, assessing and synthesising all studies and other evidence relevant to that question, using explicit and systematic methods focused on transparency and minimising bias. This should include evidence from grey literature that falls outside of traditional commercial publishing to mitigate publication bias. See the systematic review library guide for further information.

Systematic review (SR) toolbox

Systematic Review (SR) Toolbox is a searchable online catalogue that aims to help reviewers find appropriate tools based on their needs.

Umbrella review 

A review compiling evidence from multiple systematic reviews into one document. It focuses on a broad condition or problem for which there may be various interventions and examines reviews that address these potential interventions.