This guide is brought to you by:
What is critical appraisal?
Critical appraisal is a crucial part of evidence-based medicine, yet reading and critiquing a journal article can seem like a daunting and complex task. Breaking the process down into steps should enable you to build up the necessary skills, such as:
- Skimming the article in the first instance to look for the author's main points and conclusions
- Being familiar with the way that many journal articles are structured (abstract, method, results, discussion etc)
- Reflecting on and being critical of what you are reading
A checklist or toolkit such as those on this page will guide you through this process in a structured way. This page will also direct you to articles, web pages, online guides and books to guide you toward effectively appraising scientific articles.
If you have any suggestions for helpful books, links, or resources about critical appraisal please email library@kcl.ac.uk.
Statistics
Medical Statistics Made Easy by ISBN: 9781904842552Publication Date: 2014Includes explanations of confidence intervals, p-values, risk and odds.
Statistics for People Who (Think They) Hate Statistics by Publication Date: 2010
Discovering Statistics Using SPSS by Publication Date: 2009 (3rd ed.), 2013 (4th ed.)Focuses on using SPSS but Chapter 2 gives explanations of concepts such as p-values and confidence intervals. 3rd edition available as an ebook.
Fast Facts: Medical Statistics by ISBN: 1912776685Publication Date: 2020-06-03Using real examples from oncology trials, but keeping it simple, this concise resource explains the basic principles of medical statistics so that you can better appraise clinical trial results.
- p-values and confidence intervalsBandolier guide
- KCL King's Academic Skills for Learning (KASL) Maths & Stats at University moduleOften, when students arrive at university, they find that they have forgotten much of the maths they learnt at school. The resources provided in this KASL module on KEATS available to KCL students are designed to help students gain skill and confidence in their understanding and use of mathematics and statistics in a variety of contexts. For specific queries, the Volunteer Electronic Helpdesk on Research Methods and Statistics is an ideal first port of call. Log-in to the KEATS module with your k number to access the link. Please note you may need to self-enrol first to the main KASL page https://keats.kcl.ac.uk/course/view.php?id=62576
- Unit for Medical StatisticsThe Unit for Medical Statistics (UMS) is a group of medical statisticians with a broad range of collective expertise who undertake research, consultancy, training and teaching at King’s and beyond. The Unit
provides 1-hr consultation appointments to KCL and GSTT staff.
Please note that this service is not available to KCH staff (unless they are affiliated with KCL). - IoPPN Biostatistics and Health Informatics Advisory ServiceThis service is intended to provide advice to staff and students from IoPPN departments on statistical aspects of a research project including PhD thesis and MSc dissertations, technical support with informatics related projects and qualitative research advice. Note This is only available to staff and students based at the Institute of Psychiatry, Psychology and Neuroscience KCL.
Critical Appraisal Glossaries
Trustworthiness assessment
Abstract
Background
There is increasing concern that a significant proportion of randomized controlled trials (RCTs) included in Cochrane reviews may not be trustworthy. Applying the Cochrane Pregnancy and Childbirth Trustworthiness Screening Tool (CPC-TST) has already had a clinically important effect on several reviews published by the Cochrane Pregnancy and Childbirth Group.
Objectives
We wanted to assess the impact of removing untrustworthy RCTs from already-published Cochrane reviews on a defined clinical area (antenatal and postnatal nutritional interventions).
Methods
We applied the tool to 18 Cochrane reviews (374 RCTs). The tool had four domains: (i) is the research governance trustworthy; (ii) are the baseline characteristics trustworthy; (iii) is the study feasible; (iv) are the results plausible? When additional information was needed, authors were contacted using a standard template. At least two attempts were made to contact the authors. At the end of the evaluation process each study was classified as: (i) included (YES to all questions); (ii) excluded (retracted study); or (iii) awaiting classification (any NO to the questions).
Results
Ninety-three out of 374 included studies (25%) were reclassified as “excluded” or “awaiting classification.” The number of included RCTs was reduced in 14 out of 18 reviews. Six reviews (33%) were judged to require updating because of important differences in the Summary of Findings tables (direction and size of effects and/or GRADE ratings), conclusions, implication for practice, and/or implication for research.
Conclusions
Formal assessment of trustworthiness, and inclusion only of studies that satisfy prespecified criteria for trustworthiness, affect conclusions in a relatively large number of Cochrane reviews, with potentially important clinical implications for practice and research.
Weeks J, Cuthbert A, Alfirevic Z. Trustworthiness assessment as an inclusion criterion for systematic reviews—What is the impact on results? Cochrane Ev Synth. 2023;e12037. doi:10.1002/cesm.12037
Toolkits and checklists for critical appraisal
- Critical Appraisal Skills Programme (CASP)- eight critical appraisal tools to be used when reading research.
- tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies, Qualitative studies and Clinical Prediction Rule. - JBI’s critical appraisal toolsJBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers. They provide checklists for Analytical Cross Sectional Studies; Case Control Studies; Case Reports; Case Series; Cohort Studies; Diagnostic Test Accuracy Studies; Economic Evaluations; Prevalence Studies; Qualitative Research; Quasi-Experimental Studies; Randomized Controlled Trials; Systematic Reviews and Text and Opinion.
- Mixed Methods Appraisal Tool (MMAT)Appraisal tool used for qualitative, quantitative, and/or mixed methods studies. Helpful for those working on a Mixed Methods Review.
- CONSORT Statement- set of recommendations for reporting Randomised Controlled Trials.
- Use the checklist to help you assess the different aspects of an RCT paper you are reading. - PRISMA Statement- aims to help authors improve the reporting of systematic reviews and meta-analyses by providing a minimum set of items to be included.
- use the checklist to help you check for any gaps.in a review you are reading
- please note this is not designed explicitly to be a critical appraisal tool as it focuses on the minimum to be expected - Quality assessment with diverse studies (QuADS)An appraisal tool for methodological and reporting quality in systematic reviews of mixed- or multi-method studies
Helpful links
- BMJ - How to read a paper article seriesby Trisha Greenhalgh. Articles include: 'Getting your bearings (deciding what the paper is about)'; 'Statistics for the non-statistician'; 'Papers that report drug trials' and 'Papers that go beyond numbers (qualitative research).
- Critical appraisal linksHelpful links from the library pages for NHS staff.
- How to critically appraise an articleYoung, J. M. and Solomon, M. J. (2009) 'How to critically appraise an article', Nature Clinical Practice Gastroenterology & Hepatology, 6(2), 82-91. (log in via your institution with King's username and password off campus)
Books and ebooks
New books are published regularly on the topic of reading a paper, study skills, critical appraisal etc - click the books below or use Library Search to explore books in our collection:
- Library Search @ King's
Find books and full text journal articles on your subject online!
A Beginner's Guide to Critical Thinking and Writing in Health and Social Care by Publication Date: 2015
Statistics for People Who (Think They) Hate Statistics (International Student Edition) by ISBN: 9781506361161Publication Date: 2016
Qualitative Research in Health Care by
Provides the essential information that health care researchers and health professionals need to understand the basics of qualitative research Now in its fourth edition, this concise, accessible, and authoritative introduction to conducting and interpreting qualitative research in the health care field has been fully revised and updated. Continuing to introduce the core qualitative methods for data collection and analysis, this new edition also features chapters covering newer methods which are becoming more widely used in the health research field; examining the role of theory, the analysis of virtual and digital data, and advances in participatory approaches to research. Qualitative Research in Health Care, 4th Edition looks at the interface between qualitative and quantitative research in primary mixed method studies, case study research, and secondary analysis and evidence synthesis. The book further offers chapters covering: different research designs, ethical issues in qualitative research; interview, focus group and observational methods; and documentary and conversation analysis. A succinct, and practical guide quickly conveying the essentials of qualitative research Updated with chapters on new and increasingly used methods of data collection including digital and web research Features new examples and up-to-date references and further reading The fourth edition of Qualitative Research in Health Care is relevant to health care professionals, researchers and students in health and related disciplines.
ISBN: 9781119410836Publication Date: 2020-02-03
New to using Library Search?
You can find online tutorials on using Library Search to find books and journals around your subject. Find them in the Getting Started With Your Studies section of KLaSS:
- KLaSS Module (KEATS)
Online tutorials and forums available 24/7; to help you find academic materials for your topics and choose your referencing style