Skip to main content

Systematic Reviews, Scoping Reviews, and other Knowledge Syntheses

Screening resources

Screening resources

Screening tools

Software packages specifically designed for knowledge synthesis will typically include a record screening/study selection function. This allows more than one reviewer to independently screen the records without seeing other reviewers' decisions to include or exclude studies, and thus avoids bias.

The following systematic review addresses how text mining is being used in the screening process:

Useful resource to identify screening tools:

PICo-based title-only screening method for scoping searches and rapid reviews

See: Rathbone J, Albarqouni L, Bakhit M, Beller E, Byambasuren O, Hoffmann T, et al. Expediting citation screening using pico-based title-only screening for identifying studies in scoping searches and rapid reviews. Syst Rev. 2017;6(1):233. 

Rayyan for screening

Rayyan for screening

The McGill Library supports use of the following tool:

Critical appraisal and tools to identify risk of bias

Critical appraisal and tools to identify risk of bias

Critical appraisal should involve an assessment of the risk of bias in the relevant studies and may also involve an assessment of how the studies were reported.

As librarians, we are generally not involved in the appraisal process, but we can provide guidance on finding critical appraisal tools if needed. They are generally specific to a given study design or research methodology. The following are some suggested tools but this list is not exhaustive and they may not have been validated.

Critical appraisal of randomized studies on effects of interventions:
  • Julian PT Higgins, Douglas G Altman and Jonathan AC Sterne on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group. Assessing risk of bias in included studies. In Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011). Chapter 8. Available from http://handbook-5-1.cochrane.org

  • RoB-2.0 tool: For risk of bias in individually randomized, parallel group trials; cluster randomized, parallel group trials; and individually randomized, cross-over trials
Critical appraisal of non-randomized studies on the effects of interventions (includes observational studies):
Critical appraisal of diagnostic test accuracy studies:
Critical appraisal of measurement instruments:
  • COSMIN Checklist: Risk of bias checklist for assessing the quality of studies involving outcome measurement instruments (mainly for patient reported outcome measures, or PROMs); see the section on risk of bias in the Guideline for Systematic Reviews of Outcome Measurement Instruments
  • QODA Checklist: For critically appraising questionnaire origin (sources from which questionnaire elements are made) and initial development (process of making the questionnaire from its elements)
Critical appraisal in mixed studies reviews::
Critical appraisal of prediction model studies:
  • PROBAST form
  • See: Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., . . . Group†, f. t. P. (2019). PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies. Annals of Internal Medicine, 170(1), 51-58. doi:10.7326/m18-1376
Critical appraisal of qualitative research findings:
  • See: Noyes, J., Booth, A., Flemming, K., Garside, R., Harden, A., Lewin, S., . . . Thomas, J. (2018). Cochrane Qualitative and Implementation Methods Group Guidance Series—Paper 3: Methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. Journal of Clinical Epidemiology, 97, 49-58. doi:10.1016/j.jclinepi.2017.06.020
  • See: Santiago-Delefosse, M., Gavin, A., Bruchez, C., Roux, P., & Stephen, S. L. (2016). Quality of qualitative research in the health sciences: Analysis of the common criteria present in 58 assessment guidelines by expert users. Social Science and Medicine, 148, 142-151. doi:10.1016/j.socscimed.2015.11.007
Critical appraisal of systematic reviews of intervention studies:

We recommend consulting these prior to starting your own systematic review of intervention studies, as they will draw your attention to important elements in a high quality review.

  • AMSTAR 2: For critically appraising systematic reviews of randomised and non-randomised studies of healthcare interventions
  • ROBIS: For risk of bias in systematic reviews
Critical appraisal of systematic reviews of public health interventions:

 

Additional critical appraisal tools:

Resources on synthesizing findings

Resources on synthesizing findings

Campbell, M., McKenzie, J. E., Sowden, A., Katikireddi, S. V., Brennan, S. E., Ellis, S., Hartmann-Boyce, J., Ryan, R., Shepperd, S., Thomas, J., Welch, V., & Thomson, H. (2020). Synthesis without meta-analysis (SWiM) in systematic reviews: Reporting guideline. BMJ, 368, l6890. doi:10.1136/bmj.l6890

Contact us

Find a librarian in your subject area to help you with your knowledge synthesis project.

 

Or contact the librarians at the
Schulich Library of Physical Sciences, Life Sciences, and Engineering
schulich.library@mcgill.ca

Need help? Ask us!

McGill LibraryQuestions? Ask us!
Privacy notice