Skip to Main Content

Systematic Reviews

Learn about conducting systematic reviews

Question Guidance

The first and most important decision in preparing a systematic review is to determine its focus. This is best done by clearly framing the questions the review seeks to answer. 

  • Systematic reviews should address answerable questions and fill important gaps in knowledge.
  • Developing good review questions takes time, expertise and engagement with intended users of the review.
  • Cochrane Reviews can focus on broad questions, or be more narrowly defined. There are advantages and disadvantages of each.
  • Logic models are a way of documenting how interventions, particularly complex interventions, are intended to ‘work’, and can be used to refine review questions and the broader scope of the review.
  • Using priority-setting exercises, involving relevant stakeholders, and ensuring that the review takes account of issues relating to equity can be strategies for ensuring that the scope and focus of reviews address the right questions.

From Chapter 2 of the Cochrane Handbook for Systematic Reviews of Interventions. This chapter provides detailed guidance for developing a research question. 

Research Question Frameworks

As you consider the scope of your research, think about how you will define these concepts:

  • Population / Problem: who are you screening? Why?
  • Intervention: what are you evaluating? e.g., a treatment, an intervention, etc.
  • Comparison: are you comparing this group to another group, e.g. a placebo group?
  • Outcome: what are the outcomes? Is there a specific one you are looking at?

Qualitative PICo

  • Population / Problem
  • Phenomenon of Interest
  • Context

PICO variations

  • PEO: Exposure
  • PICOT: Timeframe 
  • PICOTS: Timeframe, Setting
  • PICOS: Study Design, e.g. cohorts or randomized controlled trials.

(From Lackey, M. (2013). Systematic reviews: Searching the literature [PowerPoint slides].

In adults, is screening for depression and feedback of results to providers more effective than no screening and feedback in improving outcomes of major depression in primary care settings?

  • Population / Problem: adults / depression (major)
  • Intervention: screening, feedback
  • Comparison: none
  • Outcome: no particular outcomes specified

(From Lackey, M. (2013). Systematic reviews: Searching the literature [PowerPoint slides]. Retrieved from http://guides.lib.unc.edu/ld.php?content_id=258919)

Table 1. Five paradigmatic approaches and examples for identifying the exposure and comparator in systematic review and decision-making questions.

Potential systematic-review or research context

Approach

PECO example

1. Calculate the health effect from an exposure; describing the dose-effect relationship between an exposure and an outcome for risk characterization.

Explore the shape and distribution of the relationship between the exposure and the outcome in the systematic review.

Among newborns, what is the incremental effect of 10 dB increase during gestation on postnatal hearing impairment?

2. Evaluate the effect of an exposure cut-offa on health outcomes, when the cut-off can be informed iteratively by the results of the systematic review.

Use cut-offs defined based on distribution in the studies identified in the systematic review.

Among newborns, what is the effect of the highest dB exposure compared to the lowest dB exposure (e.g. identified tertiles, quartiles, or quintiles) during pregnancy on postnatal hearing impairment?

3. Evaluate the association between an exposure cut-off and a comparison cut-off, when the cut-offs can be identified or are known from other populations.

Use mean cut-offs from external or other populations (may come from other research).

Among commercial pilots, what is the effect of noise corresponding to occupational exposure compared to noise exposure experienced in other occupations on hearing impairment?

4. Identify an exposure cut-off that ameliorates the effects on health outcomes.

Use existing exposure cut-offs associated with known health outcomes of interest.

Among industrial workers, what is the effect of exposure to <80 dB compared to ≥80 dB on hearing impairment?

5. Evaluate the potential effect of a cut-off that can be achieved through an intervention to ameliorate the effects of exposure on health outcomes.

Select the comparator based on what exposure cut-offs can be achieved through an intervention.

Among the general population, what is the effect of an intervention that reduces noise levels by 20 dB compared to no intervention on hearing impairment?
  • dB: decibel; PECO: population, exposure, comparator, outcome(s).
  • a. Cut-off is a broad term referring to thresholds, levels, durations, means, medians, or ranges of exposure.

Morgan, R. L., Whaley, P., Thayer, K. A., & Schünemann, H. J. (2018). Identifying the PECO: a framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environment international, 121(Pt 1), 1027.

As well as PICO there are many other frameworks for conceptualizing your question:

BeHEMoTh- Behavior of interest, Health context,  Exclusions, Models or Theories

  • Booth, A., & Carroll, C. (2015). Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable?. Health Information & Libraries Journal, 32(3), 220-235.

ECLIPSE- Expectation/Client group/Location/Impact/Professionals/Service (Evaluating services) 

  • Wildridge, V., & Bell, L. (2002). How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Health Information and Libraries Journal, 19(2), 113-115. doi: 10.1046/j.1471-1842.2002.00378.x

FINER- Feasibility, Interesting, Novel, Ethical, Relevant

  • Cummings, S. R., Browner, W. S., & Hulley, S. B. (1988) Conceiving the research question. In S. B. Hulley, & S. R. Cummings SR (Eds), Designing Clinical Research. (pp. 12 - 17). Baltimore, MD: Williams & Wilkins

SPIDER- Sample/Phenomenon of Interest/Design/Evaluation/Research type (Qualitative studies, especially with samples rather than populations) 

  • Cooke, A., Smith, D., & Booth, A. (2012). Beyond PICO: The SPIDER tool for qualitative evidence synthesis. Qualitative Health Research, 22(10), 1435-1443. doi: 10.1177/1049732312452938

SPICE- Setting/Perspective (or Population)/Intervention/Comparison/Evaluation (Evaluating outcomes of a specific intervention) 

  • Maryse C. Kok, Hermen Ormel, Jacqueline E. W. Broerse, Sumit Kane, Ireen Namakhoma, Lilian Otiso, Moshin Sidat, Aschenaki Z. Kea, Miriam Taegtmeyer, Sally Theobald, Marjolein Dieleman. (2017) Optimising the benefits of community health workers’ unique position between communities and the health sector: A comparative analysis of factors shaping relationships in four countries. Global Public Health 12:11, pages 1404-1432.

From Monash University Library  

Type of Study
Models
Disciplines
Clinical questions
 PICO
Variants - PIO, PICOT, PICOS		 Health
Quantitative

PICO & variants PIO, PICOT, PICOS

 Health; Social Sciences

Business & Policy; Environment; Ecology
Qualitative

PEO,  PICo, ECLIPSE

 Social Sciences
Management
Mixed Methods
(use for either qualitative
or quantitative)

SPICE
SPIDER
FINER

Health
Social Sciences
Methodology or theory

BeHEMoTH

 Health

From James Cook University Library

From: Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). . BMC medical research methodology, 18(1), 1-9.

Review typology chart with framework suggestions and examples

What framework can be applied to our example article? 

Dhillon, J., Jacobs, A. G., Ortiz, S., & Rios, L. (2022).  Advances in nutrition (Bethesda, Md.). Advance online publication. https://doi.org/10.1093/advances/nmac002

  

 

Specifying Your Criteria

Chapter 3 of the Cochrane Handbook for Systematic Reviews of Interventions, Defining the criteria for including studies and how they will be grouped for the synthesis provides detailed guidance for developing a research question. 

  • Predefined, unambiguous eligibility criteria are a fundamental prerequisite for a systematic review.
  • The criteria for considering types of people included in studies in a review should be sufficiently broad to encompass the likely diversity of studies, but sufficiently narrow to ensure that a meaningful answer can be obtained when studies are considered in aggregate.
  • Considerations when specifying participants include setting, diagnosis or definition of condition and demographic factors.

Criteria Considerations:

  • The population, intervention and comparison components of the question, with the additional specification of types of study that will be included, form the basis of the pre-specified eligibility criteria for the review. It is rare to use outcomes as eligibility criteria...
  • Cochrane Reviews should include all outcomes that are likely to be meaningful and not include trivial outcomes. Critical and important outcomes should be limited in number and include adverse as well as beneficial outcomes.
  • Review authors should plan at the protocol stage how the different populations, interventions, outcomes and study designs within the scope of the review will be grouped for analysis.

Criteria to Specify: 

  • Language of publication (ideally this would not be limited) 
  • Year or year range of publications
  • How is the disease/condition defined?
  • What are the most important characteristics that describe these people (participants)?
  • Are there any relevant demographic factors (e.g. age, sex, ethnicity)?
  • What is the setting (e.g. hospital, community, etc)?
  • Who should make the diagnosis?
  • Are there other types of people who should be excluded from the review (because they are likely to react to the intervention in a different way)?
  • How will studies involving only a subset of relevant participants be handled?
  • Types of intervention
  • Data type/study type
  • Study design