Zoe G. Davies Centre for Evidence-Based Conservation University of Birmingham, UK Systematic Review Protocol Development

Overview What is a protocol? What is the value of a protocol? Question formulation Components of a protocol

What is a Protocol? The starting point Explicitly states the methodology to be followed during the systematic review process Available for peer-review by subject experts

What is the value of a protocol? Formalises the question under review Helps to avoid bias Ensures transparency

Question: formulation Define the systematic review question Key elements: – Subject (i.e., unit of study to which the intervention is to be applied) – Intervention (i.e., policy or management intervention under scrutiny)

Question: formulation – Outcome (i.e., any measured outcome that can be used to judge the effectiveness of the intervention) – Comparators (e.g., intervention vs. no intervention or intervention y vs. intervention z)

Question: making it relevant Dependent on the purpose of the systematic review – Management, policy or research driven question Consultation with interested stakeholders, end-users and subject experts Striking the balance – Not too broad, not too specific

Question: secondary objectives Identify factors that may influence the outcome of studies Also referred to as: – Reasons for heterogeneity – Effect modifiers (meta-analytical terminology)

Case Study: hedgerow corridors Evaluate the effectiveness of habitat corridors in promoting population viability of target species and biodiversity within fragments of remnant habitat 20 years of debate within the ecological literature

Case Study: hedgerow corridors Do hedgerows mitigate woodland habitat fragmentation? Do hedgerows increase the population viability of target species occupying otherwise isolated fragments of woodland habitat?

Case Study: hedgerow corridors Key question elements: – Subject: mammal, bird, invertebrate or amphibian populations or assemblages – Intervention: a hedgerow, or hedgerow network, connecting two or more woodland habitat fragments

Case Study: hedgerow corridors – Outcome: desired primary outcomes were change in population density for a target species or change in species richness within assemblages – Comparator: No comparator was necessary for inclusion (although appropriate spatial or temporal controls were a prerequisite for studies to be included in any subsequent meta-analysis)

Case Study: hedgerow corridors Reasons for heterogeneity: – Physical structure of the hedgerow – Vegetation composition of the hedgerow – Nature of the non-habitat matrix – Life history stage of the target species (e.g., dispersing juveniles)

Protocol: background Rational behind the systematic review Put the review question into context – Woodland fragmentation – Habitat connectivity – Hedgerow ecology

Protocol: literature search strategy State how and which information sources will be searched Key words: – Reflect the population, intervention and outcome – Consider synonyms, alternative spellings and abbreviations (e.g., colonise and colonize) – Foreign language translations

Protocol: literature search strategy Hedgerow* AND corridor* Hedgerow* AND movement* Hedgerow* AND dispersal Hedgerow* AND colonisation Hedgerow* AND colonization Hedgerow* AND connectivity Hedgerow* AND population* Hedgerow* AND communit*

Protocol: literature search strategy Combinations and permutations – Trade off between effort and return – Sensitivity vs. specificity Scope searches and refine Search generic and specific information sources

Protocol: literature search strategy Online databases and libraries – Scientific literature databases (e.g., WOK and JSTOR) – Statutory and non-governmental organisation websites (e.g., Defra, EN and RSPB) – Internet search engines (e.g., Dogpile and Google Scholar) Specialists in the field Bibliographies Hand searching libraries and museums

Protocol: study inclusion criteria Based on the key elements of the question State the filtering process to be used – Title – Abstract (Assessed by a second review and test for agreement) – Full text (Assessed by a second review and test for agreement)

Protocol: study quality assessment Hierarchy of evidence – RCT’s – Control trials without randomisation – Site comparisons – Time series data Used to determine study quality thresholds for included articles – Dependent on the purpose of the review

Protocol: data extraction What data needs to be extracted from the accepted studies? – Pilot data extraction forms – Contact authors or organisations for retrieval of missing data – Database or spreadsheet of all information relevant to the review

Protocol: data synthesis How are the studies to be pooled? – How will differences in the studies be taken into account? Propose analyses to be undertaken May not be possible to be specific at the protocol stage

Protocol: timescale Set out key milestones (e.g., completion of searching, study selection, etc.) Possible delays (e.g., consultation periods, inter library loans, etc.) Some stages may overlap

Protocol: modifications Amendments to the methodology (e.g., if there are no studies which meant the inclusion criteria) Modifications must be documented and justified – Maintain transparency – Allow independent parties to judge review validity

Further Information Available from our website: – Medical systematic review centres: – The Cochrane Collaboration ( – NHS Centre for Reviews and Dissemination (