Systematic mapping to inform decision making Nicola Randall Centre for Evidence Based Agriculture Harper Adams University

Types of evidence syntheses Systematic reviews Systematic maps Rapid Evidence Assessments Quick Scoping reviews Defra Extensive literature reviews EFSA

Background The effectiveness of integrated farm management, organic farming and agri-environment schemes as interventions for conserving biodiversity.

Background Multiple interventions, exposures, populations and outcomes Different study designs Needed an alternative to Systematic Review

What is systematic mapping? A structured method to describe the state of available research evidence for a topic or question Describe the nature, coverage and possibly the ‘robustness’ of available research Identify ‘knowledge gaps’ sub-topics under-represented by the literature that may benefit from primary research Identify ‘knowledge clusters’ sub-topics that are well represented by the literature that may be suitable for further secondary synthesis

Systematic mapping Follows objective, repeatable and transparent processes to search for, collate and catalogue published and unpublished literature relating to a topic or question. ‘critical appraisal’ of included studies may also be carried out.

Typical systematic map process Protocol Search for literature Screen studies against inclusion criteria ‘Coding’ - Extract study meta-data and create map database Critical appraisal (optional) Describing and visualising the evidence base Publish narrative report alongside map database Typical systematic map process The first stages (the first 3 boxes) in a systematic map are similar to those set out for systematic review ie. establishing the review team, engaging stakeholders. Development of the protocol includes setting the question or topic, developing search terms and search strategy (ie. which sources will be searched), carrying out a scoping study to test search terms and the amount of information that will be retrieved., developing inclusion criteria (ie. what a study must contain to be included), critical appraisal strategy and coding strategy (ie. what variables will be extracted from the studies). However, following screening a database is populated with study meta-data - variables that describe the study’s setting and design. The result is a searchable database of research studies that can be interrogated to provide an overview of the evidence base.

Uses for Systematic Maps

The Effectiveness of five different on-farm mitigations for delivering an improved water environment. Defra/NERC funded

Water quality Broad Question – High policy relevance, but lots of data to examine >146,000 references >74,000 after duplicates removed 1359 sorted by title and abstract 718 contain evidence of farming interventions 495 full text 410 full text and no confounding factors

Bo Söderström, Katarina Hedlund, Louise E Jackson, Thomas Kätterer, Emanuele Lugato, Ingrid K Thomsen and Helene Bracht Jørgensen. What are the effects of agricultural management on soil organic carbon (SOC) stocks? Environmental Evidence 2014,3:2 http://www.eviem.se/en/projects/Soil-organic-carbon-stocks/

Bo Söderström, Katarina Hedlund, Louise E Jackson, Thomas Kätterer, Emanuele Lugato, Ingrid K Thomsen and Helene Bracht Jørgensen. What are the effects of agricultural management on soil organic carbon (SOC) stocks? Environmental Evidence 2014,3:2 http://www.eviem.se/en/projects/Soil-organic-carbon-stocks/

Types of output Searchable database Scientific publications Website – systematic review library Guidance notes Technical press Where else?

Uses - Inform policy 90 page full report converted to: Policy brief Highlight the findings rather than the methods used Provide implications for policy, recommendations for future research, and for future research syntheses

Uses - Inform primary research Buffer strips to deliver multifunctional benefits on farmland pollinators water protection pest control

Uses - Inform further synthesis Overview of reviews A systematic map, Traditional, more focussed, systematic review of effectiveness. Ruth Stewart, Laurenz Langer, Natalie Rebelo Da Silva, Evans Muchiri, Hazel Zaranyika, Yvonne Erasmus, Nicola Randall, Shannon Rafferty, Marcel Korth, Nolizwe Madinga, Thea de Wet (2015) The Effects of Training, Innovation and New Technology on African Smallholder Farmers’ Wealth and Food Security: A Systematic Review The Campbell Library - International Development Group 11 (16) 

Where next? Common methodological advice Consideration of breadth and inclusion e.g. Impacts of Neonicotinoids

References Katy L. James, Nicola P. Randall and Neal R. Haddaway. A methodology for systematic mapping in environmental sciences. Environmental Evidence 2016 5:7 Nicola P Randall, Louise M Donnison, Paul J Lewis and Katy L James. How effective are on-farm mitigation measures for delivering an improved water environment? A systematic map. Environmental Evidence 2015 4:18 Ruth Stewart, Laurenz Langer, Natalie Rebelo Da Silva, Evans Muchiri, Hazel Zaranyika, Yvonne Erasmus, Nicola Randall, Shannon Rafferty, Marcel Korth, Nolizwe Madinga, Thea de Wet (2015) The Effects of Training, Innovation and New Technology on African Smallholder Farmers’ Wealth and Food Security: A Systematic Review The Campbell Library - International Development Group 11 (16)