Big Data infrastructures: aspects of collaboration, innovation and engagement Sander Janssen, Earth Informatics, Wageningen University and Research (@wurcgi) 21 September 2017, CGIAR Big Data Convention, Cali

Big data and the Gartner hype cycle

Set up of road map development Three workshops: First workshop, July 2017, Montpellier: available infrastructures with a mapping Second workshop, November 27-28, 2017, Wageningen: scientific challenges and e- infrastructure needs Third workshop, May 2018: synthesis in roadmap Combined with: Workshops with policy officials (3)

Outcomes of first workshop The strategic reasons justifying the need for an e-infrastructure in agri-food; The technical challenges the application of the FAIR principles which especially relies on support for interoperability and shared semantic resources, the delivery of data discovery and processing services and the integration of disruptive technologies; The cultural challenges community management and commitment for a change of research practice sustainable governance and business models that provide incentives for data sharing; The need to further identify and develop crucial use cases (both scientific and technical) that the e-infrastructure could support.

Scientific challenge: what science will become possible with e-infrastructures? Format for case studies: Impact pathways Audiences Role of Research Challenges in data, infrastructure, processing power and analytics Solutions for more efficient Research Case studies: Agricultural Monitoring SDG 2: ending hunger and the missing middle Animal genetics and breeding Data services for small holders Food Safety

www.godan.info

Mission Stakeholders benefit from improved and more effective use of open data for agriculture and nutrition by engaging with it in a practical and knowledgeable manner Many G20 countries are already GODAN partners: directly through their governments (Mexico, Argentina, Brazil, USA, UK) or through governmental technical institutions (Canada, Germany, France, China, Italy, Japan) , other are represented through civil society organizations and academia (India, Australia). We are very proud of that .We today ask all G20 countries to join GODAN and to write the GODAN goals into the Comuniques of the MACS meeting and the ministerial meeting.

Activities Improved interoperability of data through providing improved standards and innovative services. by providing examples of improved tools for impact assessment, as well as by analysing barriers that hinder the full potential of open data initiatives and investments The development of tailored training courses will increase the capacity of stakeholders on how to use and handle open data

Why weather data? Weather patterns are changing, growing seasons get less predictable Higher risk profiles for farming and food production Yields and income Access to resources and capital Local and regional food security Access to weather data is a strategic asset for farmers

Work on standards with weather data some older standards are still very much used: legacy and compliance reasons (like BUFR or GRIB), long-term practice in research (NetCDF), wide availability of tools to manipulate them. recently API-based weather data services started serving data on demand and in application-friendly formats like Json and user-friendly formats like CSV. Some work on the standardization of variable names across the different communities No naming convention that is universally used

Recommendations for improvement Promote the benefits in adopting existing metadata standards to weather data publishers. Providing “wayfinding guides” which is tied to specific use cases. A Q&A based site (like StackOverflow) focused around weather data, standards and use of data

Our understanding of Big Data Lokers et al. 2016. Big Data Technologies in Agri-Environmental Researchhttp://dx.doi.org/10.1016/j.envsoft.2016.07.017

AgroDataCube: a big open data collection for open innovation

Soil Collect Integrate Private data Use & visualize

Attributes Time Agricultural Parcels Space Soil Height Parcels and parcel size Crop Time Agricultural Parcels Space

Lessons & reflections Reflection and experimentation sessions with private sector firms to develop proto applications Data itself: Common problems have been solved Time/Resource intensive for individual firms Lowers threshold for innovation: common starting point Added value in algorithms Level playing field

Thank you! Sander.janssen@wur.nl @Wurcgi