Building a Data-Driven Public Service August 2016 Pia Waugh @piawaugh | pia.waugh@gmail.com

What data?

What data? Specialist data Regular requests Administrative data Reports Tables Maps Analysis Key indicators Official Information Act requests Lists (places, contracts, services) Services analytics Regular requests Compliance information Public reporting data Budget Policy & programme monitoring ...

Questions What is your agency data vision and strategy? A holistic approach would yield greater benefits to everyone. What barriers exist for agencies to better use, manage, share and publish data? What is needed by data publishers and users to make data sharing and publishing part of BAU? How can we bring the different NZ data agendas together? What are the benefits of open data? Is it beneficial to agencies individually and better government as a whole? What role should data.govt.nz play?

Public sector open data motivators Efficiencies from proactively publishing common requests Cheaper and more modular services delivery Reduced regulatory burden through machine readable data supporting compliance and automated reporting Better policy outcomes by leveraging cross-agency data More consistency & less duplication across government Improved opportunities to leverage innovation and collaboration (citizens, industry, other depts) Opportunities to improve data quality through verifiable public contributions Access to data from other agencies

NZ Data Landscape Reuse projects Catalogs Publishing Sharing & Catalyst projects FigureNZ GovHack ODP advocacy Reuse projects No catalogue for sensitive data Specialist catalogues Data.govt.nz (ostensibly AoG catalogue) Geoportal Council catalogues Catalogs Publishing Stats Aggregate LINZ AoG Publishing? Sharing & Integration IDI SIU Internal Analytics LINZ (spatial and spatially enabled data) AoG Services Analytics Non-sensitive Agency data (internal, on websites) Council data Statistics (statistical data) Sensitive Agency data (internal) Data Agency LDS Agency LDS Agency LDS

How Do Agencies Publish? Some agencies have specialised data expertise and publishing capabilities. Some are developing capacity to publish. Most, if they publish data at all, publish data un-usefully. AoG data platform required for this barrier. Skills, guidance and encouragement help. ? DIA Stats LINZ

Data Needs in Agencies by Maturity Immature Maturing Mature Internal capacity Public and agency engagement Holistic data strategy Value & efficiencies Specialised access Mature sharing & publishing Access to specialised platforms & expertise Building analytics capacity Efficiencies realisation Budget for data program Strategic implementation Greater collaboration Public & agency engagement Data from other sources Internal business case dev Guidance and encouragement Publishing platform(s) Tools for internal reuse Build basic internal capacity Access to expertise Strategic development Safe sandbox (eg GovHack) Largely reliant on externals

Customer Pain Points Most Agencies have little motivation, resources or expertise to publish data. In most cases do not have a broader data strategy. If agencies consumed more data (internally & across govt) it would naturally lead to more data sharing and public availability. Data users find it hard to discover data and usually if it is findable, it isn't usable or machine readable. Often user logins are required. This creates a lack of experimentation of and confidence in government data resulting in a lack of reuse and lack of outcomes realised. Every individual barrier to using or publishing data dramatically reduces activity and the possibility of benefits realisation!

User Needs for data.govt.nz Data Users (Public, Business, Journalists, Media, Agencies) Discovery Usage Rights Usable Data (MR, APIs) Support, Requests Data Publishers (Agencies) Internal Data Strategy Easy Management Of Catalog Easy Data Publishing Access to Expertise, Docs & Training Access to Data from Across Govt Value from Open Data Analytics & Reporting Case Studies & Business Case for OD We need to educate on benefits and reduce barriers to using and publishing data so people can get on with it.

Data and Benefits What Government data broadly drives what outcomes? Policy & Investment Prioritisation Informing Policy, Decision Making Program Design, Implementation & Monitoring Less Duplication Longitudinal Analysis & Research (de-identified) Secure Sensitive data Public Aggregate data Public Non-sensitive data Agency Efficiencies Intervention, Protection & Enforcement (identified) 3rd Party Products & Services Service Delivery Improvements Public Analysis, Engagement & Value Add

Data and Benefits What systems support which outcomes? Policy & Investment Prioritisation Informing Policy, Decision Making Program Design, Implementation & Monitoring Less Duplication Longitudinal Analysis & Research (de-identified) Banks, Insurance Google OECD, WB Secure Sensitive data Public Aggregate data Public Non-sensitive data Agency Efficiencies Stats, IDI, SIU ?? LINZ, data.govt.nz Intervention, Protection & Enforcement (identified) 3rd Party Products & Services Service Delivery Improvements Public Analysis, Engagement & Value Add

Data and Benefits Investment is driving some outcomes more than others Policy & Investment Prioritisation Informing Policy, Decision Making Program Design, Implementation & Monitoring Less Duplication? Longitudinal Analysis & Research (de-identified) Secure Sensitive data Public Aggregate data Public Non-sensitive data Agency Efficiencies? Stats, IDI, SIU ?? LINZ, data.govt.nz Intervention, Protection & Enforcement (identified) 3rd Party Products & Services? Service Delivery Improvements? Public Analysis, Engagement & Value Add?

Parts of a data strategy Business Reuse (Products, services) Govt Investment (Catalyst projects) Analysis, Presentation (eg, datavis, FigureNZ) Better Govt Policy, Decisions, Efficiencies Public Innovation (eg, Govhack) Data reuse Governance, leadership Advocacy Incentives, Performance Data Policy Strategy Infrastructure (Specialist & General) Skills, Resources, Training Full Discovery Standards Implementation

Critical aspects for data.gov.au (for comparison) Harvesting metadata from existing catalogues with common metadata mapping Free data hosting (with automatic APIs) to serve the majority of agencies that can't easily publish data Education of the business case for agencies A committed central technical team – agency support, platform functional improvements, docs A distributed publishing model where agencies manage their own data linked or hosted on data.gov.au. Constant public success and communication Modular, iterated technical design, scalable on the cloud

Future data.govt.nz options Simple catalogue only Currently live Consistent catalogue Tabular data hosting Automated APIs Basic Stats Basic Charts & Mapping Beta Future Options Consistent catalogue Tabular data hosting Automated APIs Analytics & Reporting Basic Charts & Mapping Metadata Mapping Data Services Distributed Publishing Quality Frameworks Collaborative Strategy AoG Data Vision Data Models Sharing Part of Services Delivery Additional Data Hosting As Required High Volume Charging Framework Artifacts Sharing Hub Guidance And Shared Resources Scalable Cloud AoG Data Practitioner Community Of Practice Cross Agency Hackfests Practitioner Governance Group Exemplar Agency Projects Business Benefits Case Studies Sustainable Reuse Projects

Parallel Initiatives Needed To Drive Change Data reuse projects to demonstrate value and drive appetite Eg - Catalyst projects, FigureNZ Development of data driven culture across govt Eg - growing demand from policy teams, investment, services delivery AoG holistic data publishing, sharing, skills and capability improvements Eg – data.govt.nz, LINZ, Statistics Specialist data programs and integration initiatives for specialised outcomes Eg – SIU, DFP, LINZ, Statistics, IDI