A realistic look at open government data Sharon Dawes

Open data philosophy If government –publishes its data in structured, machine readable form –provides easy one-stop public access to all data from all departments –without fees or other restrictions on access or use Then –social, economic, and democratic benefits will flow to society

A built-in problem Supply-side perspective + Myths + Ambiguity = Disappointing results

Supply-side perspective Focus is mainly on what government does or should do: Adopt open data policies Devise and implement open data practices Publish the data Step aside and great things will happen

Supply-side perspective Focus is mainly on what government does or should do: Adopt open data policies Devise and implement open data practices Publish the data Step aside and great things will happen Problem 1: a government-centric view Problem 2: ignores the limits of resources and capabilities Problem 3: promises value will be created by someone else

Myths (Janssen, Charalabidis, & Zuiderwijk, 2012) Data publication = data use = public benefits All government needs to do is publish data All constituents can use published data All data should be published without restriction Open data will produce open government M. Janssen, Y. Charalabidis & A. Zuiderwijk (2012)

Myths (Janssen, Charalabidis, & Zuiderwijk, 2012) Data publication = data use = public benefits All government needs to do is publish data All constituents can use published data All data should be published without restriction Open data will produce open government Problem 1: simplistic Problem 2: naïve Problem 3: magical thinking M. Janssen, Y. Charalabidis & A. Zuiderwijk (2012)

Ambiguity About the purpose of open data: (Yu & Robinson, 2012) –Transparency and accountability (to see what the government is doing and how it is doing it) –Economic and social development (to create new products and services for society) About who is able to use open data –Rhetoric is about “citizens” –Actual users are expert analysts and application developers

Ambiguity About the purpose of open data: (Yu & Robinson, 2012) –Transparency and accountability (to see what the government is doing and how it is doing it) –Economic and social development (to create new products and services for society) About who is able to use open data –Rhetoric is about “citizens” –Actual users are expert analysts and application developers Problem 1: sends mixed messages to the public Problem 2: sends mixed messages to administrators

Result: disappointment In uptake by citizens and businesses In the type of applications produced In the sustainability and economic value of applications in the market In the effect on openness and democracy

Sidney Harris, 2012 In a nutshell...

An OGD miracle depends on whether... The policies and purposes are clear The practices are effective The data are desirable and good quality The users (analysts and developers) are capable The public wants and can use what the users create

The miracle depends on whether... The policies and purposes are clear The practices are effective The data are desirable and good quality The users (analysts and developers) are capable The public wants and can use what the users create

A different approach to the data dimension of OGD The value of open data lies in data use. This value of depends on perspective and capabilities of data users and consumers outside the government Value generated depends on the quality of the data for a given use by a given user/consumer. Consequently, there can be no one standard for data quality but rather data need to be “fit for use” (Wang & Strong, 1996, Ballou & Pazer, 1995)

Data quality challenges Conventional wisdom Provenance Practices Consequences Underuse Misuse Non-use Shifting costs and responsibilities

Conventional wisdom aka “untested assumptions” Quantitative data is “better” than qualitative data Digital data is “better” than other formats The data you need –is available and sufficient –objectively neutral –understandable –relevant for your purpose Government organizations record and organize their data in the same, predictable way

Provenance or “where do open data come from?” Administrative systems Embedded in program or service operations Governed by specific policies and laws Gathered in particular contexts for certain internal purposes By people with different kinds and levels of knowledge and expertise

Practices & processes that produce data Data definition Data collection Data management & maintenance Documentation Audit and quality control Change management Security Priorities & capabilities for all the above

Three examples

Example 1: Give me shelter

Example 2: Cadastral records

Example 3: Where does the money go?

Data quality = fitness for use Matters most from the user’s point of view Depends on the user’s purpose Usually involves trade offs –e.g., timeliness vs. completeness (Wang & Strong, 1996, Ballou & Pazer, 1995)

Dimensions of data quality AccessibilityExtent to which data is available, or easily and quickly retrievable Appropriate Amount of DataExtent to which the volume of data is appropriate for the task at hand BelievabilityExtent to which data is regarded as true and credible Completeness Extent to which data is not missing and is of sufficient breadth and depth for the task at hand Concise RepresentationExtent to which data is compactly represented Consistent RepresentationExtent to which data is presented in the same format Ease of ManipulationExtent to which data is easy to manipulate and apply to different tasks Free-of-ErrorExtent to which data is correct and reliable Interpretability Extent to which data is in appropriate languages, symbols, and units, and the definitions are clear ObjectivityExtent to which data is unbiased, unprejudiced, and impartial RelevancyExtent to which data is applicable and helpful for the task at hand ReputationExtent to which data is highly regarded in terms of its source or content SecurityExtent to which access to data is restricted appropriately to maintain its security TimelinessExtent to which the data is sufficiently up-to-date for the task at hand UnderstandabilityExtent to which data is easily comprehended Value-AddedExtent to which data is beneficial and provides advantages from its use Pipino, et al, 2002

637 pages Metadata

Data quality “tools” For open data providers Appreciate data as an asset, a source of value Adopt information policies to preserve and enhance usability Create and maintain metadata to support unknown users Adopt stewardship practices For open data users Be skeptical, ask questions Demand good quality metadata Understand the nature and context of the data Use data sets with caution Combine data sets with great caution

What else can governments do? Engage with the civic technology community about data needs and data problems Set publication priorities around known data needs of potential users and beneficiaries Support topical data communities Direct contests or challenges toward solving specific public problems (e.g. affordable housing, transportation congestion, neighborhood safety)

management technology data policy context

Thank