1 Data Quality Plan Pilot Tutorial: EPA Report on the Environment Scientific Data Management Workshop Planning Group Brand Niemann Senior Enterprise Architect U.S. EPA March 13,

2 Overview 1. Background 2. EPA Report on the Environment –2.1 EPA’s Web and Electronic Versions –2.2 My First Semantic Publishing Version (2004) –2.3 My Second Semantic Publishing Version (2007) –2.4 My Third Semantic Publishing Version (2010) 3. Suggestions 4. Questions and Answers

3 1. Background 1.1 Open Government Directive 1.2 Data Quality Viewpoints: –1.2.1 Federal Data Architecture Subcommittee –1.2.2 Data Management Association –1.2.3 Interagency Working Group on Digital Data –1.2.4 Ken Orr –1.2.5 Michael Daconta –1.2.6 Mills Davis 1.3 Strategy 1.4 History

4 1. Background 1.1 Open Government Directive, December 8, 2009: –Implement the principles of transparency, participation, and collaboration set forth in the President’s Memorandum (January 21, 2009): 1. Publish Government Information Online: –Publish online in an open format at least three high-value data sets*. –Create an Open Government Webpage*. 2. Improve the Quality of Government Information: –Make certain that the information conforms to OMB guidance on information quality. 3. Create and Institutionalize a Culture of Open Government: –Develop and publish on its Open Government Webpage an Open Government Plan*. »Flagship Initiative: Each agency’s Open Government Plan should describe at least one specific, new transparency, participation, or collaboration initiative*. 4. Create an Enabling Policy Framework for Open Government: –Help OMB identify impediments to open government and to the use of new technologies. * Done by this pilot as of March 13, 2010.

5 OMB Guidance on High-value Information Information that can be used to increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation.

6 OMB Guidance on Information Quality The Federal Government has defined quality and objectivity: –Quality is “…the encompassing term, of which ‘utility,’ ‘objectivity,’ and ‘integrity’ are the constituents. –‘Objectivity’ focuses on whether the disseminated information is being presented in an accurate, clear, complete, and unbiased manner, and as a matter of substance, is accurate, reliable, and unbiased. Source: “Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies” (67 FR 8452).

7 1. Background 1.2 Data Quality Viewpoints: –1.2.1 Federal Data Architecture Subcommittee: On Authoritative Data Sources: One Data Element at a Time, March 11, 2010, Dr. Richard Wang, Chief Data Quality Officer, Office of the U.S. Army CIO/G-6, and Director, MIT Information Quality Program: –Conventional view: Data Quality = Accuracy –New research finding: Data Quality Goes Beyond Accuracy.

8 1. Background 1.2 Data Quality Viewpoints: –1.2.1 Federal Data Architecture Subcommittee: On Authoritative Data Sources: One Data Element at a Time (continued): –Four Principles: »1.Understand the consumer’s information needs »2.Manage information as the product of a well- defined production process »3.Manage information as a product with a life cycle »4. Appoint an Information Product Manager to manage the information product.

9 1. Background 1.2 Data Quality Viewpoints: –1.2.1 Federal Data Architecture Subcommittee: On Authoritative Data Sources: One Data Element at a Time (continued): –Solution: »1. Address one data element at a time using priority data elements within priority projects. »2. Take a first few data elements through the entire TDQM cycle to educate and illustrate value. »3. Establish and populate a catalog of data element quality specifications containing priority data elements for broad use.

10 1. Background 1.2 Data Quality Viewpoints: –1.2.2 Data Management Association: One of the Ten Data Management Functions in the DAMA-DMBOK Functional Framework V3 are:DAMA-DMBOK Functional Framework V3 –Data Quality Management: »Planning, implementation and control activities that apply quality management techniques to measure, assess, improve and ensure the fitness of data for use.

11 1. Background 1.2 Data Quality Viewpoints: –1.2.3 Interagency Working Group on Digital Data: Harnessing the Power of Digital Data for Science and Society, January 2009:Harnessing the Power of Digital Data for Science and Society –The report lays out a strategic vision for “a digital scientific data universe in which data creation, collection, documentation, analysis, preservation, and dissemination can be appropriately, reliably, and readily managed, thereby enhancing the return on our nation’s research and development investment by ensuring that digital data realize their full potential as catalysts for progress in our global information society.”

12 1. Background 1.2 Data Quality Viewpoints: –1.2.3 Interagency Working Group on Digital Data: Harnessing the Power of Digital Data for Science and Society, January 2009:Harnessing the Power of Digital Data for Science and Society –The report includes three key recommendations to pursue this vision. The first is to create an Interagency Subcommittee under NSTC that will focus on goals that are best addressed through continuing broad cooperation and coordination across agencies. The second key element of the strategic framework is for departments and agencies to lay the foundations for agency digital scientific data policy and make the policy publicly available. In laying these foundations, agencies should consider all components of a comprehensive policy to address the full data management life cycle. The third key element is for all agencies to promote a data management planning process for projects that generate scientific data for preservation.

13 1. Background 1.2 Data Quality Viewpoints: –1.2.3 Interagency Working Group on Digital Data: Subgroups on Data Management Plans and Agency Science Data Policies March 29-31, 2010, NIST, Gaithersburg, Maryland, 1st International Digital Preservation Interoperability Framework Symposium1st International Digital Preservation Interoperability Framework Symposium –Scientific research is generally held to be of good provenance when it is documented in detail sufficient to allow reproducibility. (Source: Wikipedia)Scientific researchprovenancereproducibility June 29-July 1, 2010 Workshop on Scientific Data Management, US EPA Headquarters, Federal Triangle, Washington, DC

14 1. Background 1.2 Data Quality Viewpoints: –1.2.4 Ken Orr:Ken Orr Data quality can only be maintained in the long run by ensuring continuous and intensive use of both the data and the metadata (data about the data). “Use-based design means focusing on exactly how the data will be used and trying to identify inventive ways to ensure that the data are used more strenuously. In many cases, this means creatively persuading the people most knowledgeable about the data to take responsibility for it.” Taking responsibility for the data is not simply taking the blame when they are wrong, but making the data so central to one’s real job that its quality becomes important for day-to-day work (1998).

15 1. Background 1.2 Data Quality Viewpoints: –1.2.5 Michael Daconta, March 10, 2010, 10 flaws with the data on Data.gov: Recently released high-value datasets reveal 10 types of deficiencies:March 10, Omission errors. 2. Formatting errors. 3. Accuracy errors. 4. Incorrectly labeled records. 5. Access errors. 6. Poorly structured data. 7. Non-normalized data. 8. Raw database dumps. 9. Inflation of counts. 10. Inconsistent data granularity. –So what can we do? Here are three basics steps: Attract more citizen involvement to police the data; implement the top ideas on and ensure agency open-government plans address, in detail, their data quality processes.

16 1. Background 1.2 Data Quality Viewpoints: –1.2.6 Mills Davis, February 9, 2010, Second Open Government Deadline: US Agencies Still Flexing Their Muscles:February 9, 2010 The National Renewable Energy Laboratory (NREL) is developing an Open Energy Information ( initiative. Its goal is to establish a (US and global) community-based “renewable energy data commons” based on linked open data principles. Right now they’re working through issues relating to how best to manage data quality across the lifecycle of data assets contributed from multiple sources. Entities aligned with Federal Agencies and Departments have multiple requirements to meet relating to quality of data and information being made public in addition to the principles for data quality principles enunciated as part of Data.gov. What is new is figuring out how to best frame data policies and align practices and procedures to fully take advantage of semantic web technologies. I doubt that NREL is the only group that is exploring how best to marry semantic web technologies with appropriate data quality policies, best practices, and procedures. For example, the UK has taken this path. Also, the NASCIO organization is recommending linked open data for States to share public information.

17 1. Background 1.3 Strategy: –Bridge across: The Federal Cloud Computing Communities The Interagency Working Group on Digital Data W3C Special Interest Group on eGov (Linked Open Data) EPA Statistical, OGD and Wiki and Blog Work Groups –Tie Scientific Data Management (SDM) to the OGD and Data.gov/semantic: By implementing my “Put Your Desktop in the Cloud in Support of the OGD and Data.gov/semantic” (see Section 1.4) –Use EPA’s “best practice” for SDM, the Report on the Environment 2008: Complete metadata, peer review, provenance, updating, etc. –Help Subject Matter Experts use Semantic Publishing Technologies: Build an ontology (see next slide) Implement the various Data Quality Viewpoints –The Third International Provenance and Annotation Workshop, June 15-16, 2010.The Third International Provenance and Annotation Workshop Facilitate access and reuse Try new tools (e.g. Sheetster, Open Data Registry, etc.)

18 1. Background 1.3 Strategy: –Ontology, as used here, means categories of interest in a domain and the relationships among them (note this is independent of syntax and technology) Source: John Sowa, Knowledge Representation, A recent book entitled "Semantic Web for the Working Ontologist: Effective Modeling in RDFS and OWL contains an excellent example of the application of ontology to Enterprise Architecture, specifically, the Federal Enterprise Architecture Reference Model Ontology (FEARMO) (see pages ), as well as many other practical examples.Semantic Web for the Working Ontologist

19 1. Background 1.3 Strategy: –That given a data URI, there is no standard way to programmatically access the metadata about the resource. Josh Tauberer: The Linked Data 'movement' started, as I recall, in the use of URIs as the convention for all RDF resources. By using there's implicitly a web page that can be accessed for more information about that resource --- a page that can be in RDF format itself.Josh Tauberer –That given a data URI there is no standard programmatic way to access a SPARQL query point for that resource and/or for associated resources. Josh Tauberer: See Taubererhttp://sw.deri.org/2007/07/sitemapextension/ –That the metadata accessed should have standard links for provenance – even very simple provenance that does not require research. Brand Niemann: See this pilot! Source: W3C eGov Projects Government Linked Data: Demonstration Discussion, March 12, 2010.W3C eGov Projects

20 1. Background 1.3 Strategy: –How do we contextualize a query such that all data resources of interest within a certain context are included in a query, without the user having to know all the details of the data sets involved? No response yet. –All of the above could be accomplished with URI conventions and supporting ontologies. My question is: What are the existing or proposed conventions and ontologies to satisfy these requirements? Should the eGov group provide or reference such conventions for use by the government and/or within our government demos? Brand Niemann: See this pilot! Source: W3C eGov Projects Government Linked Data: Demonstration Discussion, March 12, 2010.W3C eGov Projects

21 1. Background 1.4 History: –October 28, 2009: Federal IT Summit Cloud Computing Panel: EPA work on "a cloud computing desktop" cited as a best practice by Peter Tseronis, Co-Chair, Cloud Computing Advisory Committee. –December 10, 2009: Data Architecture Subcommittee Meeting with Donna Roy (NIEM) and Jim Hendler (Data-gov Semantic Wiki) and discussions of Data.gov CONOPS and its evolution to the Semantic Web. –December 15, 2009: Semantic Web Meetup: Party Cloudy with a Chance of Semantics. (Put Your Desktop in the Cloud to Support the Open Government Directive and Data.gov/semantic!). –December 17, 2009: Federal Cloud Computing Advisory Council Meeting (same as above). –December 24, 2009: 2010 Office of Environmental Information National Symposium, May 11 – 13, 2010, Proposal for Presentation and Tutorial.

22 1. Background 1.4 History: –January 11, 2010: Open Government Directive Workshop and Government Desktop in the Cloud Breakout Session at the US Department of Transportation at the Direction of the Office of the EPA Administrator and the National Center for Innovation. –January 14-15, 2010: Federal CIO Council Boot Camp Training, Partnership for Public Service SAGE Program, Washington, DC. My Suggested 2010 Plan for an Open Government Directive Roadmap Using Cloud Computing Desktop / Mobile Apps as a Member of the Federal CIO Council's Best Practice Committee. –January 19-21, 2010: EPA Enterprise Architecture Strategic Planning Sessions –January 21, 2010: Put Your Statistical Work in the Cloud in Support of the Open Government Directive and Data.gov/semantic (EPA Statistics Users Group). –January 28, 2010: Tutorial for the EAWG: Solution Architecture for 2010.

23 1. Background 1.4 History: –February 4, 2010: Design Suggestions for EPA’s One Wiki in Support of the EPA OGD Work Group. –March 3, 2010: Joint Services / Cloud Computing Session: Semantic Cloud Computing and Linked Open Data Discussion of Three Uses Cases. –March 3rd, 2010: For Open Government, Technology Is The Least of Your Problems (Gartner Blog Comment). –March 4, 2010: 2010 Annual Statistical Abstract of the United States: More than 1300 high- value data sets and graphs with standard metadata, and EPA Report on the Environment Indicators: More than 200 high-value data sets and graphs with standard metadata. –March 6, 2010: Flagship Initiative Pilot Tutorial: EPA Report on the Environment, EPA Wiki and Blog Work Group.

24 2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions Home Page Chapters

25 2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions Chapters (expanded)Indicator (Question)

26 2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions Charts and Graphs References

27 2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions MetadataOutput to PDF

28 2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions Results (no highlighted terms)Search

29 2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions Report on the Environment: – Chapters: – Indicators: – yChapter&r=209838http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewInd&ch=46&lShowInd=0&subtop=341&lv=list.listB yChapter&r= Charts and Graphs: – listByChapter&r=209838http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewMidImg&ch=46&lShowInd=0&subtop=341&lv=list. listByChapter&r= Download Spreadsheets (xls): – – References: – ist.listByChapter&r=209838http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewReference&ch=46&lShowInd=0&subtop=341&lv=l ist.listByChapter&r= Metadata: – tByChapter&r=209838http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewMeta&ch=46&lShowInd=0&subtop=341&lv=list.lis tByChapter&r= Create PDF: – ByChapter&r=209838http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewPDF&ch=46&lShowInd=0&subtop=341&lv=list.list ByChapter&r= Search: – ubmit=Go&fld=nceawww1|eroe&areaname=EPA+Report+on+the+Environment&areacontacts= pa.gov/ncea/cfm/nceacontact.cfm&areasearchurl=&result_template=epafiles_default.xsl&filter=sample4filt.ht shttp://nlquery.epa.gov/epasearch/epasearch?typeofsearch=area&querytext=carbon+monoxide+emissions&s ubmit=Go&fld=nceawww1|eroe&areaname=EPA+Report+on+the+Environment&areacontacts= pa.gov/ncea/cfm/nceacontact.cfm&areasearchurl=&result_template=epafiles_default.xsl&filter=sample4filt.ht s

30 EPA Report on the Environment 2.2 My First Semantic Publishing Version (2004) Home Page Table of Contents

31 EPA Report on the Environment 2.2 My First Semantic Publishing Version (2004) There have been a number of efforts to integrate environmental information and indicators during the past 20 years or so. The author has been involved in at least eight of those efforts and now has integrated examples from all of those into the re-design of the Sustainable Development Indicators Web Site which has enjoyed sustained popularity with search engines over the years. –Integration of Environmental Information and Indicators, December 17, 2007.Integration of Environmental Information and Indicators

32 EPA Report on the Environment 2.3 My Second Semantic Publishing Version (2007) Structure: –Introduction –What the Data Show –Limitations –Data Sources –References –Downloads –Metadata Introduction

33 EPA Report on the Environment 2.3 My Second Semantic Publishing Version (2007) What the Data Show Limitations

34 EPA Report on the Environment 2.3 My Second Semantic Publishing Version (2007) Data Sources References

35 EPA Report on the Environment 2.3 My Second Semantic Publishing Version (2007) MetadataDownloads (did not have xls’s)

36 EPA Report on the Environment 2.3 My Second Semantic Publishing Version (2007) Expert SearchResults

37 EPA Report on the Environment 2.3 My Second Semantic Publishing Version (2007) Introduction: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 What the Data Show: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 Limitations: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 Data Sources: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 References: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 Downloads: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 Metadata: – j.htm&f=templates&2.0http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main- j.htm&f=templates&2.0 Search: – frames.htm&id= /ExpertSearch& frames.htm&id= /ExpertSearch&

38 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Home Page Chapter

39 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Indicators Structure

40 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) IntroductionWhat the Data Show

41 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Limitations Charts and Graphs

42 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Data Sources (Provenance)References

43 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) DownloadsMetadata

44 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Site MapSearch Results

45 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Home Page: – Chapter: – Indicators: – ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicatorshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_The ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators Structure: – ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissionshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_The ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions Introduction: – ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions/ _Introductionhttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_The ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions/ _Introduction What the Data Show: – ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions/ _What_the_Data_Showhttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_The ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions/ _What_the_Data_Show Limitations: – ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions/ _Limitationshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_The ir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Car bon_Monoxide_Emissions/ _Limitations

46 EPA Report on the Environment 2.4 My Third Semantic Publishing Version (2010) Charts and Graphs: – Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Charts_and_Graphshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_ Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Charts_and_Graphs Data Sources: – Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Data_Sourceshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_ Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Data_Sources References: – Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Referenceshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_ Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _References Downloads: – Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Downloadshttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_ Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Downloads Metadata: – Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Metadatahttp://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_ Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/ _Carbon_Monoxide_Emissions/ _Metadata Site Map: – Search Results: –

47 3. Suggestions The Scientific Data Management Workshop can: –Tie Scientific Data Management (SDM) to the OGD and Data.gov/semantic. –Construct a set of well-defined URIs/URLs for their best content like the third semantic publishing example. –Use new Semantic Publishing tools to deliver their content to the Linked Open Data Web of the W3C. –Deal with Data Quality by addressing the viewpoints contained in this tutorial. –Try to produce the three uses cases: basic, intermediate, and advanced described elsewhere.elsewhere

48 4. Questions and Answers