Government Challenges With Big Data: A Semantic Web Strategy for Big Data Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ January 24, 2013 Link

Outline Data Science Team Semantic Community 2013 Mission Statement Why We Are Here NIST Cloud Computing AND Big Data Forum and Workshop Spotfire for Big Data Analytics and Data Science Analytics Library From the Year of Big Data to the Year of the Data Scientist Working With Big Data Cross-Walk Table (in progress) The Practice of Data Science Current US Government Semantic Web Strategy International Linked Open Data Strategy Our Semantic Web Strategy for Big Data My 5-Step Method To Get to 5-Stars With Open Data System of Systems Architecture Data Federation in Spotfire 15th SOA, Shared Services, and Big Data Analytics Conference (DRAFT) Comments: Semantic Medline, Noblis, Cray, and ORBIS Technologies Q & A

Data Science Team Dr. Brand Niemann, Director and Senior Data Scientist, Semantic Community Dr. Tom Rindflesch, Research Group Lead for Semantic Medline, National Library of Medicine Dr. Victor Pollara, Senior Principal Scientist, Noblis Dr. Eric Little, Director of Information Management, Orbis Technologies Mark Guiton, Director,  Government Relations, Cray Inc. Cray-YarcData announced semifinalists last month http://www.yarcdata.com/press-release-12-3-12.html

Semantic Community: Mission Statement for 2013 Help the Data Transparency Coalition help the 113th Congress with the re-introduction of the Data Act by Building the Federal Financial Information Network in the Cloud for the 113th Congress, January 4, Slides. Continue to work with Big Data Analytics (e.g. Recorded Future, Spotfire, etc.), Content Analytics and Knowledge Management (e.g. MindTouch), and Semantic Technologies (e.g. Be Informed, Semantic Insights, etc.) for data science and data journalism. Slides. Help start Open Government Data for Japan (and the US and Europe) with the Right Data (Statistical) with the Right People (Data Scientists) Working on the Right Business Problems (Return on Investment): January 21, Slides. Help the Federal Big Data Senior Steering Group with A Semantic Web Strategy for Big Data and to move From the Year of Big Data to the Year of the Data Scientist Working With Big Data, January 24, Slides. Help the ACT-IAC AMWG, C&T SIG, and ET-SIG with Big Data on Mobile Devices, Collaboration and Transformation, & Government Challenges With Big Data , January 16 and February 23, Slides. http://semanticommunity.info/#Welcome_to_Semantic_Community.info:_Community_Infrastructure_Sandbox_for_2013

Why We Are Here Killer Semantic Web Application: Audit Function: George Strawn and Tom Rindflesch Data Architecture (5 Steps to Get to 5 Stars and Systems of Systems) Audit Function: Brand Niemann, Data Science Products at the US EPA and then for AOL Gov, etc. Todd Park, Data.gov and Communities (Health, Energy, Safety, and Education) Chris Greer, New Cloud AND Big Data Web Sites Using Science (and Statistics) as Evidence in Public Policy: Kenneth Prewitt, NRC Committee Chair, and Former Census Bureau Director Connie Citro, NAS National Committee on Statistics Director (CNSTAT) Niall Brennan, Director of the Office of Information Products and Data Analytics at the Centers for Medicare and Medicaid Services, and the Institute of Medicine (IOM) Study on Geographic Variation in Health Care Spending and Promotion of High-value Care Workforce Education: Professor Peter Fox, RPI, Individual and Group Projects Professor Jeffrey Hammerbacher, UC Berkley and Cloudera Chief Scientist, Exploratory Data Analysis/Visualizations and Modeling Professor Kirk Borne, GMU, Presentation to BDSSG Last September Professor Michael Joseph, Big Data Pilot for First Year Engineering and Computer Science Students Last Semester

NIST Cloud Computing AND Big Data Forum and Workshop Microscopes and Telescopes for Big Data, Professor Alexander Szalay My Note: Spotfire 5 Application Scalable Parallel Interoperable Data Analytics Library (SPIDAL), Professor Geoffrey Fox Large scale data analytics on clouds Keynote in CloudDB '12 Proceedings of the Fourth International Workshop on Cloud Data Management Pages 21-24, October 29, 2012. Interoperability Between Clouds, Alan Yoder My Note: MindTouch and Spotfire 5 Application http://semanticommunity.info/Emerging_Technology_SIG_Big_Data_Committee/Cloud_Computing_AND_Big_Data_Forum_and_Workshop_January_15-17_2013

Spotfire for Big Data Analytics: Microscope NASA GCMD: Gateway to Big Data NSF Big Data Awards: Follow the Work OSTP Harnessing The Power of Digital Data Report: Well-Defined URLs PCAST Designing a Digital Future Report: Interoperability Interface NITRD Dashboards: Live Demonstrations Four Clicks: See, Sort/Search, Download, & Share (iPad) http://semanticommunity.info/Emerging_Technology_SIG_Big_Data_Committee/Government_Challenges_With_Big_Data#Spotfire_Dashboard

Data Science Analytics Library: Telescope & Library Live Links to Outside Data Sources Live Information Links Between Analytics https://silverspotfire.tibco.com/us/library#/users/bniemann/Public

From the Year of Big Data to the Year of the Data Scientist Working With Big Data Administration Announcement NSF Leads Federal Efforts TechAmerica Report Case Studies My Big Data at the Hill Report My Cross-Walk Table NASA Big Data Challenge Series: “How to make heterogeneous data seem more homogeneous”. The Practice of Data Science http://semanticommunity.info/Emerging_Technology_SIG_Big_Data_Committee/Government_Challenges_With_Big_Data#Story

Cross-Walk Table (in progress) http://semanticommunity.info/Emerging_Technology_SIG_Big_Data_Committee/Government_Challenges_With_Big_Data#Cross-Walk_Table

The Practice of Data Science Josh Wills, Data Scientist @Cloudera, writing on The Practice of Data Science said: Key trait of all data scientists. Understanding “that the heavy lifting of [data] cleanup and preparation isn’t something that gets in the way of solving the problem: it is the problem.” (DJ Patil) Inverse problems. Not every data scientist is a statistician, but all data scientists are interested in extracting information about complex systems from observed data, and so we can say that data science is related to the study of inverse problems.   Real-world inverse problems are often ill-posed or ill-conditioned, which means that scientists need substantive expertise in the field in order to apply reasonable regularization conditions in order to solve the problem. Data sets that have a rich set of relationships between observations. We might think of this as a kind of Metcalfe’s Law for data sets, where the value of a data set increases nonlinearly with each additional observation. For example, a single web page doesn’t have very much value, but 128 billion web pages can be used to build a search engine. Open-source software tools with an emphasis on data visualization. One indicator that a research area is full of data scientists is an active community of open source developers.

Current US Government Semantic Web Strategy Data.gov Advocates RDFa 1.1 Lite for Semantic Web Strategy. See Comment From Owen Ambur on Next Slide. I believe there is a better way to handle this that I showed the W3C eGov Special Interest Group on January 21st and have recommended for the reintroduction of the Data Act to the 113th Congress. Create a Semantic Index of Strong Relationships (SR) in RDF Format in a Spreadsheet. See next slide for example (spreadsheet and words) Integrate That With Other Spreadsheets and Relational Databases in An Interoperability Interface (e.g. Dashboard) That Can Searched. Essentially: Computer Scientists Use RD2RDF (James Hendler) Data Scientists Use SR2Excel2RDF (Brand Niemann)

Comment From Owen Ambur OMB's official guidance to agencies on implementation of section 10 of the GPRA Modernization Act (GPRAMA) says they may use XML, JSON, spreadsheets or CSVs in order to meet the requirement to publish their strategic and performance plans and reports in machine-readable format... but not PDF or HTML -- at least not without "enhanced structural elements".[1] I couldn't help but chuckle at how [1] is a PDF. I get your point however, which I think reinforces mine, that there is no US federal policy that prefers RDFa 1.1 over HTML Microdata for publishing metadata in HTML. [1] RDFa Lite 1.1, W3C Recommendation, June 7, 2012, Manu Sporny, editor, see http://www.w3.org/TR/rdfa-lite/ Source: Owen Ambur, December 18, 2012, W3C eGov Mailing List.

International Linked Open Data Strategy: Linked Open Data Cloud Data My Question: Is it easy to add columns for who links to who? Answer: Not in a single table. SPARQL can't do cross-tabulation (Richard Cyganiak). http://semanticommunity.info/@api/deki/files/8824/=VIVO.xlsx

International Linked Open Data: Comments to David Wood The Linked Open Data Cloud is not actually “linked data”. RDF at Data.gov is not linked data. The analytical and statistical communities view Data.gov and Linked Open Data as “IT projects”. Former Census Bureau Director Robert Groves. Conventional tools can do linked data and data integration. Spotfire Information Designer, Informatica, Information Builders, etc. http://manning.com/dwood/LinkedData_MEAP_ch1.pdf http://semanticommunity.info/AOL_Government/Exploiting_Linked_Data_with_BI_Tools

International Linked Open Data: My EPA Green App Data App Example https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?EPAGreenAppsDataApp-Spotfire

Our Semantic Web Strategy for Big Data: Previous Presentations MITRE: October 2, 2012 GSA: November 29, 2012 http://semanticommunity.info/Emerging_Technology_SIG_Big_Data_Committee#A_Semantic_Web_Strategy_for_Big_Data

Our Semantic Web Strategy for Data: Simple Explanation One Table: Two Columns Example: Column 1: Section and Column 2: URL Note: A Column 3: Description could be in the URL Example: See Next Slide Three Columns: Example: Column 1: Subject, Column 2: Object, and Column 3: Predicate Note: This is the Semantic Web’s Linked Open Data Cloud as Linked Open Data for Network Analytics! Example: See Semantic Medline Four Columns: Examples: Column 1: Subject, Column 2: Attribute, Column 3: From, and Column 4: To, or Column 1: City, Column 2: Country, Column 3: Longitude, and Column 4: Latitude Note: This is the format for Spotfire’s Network Analytics Module developed for the CIA

Our Semantic Web Strategy for Data: NASA Big Data Example http://semanticommunity.info/@api/deki/files/20313/NASABigData.xls

Our Semantic Web Strategy for Data: Spotfire Network Analytics http://semanticommunity.info/AOL_Government/Social_Media_-_Six_Degrees_of_Separation_and_Now_Even_Less

My 5-Step Method So what I like to do to illustrate (data science) and explain (data journalism) is the following (like a recipe): Put the Best Content into a Knowledge Base (e.g. MindTouch*) NASA Big Data Put the Knowledge Base into a Spreadsheet (Excel*) Linked Data to Subparts of the Knowledge Base Put the Spreadsheet into a Dashboard (Spotfire*) Data Integration and Interoperability Interface Put the Dashboard into a Semantic Model (Excel*) Data Dictionaries and Models Put the Semantic Model into Dynamic Case Management (Be Informed*) Structured Process for Updating Data in the Dashboard * Examples of tools used.

To Get to 5-Stars With Open Data Definition Example / Tool* Make your stuff available on the Web (whatever format) under an open license This Story / MindTouch Make it available as structured data (e.g., Excel instead of image scan of a table) Spreadsheet / Excel Use non-proprietary formats (e.g., CSV instead of Excel) Table / MindTouch and Spotfire Use URIs to identify things, so that people can point at your stuff Table of Contents / MindTouch and Spotfire Link your data to other data to provide context * Examples of tools used. Source of Star and Definition: http://www.w3.org/DesignIssues/LinkedData.html

System of Systems Architecture Dynamic Case Management (e.g. Be Informed) Data Science Library (e.g. Spotfire) Data Science Products (e.g. Spotfire) S Semantic Index of Linked Data (e.g. Excel)

Data Federation in Spotfire: In-Memory and In-Database Data In-Memory Data When you are working with in-memory data tables (text files, Excel files, information links, etc.) you have access to all the functionality of Spotfire. You have the opportunity to use all columns as filters and perform any number of calculations. You can also use any of the tools within Spotfire to cluster data, calculate new columns, bin columns, make predictions etc. See Working With Large Data Volumes for some tips on how to improve the performance of an analysis with lots of data. In-Database Data When a connection to an external source is set up, all calculations are done using the external system and not with the Spotfire data engine. This will allow you to work with data volumes too large to fit into primary memory and take advantage of the power of the external system. When working with external data connections, you access only the current selection of data and all aggregations and calculations are made in-database (in-db).

Data Federation in Spotfire: Database Connections, Information Links, & Analytics Library A database connection dialog is used to set up a connection to say a Teradata database, where you can analyze data from the database without bringing it into your analysis. An information link is a structured request for data which can be sent to the database. These specifications include one or more columns, and may include one or more filters. Stated in plain English, an information link could be: "Fetch the Name, Address and Phone_number for employees that pass the filter High_Income." Information links can also be used to limit what data to open in an analysis in a number of different ways. The library provides publishing capabilities for all of your analysis materials, so you can share data with your colleagues. The library can be used directly from Spotfire by anyone who has at least read privileges.

Data Federation in Spotfire: Data Panel The Data panel is used to get an overview of the columns in all data tables, in-memory as well as in-database (in-db). When working with in-database data the Data panel is the starting point for configuring both visualizations and the filters panel, since no filters are created automatically for external data. https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?NASABigData-Spotfire

Data Federation in Spotfire: Information Designer The Information Designer is a tool for setting up data sources and creating and opening information links. An information link is a database query specifying the columns to be loaded and any filters needed to narrow down the data table prior to creating visualizations in Spotfire. In Information Designer, information links are created from building blocks such as columns and filters using joins, calculations and aggregations. http://semanticommunity.info/Build_DoD_in_the_Cloud/Enterprise_Information_Web_for_Semantic_Interoperability_at_DoD/Spotfire_Data_Federation http://semanticommunity.info/Build_DoD_in_the_Cloud/Enterprise_Information_Web_for_Semantic_Interoperability_at_DoD/Spotfire_Information_Designer

15th SOA, Shared Services, and Big Data Analytics Conference (DRAFT) Jason Bloomberg, President, ZapThink, a Dovel Technologies: New Book and Poster. The Agile Architecture Revolution: How Cloud Computing, REST-Based SOA, and Mobile Computing Are Changing Enterprise IT Steven Woodward, CEO and Founder of Cloud Perspectives, Member of the Shared Services Canada Advisory Committee, Leading Contributor to the NIST FedRamp, and Canadian Cloud Council Government CTO. Shared Services Canada Geoffrey Fox, Professor of Informatics and Computing, Physics, Indiana University Large Scale Data Analytics on Clouds Dr. Christopher L. Greer, Acting Senior Advisor for Cloud Computing and Associate Director, Program Implementation Office, ITL, NIST. NIST Cloud Computing AND Big Data Forum and Workshop NITRD Interagency Big Data Senior Steering Group (BDSSG) Representative (s) What Does the Interagency Big Data Senior Steering Group Do? Peter Lyster, Program Director, Division of Biomedical Technology, Bioinformatics, and Computational Biology, National Institute of General Medical Sciences, National Institutes of Health, (NIH/NIGMS). NIH Chief Data Scientist. Adam Fuchs, Chief Technology Officer, Sqrrl Accumulo for the NSA and Sqrrl for Open Source Will Cukierski, Data Scientist, Kaggle What is a Data Scientist? Big Data Use-Cases and Pilots Data Science Teams working on Semantic Medline and Other Government Big Data Projects http://semanticommunity.info/Federal_SOA

Comments: Semantic Medline, Noblis, Cray, and ORBIS Technologies Semantic Medline, Tom Rindflesch Noblis, Victor Pollara Cray, Mark Guiton ORBIS Technologies, Eric Little

Q & A Contact Information: Brand Niemann bniemann@cox.net Tom Rindflesch trindflesch@mail.nih.gov Victor Pollara victor.pollara@noblis.org Mark Guiton mguiton@cray.com Eric Little elittle@orbistechnologies.com