1 Build EPAs BP Oil Spill Data Tools in the Cloud Brand Niemann US EPA July 10, Disclaimer: These slides do not reflect the views of the U.S. Environmental Protection Agency and does not constitute endorsement by the EPA of the standards or products mentioned.

2 Overview The Challenge The Socrata and Google Earth Programs The Expert and His Advice The Cloud Tools The Inspiration The Data Sources Other Sources of Data The Process The Results Comments Acknowledgements References Footnotes

3 The Challenge BP Oil Spill Data Tools: –As part of our response to the ongoing BP oil spill in the Gulf of Mexico, weve been analyzing environmental conditions, including air, water, sediment, and oily wastes. Right from the beginning, weve followed our open government approach, which means providing the data as soon as we have it. Weve posted data in CSV files, a format that people could download and open in a spreadsheet. Weve also provided printable PDF tables and summaries on our BP spill site.BP spill site –But we knew that people needed other ways to get the data, and weve been working on several options. We launched Google Earth a few weeks ago, and today we launched Socrata. –Source: Jeffrey Levy, EPAs Director of Web Communications, Greenversations, July 8, Greenversations

4 The Challenge BP Oil Spill Data Tools (continued): –Im excited to share these tools, but we can always improve them. In the comments section of this post, please give us your suggestions. Some examples: Filtered views to provide beyond just what weve detected Different ways of sorting the data Mashups (ways to combine the data with other information) –And if you create your own views or download the data and produce interesting stuff, let us know! –Source: Jeffrey Levy, EPAs Director of Web Communications, Greenversations, July 8, 2010.Greenversations

5 The Socrata Program See next two slides

6 The Socrata Program

7 The Socrata Program See next slide

8 The Socrata Program Download data

9 The Socrata Program Also download data Metadata

10 The Google Earth Program See next slide

11 The Google Earth Program Click to download File in Google Earth. Note: Need to install Google Earth client first.

12 The Google Earth Program in Google Earth

13 The Expert and His Advice Edward Tufte Presidential appointment announced by White House, March 5, announced by White House Tufte Comment on iPhone interface design: Better to have users looking over material adjacent in space within our eyespan rather than stacked in time. This is especially the case for statistical data, where the fundamental analytical task is to make comparisons. Also see page 159 in the book reference below.Comment on iPhone interface design –Edward R. Tufte, Beautiful Evidence (2006), Graphics Press LLC.Graphics Press LLC

14 The Cloud Tools

15 The Cloud Tools

16 The Cloud Tools

17 The Cloud Tools

18 The Inspiration H1N1 Spread Courtesy of TIBCO Spotfire. See Web Player.Web Player

19 The Data Sources

20 Other Sources of Data NOAA interactive map tracks Gulf oil spill Online mapping tool allows public to follow closures, spill trajectory The public can now track Deepwater Horizon-BP oil spill recovery data online via a near-real-time interactive map at a new Web site created by the National Oceanic and Atmospheric Administration. The GeoPlatform Web site, launched June 15, includes regularly updated geospatial data from several federal and state agencies on the oil spill trajectory, closed fishery areas, impact on wildlife and Gulf resources, daily position of research ships, and affected shorelines.GeoPlatform Web site Data published on the site is received from NOAA, the U.S. Coast Guard, the Environmental Protection Agency, the U.S. Fish and Wildlife Service, the U.S. Geological Survey, the Homeland Security Department, NASA and several states. This Web site provides users with an expansive, yet detailed geographic picture of whats going on with the spill, Jane Lubchenco, NOAA administrator, said in the news release June 15. Its a common operational picture that allows the American people to see how their government is responding to the crisis. NOAA worked with the University of New Hampshire's Coastal Response Research Center to develop the Web-based geospatial platform designed for federal and local response activities and adapted for public use. A separate public Web site, Deepwater Horizon Response sponsored by federal agencies, has been operating for several weeks offering news, announcements and information about the disaster. It is operated by the Deepwater Horizon Unified Command, which consists of DHS and the Defense and Interior departments, as well as other federal agencies, BP and other private entities.

21 Other Sources of Data

22 Other Sources of Data

23 The Process The Basic Steps: –Inventory Data Sources and Plan Application –Prepare and Import Data and Metadata –Implement Layout and Analytics –Add Bookmarks and Create Data Stories –Publish and Test in Web Player –Get Feedback and Improve First create visualizations, faceted search (filters), and analytics for each individual data source and then look for relationships between the data sources.

24 The Results Spotfire on the PC

25 The Results Spotfire on the PC

26 The Results Spotfire on the PC

27 The Results Spotfire Webplayer

28 Comments The initial objective to see how fast one could create this basic application using Spotfire's Multiple Visualizations (Cross Table, Table, and Scatter Plot) and Hyperlinks (Between Multiple Tabs and to the Web) and mash it up with the the NOAA GeoPlatform layers. It provides the same results as Socrata's More Views and builds on the NOAA GeoPlatform layers by adding statistical visualizations of the data (see The Challenge and The Socrata and Google Programs previously). Please use the Add Comment feature at the bottom of this wiki page to provide feedback and suggest additional analyses you would like to see. To use the Add Comment feature you first need to register by providing your address. Your privacy will be respected and your address will not be available to others or used for any other purpose. You can also download the Spotfire File (about 2 MB) from this Wiki and a 30-day free evaluation copy from and reuse these analyses, add your own data to this file or new Spotfire files that you create. Have fun and give us your feedback!

29 Acknowledgements The author acknowledges gratefully Dean Allemang, Cory Casanave, Sean Connors, Mills Davis, Li Ding, David Eng, Lee Feigenbaum, Aaron Fulkerson, Jim Hendler, Ralph Hodgson, Kevin Kirby, Kevin Jackson, Bob Marcus, John McMahon, Richard Murphy, Brand Niemann, Jr., Barry Nussbaum, Matthew Phoenix, Tony Shaw, Jeff Stein, George Strawn, George Thomas, Pete Tseronis, and Edward Tufte.

30 References Brand L. Niemann, Put Your Desktop in the Cloud to Support the Open Government Directive and Data.gov/semantic, Semantic Universe, April 19, 2010.Semantic Universe Brand L. Niemann, Build Your Own Data.gov (Spotfire) and EPA Microsite (Spotfire) with Semantics and Statistics in the Cloud, Slides, May 15, 2010.Own Data.govSpotfireEPA MicrositeSpotfireSlides Brand L. Niemann, Build Your Community Health Information "Design for America" Using Mindtouch and Spotfire, Slides, May 17, MindtouchSpotfireSlides Brand L. Niemann, Build EPA's CASTNET in less that two hours, see Mindtouch and Spotfire, Slides, May 21, Completed in 8 more hours by May 30, 2010.CASTNETMindtouch SpotfireSlides Brand L. Niemann, Build Your Own Data.gov/semantic with Mindtouch and Spotfire in the Cloud: The White House Visitor Database, Slides, May 22, See Data.gov takes the 'Mumsy' test, FCW, May 26, See Mindtouch and Spotfire, Updated July 7, 2010 (see below). Compare to Findthebest's WH Visitor Logs Comparison Tool.MindtouchSpotfireSlidesFCWMindtouchSpotfireFindthebest's WH Visitor Logs Comparison Tool Brand L. Niemann, Build EPA's EPA's Facility Registry System (FRS) and Locational Reference Database with Mindtouch and Spotfire in the Cloud: Virginia, No Slides, June 1, 2010.MindtouchSpotfire Brand L. Niemann, Build the UKs COINS in the Data Science Library Cloud (Mindtouch and Spotfire), Slides, June 9, 2010.Mindtouch SpotfireSlides Brand L. Niemann, Build EPA's Envirofacts in the Cloud: Virginia FRS, NPL, and TRI (Mindtouch and Spotfire), Slides. June 14, 2010.MindtouchSpotfireSlides Brand L. Niemann, Build the SemTech 2010 in the Cloud (Mindtouch and Spotfire), No Slides, July 2, 2010.MindtouchSpotfire Brand L. Niemann, Build the White House Staff Salaries in the Cloud (Mindtouch and Spotfire). Slides, July 3, 2010.Mindtouch SpotfireSlides

31 Footnotes 1 From: Jeffrey To: Date: Thursday, July 08, :21PM Subject: We're using Google Earth and Socrata to provide data about the oil spill –Hi everyone. On our BP oil spill site, we've been providing CSV and PDF files from the beginning. We're working on some enhancements to our database's capabilities, too, especially around providing context and summaries. –But we're also using two tools to expand how people can access and download our data, epsecially raw data in the spirit of open government. –Here's the blog post I just wrote about it: Jeffrey Levy Director of Web Communications Office of Public Affairs U.S. Environmental Protection Agency (cell for emergencies: ) EPA's blog: | Twitter: EPA news: | mobile: | Twitter:

32 Footnotes 2 EPA's Work on the Gulf Coast –