1. Big Data Symposium: Analytics and Applications for Federal Big Data - FEMA Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ March 5-6, 2013 http://semanticommunity.info/Big_Data_Symposia 1

2. Begin With the End in Mind Ms. Jo Strang, Associate Administrator, Safety, Federal Railroad Administration, Department of Transportation “Open Gov 2.0 and Safety.Data.Gov” – New safety data sources and challenge from the National Institute of Justice and new data from Open FEMA Open FEMA – Could not find with Google search – Start with Fema.gov http://www.fema.gov/ Safety.Data.Gov – Could not find at that Web site – Start with Bureau of Justice Statistics http://www.bjs.gov/ 2

3. My Process Found Data Feeds at Fema.gov Documented the Web Site, Metadata, and Data Sources for: – USGS Earthquakes – World Map – FEMA Disaster Declarations Summary – FEMA Hazard Mitigation Program Summary – Public Assistance Subgrantee Funded Projects Detail – Public Assistance Funded Projects Summary Used Google Earth for the FEMA and USGS KML Files Imported the Above Data Sets Into Spotfire Created a Separate Visualization Tab for Each Data Set in Spotfire Started a Unified Data Architecture Submitted a Question About the Data Definitions to FEMA Documented My Data Science Work in PowerPoint Slides Provided Conclusions and Recommendations 3

4. Fema.gov Home Page 4 http://www.fema.gov/ Data Feeds

5. Fema.gov Data Feeds 5 http://www.fema.gov/data-feeds

6. Fema.gov Data Feeds Examples RSS: – Disaster Declarations Query Form and List of Links KMZ: – Disaster Declared Counties - National View Designated Counties META and XLS: – Disaster Declarations Summary Same as RSS Disaster Declarations – Hazard Mitigation Program Summary Web Page – Public Assistance Funded Projects Detail No Web Page – Public Assistance Funded Projects Summary Web Page KML and CSV: – Worldwide M1+ Earthquakes, Past 7 Days (USGS) KMZ and TXT (No Web Page) My Note: But I found a Way Cool One! – http://earthquake.usgs.gov/earthquakes/map/ http://earthquake.usgs.gov/earthquakes/map/ 6

7. RSS: Disaster Declarations 7 http://www.fema.gov/disasters

8. META: Disaster Declarations Summary 8 http://www.fema.gov/library/viewRecord.do?id=6292 XLS

9. XLS: Disaster Declarations Summary 9 View / Download / PrintView / Download / Print (xls 9004K) Instructions Metadata

10. Hazard Mitigation Program Summary 10 http://www.fema.gov/hazard-mitigation-grant-program

11. META: Hazard Mitigation Program Summary 11 http://www.fema.gov/library/viewRecord.do?id=6293 XLS

12. XLS: Hazard Mitigation Program Summary 12 View / Download / PrintView / Download / Print (xls 16563K)

13. META: Public Assistance Funded Projects Detail 13 http://www.fema.gov/library/viewRecord.do?id=6295 XLS

14. XLS: Public Assistance Funded Projects Detail 14 View / Download / PrintView / Download / Print (xls 129914K)

15. Public Assistance Funded Projects Summary 15 http://www.fema.gov/public-assistance-local-state-tribal-and-non-profit

16. META: Public Assistance Funded Projects Summary 16 http://www.fema.gov/library/viewRecord.do?id=6299

17. XLS: Public Assistance Funded Projects Summary 17 View / Download / PrintView / Download / Print (xls 18935K) Instructions Metadata Projects 1-9 Applicants

18. Worldwide M1+ Earthquakes, Past 7 Days (USGS) 18 http://earthquake.usgs.gov/earthquakes/catalogs/eqs7day-M1.txt TXT not CSV Spotfire handles that!

19. Import All Data Sets (15) Into Spotfire 19

20. Default View in Spotfire 20 Illustrates Data Visualization Design Using Shneiderman’s Mantra: Overview First, Zoom and Filter, Then Details-on-Demand

21. KMZ: FEMA Designated Counties and USGS Worldwide M1+ Earthquakes, Past 7 Days (USGS) 21 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

22. USGS Earthquakes: Spotfire 22 My Note: It is hard to see at this scale, but the data points are all related in each visualization. My Note: Could Spotfire do this? Can USGS do what Spotfire does? https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

23. World Map and Data Tables: Spotfire 23 My Note: I needed a world map and to see how to integrate it with the Earthquake and other data tables. This Spotfire functionality helps me do this and is the beginning of a “unified data architecture”(Scott Gnau, President, Terra Data Labs). https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

24. FEMA Disaster Declarations Program Summary: Spotfire 24 My Note: The visualizations are connected! https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

25. FEMA Hazards Mitigation Program Summary: Spotfire 25 My Note: The visualizations are connected! https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

26. FEMA Public Assistance Funded Projects Detail: Spotfire 26 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

27. FEMA Public Assistance Funded Projects Summary: Spotfire 27 https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?FEMA.gov-Spotfire

28. Conclusions and Recommendations USGS Earthquakes: – Could Spotfire do this? Can USGS do what Spotfire does? Google Earth of FEMA and USGS: – Need a way to access Google Earth within Spotfire. World Map: – One can easily find and import geospatial data files (Shape) into Spotfire. Unified Data Architecture: – Spotfire provides a way to produce this by displaying all the data tables and their columns. FEMA Disaster Declarations Summary: – I submitted a question about the data definitions to FEMA and am waiting for an answer. FEMA Hazard Mitigation Program Summary: – There is county data, but the names do not match those used in the standard Shape file for county maps so I had to use the State designation. Public Assistance Subgrantee Funded Projects Detail: – Parts 1 and 2 of the Data Set need to be merged. Public Assistance Funded Projects Summary: – The nine additional tabs of spreadsheet data should be used in a separate Spotfire file. 28