1. Creation, Analysis, Sharing, and Visualization of Complex Spatiotemporal Data FOSS4G North America Dan Getman | Match10, 2015 using Free and Open Source Software at the National Renewable Energy Laboratory

2. DOE National Labs: AcademiaIndustry National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554 http://en.wikipedia.org/wiki/Harvard_Law_School http://en.wikipedia.org/wiki/New_York_City

3. DOE National Labs: Academia Unicorn http://sovaeart.deviantart.com/art/Aura-183473554 http://en.wikipedia.org/wiki/Harvard_Law_School http://en.wikipedia.org/wiki/New_York_City Industry National Renewable Energy Laboratory

4. NREL : Department of Energy National Laboratory o ~ 2400 People Focused on o Energy Efficiency o Renewable Energy Geospatial Data Science Team o ~ 14 People Focused on o Spatial Analysis – Traditional GIS focused on EE and RE o “Big Data” and Spatial Statistical Analysis o Spatiotemporal Data Visualization o Spatial Application Development What’s The Point? o Making sure the decisions we make with RE/EE are good ones National Renewable Energy Laboratory

5. NREL : Department of Energy National Laboratory o ~ 2400 People Focused on o Energy Efficiency o Renewable Energy Geospatial Data Science Team o ~ 14 People Focused on o Spatial Analysis – Traditional GIS focused on EE and RE o “Big Data” and Spatial Statistical Analysis o Spatiotemporal Data Visualization o Spatial Application Development What’s The Point? o Making sure the decisions we make with RE/EE are good ones National Renewable Energy Laboratory

6. What are we talking about here? o Looking at a progression from proprietary to open source software for analysis and application development o What were the drivers for this? o What has it meant for our team? o What are the Good and Bad outcomes of the decision? o What are we able to do with all of this anyway? National Renewable Energy Laboratory

7. Starting Point: Analysis (5 years back) Traditional Spatial Analysis Big Data Analysis (Solar/Wind Data) C/C++ TB of Data National Renewable Energy Laboratory

8. Starting Point: App Development Featur e Server National Renewable Energy Laboratory

9. Starting Point: App Development Featur e Server National Renewable Energy Laboratory

10. Current : Spatial Analysis Pathway National Renewable Energy Laboratory

11. Current : Spatial Analysis Pathway National Renewable Energy Laboratory

12. Current : Spatial Analysis Pathway National Renewable Energy Laboratory

13. Current : Spatial Analysis Pathway Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV National Renewable Energy Laboratory

14. Current : Spatial Analysis Pathway Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV National Renewable Energy Laboratory

15. Current : Spatial Analysis Pathway National Renewable Energy Laboratory

16. Analysis o R and libs o Python and libs o Postgres/PostGIS o Quantum Open Source Hit List: Application Development o Ruby on Rails o Python and libs o Node JS o Geoserver o Postgres/PostGIS o Tons of JS libs o Backbone and Ember o D3 o jQuery o Leaflet o OpenLayers General o Github o Linux o HDF National Renewable Energy Laboratory

17. Benefits: o Flexibility and Scalability o Laptop -to- Server -to- Servers -to- HPC o Ability to process multiple billions of data values o Costs and Ease of Use o Experimentation without having to purchase software o So we can use the right tools for the problem at hand o Promotes Modular Design o Design to support substitution of tools as needed o Promotes Integrated Architecture o Analysis uses the same data and design as applications Open Source Reflections National Renewable Energy Laboratory

18. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP Open Source Reflections National Renewable Energy Laboratory

19. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = Open Source Reflections National Renewable Energy Laboratory

20. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554

21. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = o Geospatial + Visualization + Advanced JS = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554

22. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = o Geospatial + Visualization + Advanced JS = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554

23. Examples of Our Work National Renewable Energy Laboratory o Spatiotemporal Data Visualization o Renewable Electricity Futures Study Visualization o 40k possible maps in one visualization o Spatial Application Development o Open Carto – Web Based Mapping Platform o HyDRA – Hydrogen Vehicle Technology o Geothermal Prospector o Wind Prospector for Philippines o Spatial Analysis o Creation of large spatiotemporal resource datasets o Comparisons of large spatiotemporal resource datasets

24. Your Questions and Feedback are Welcome National Renewable Energy Laboratory Dan Getman – dan.getman@nrel.gov

33. 33 Current Workflow

34. Your Questions and Feedback are Welcome National Renewable Energy Laboratory Dan Getman – dan.getman@nrel.gov