Presentation is loading. Please wait.

Presentation is loading. Please wait.

Creation, Analysis, Sharing, and Visualization of Complex Spatiotemporal Data FOSS4G North America Dan Getman | Match10, 2015 using Free and Open Source.

Similar presentations


Presentation on theme: "Creation, Analysis, Sharing, and Visualization of Complex Spatiotemporal Data FOSS4G North America Dan Getman | Match10, 2015 using Free and Open Source."— Presentation transcript:

1 Creation, Analysis, Sharing, and Visualization of Complex Spatiotemporal Data FOSS4G North America Dan Getman | Match10, 2015 using Free and Open Source Software at the National Renewable Energy Laboratory

2 DOE National Labs: AcademiaIndustry National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554 http://en.wikipedia.org/wiki/Harvard_Law_School http://en.wikipedia.org/wiki/New_York_City

3 DOE National Labs: Academia Unicorn http://sovaeart.deviantart.com/art/Aura-183473554 http://en.wikipedia.org/wiki/Harvard_Law_School http://en.wikipedia.org/wiki/New_York_City Industry National Renewable Energy Laboratory

4 NREL : Department of Energy National Laboratory o ~ 2400 People Focused on o Energy Efficiency o Renewable Energy Geospatial Data Science Team o ~ 14 People Focused on o Spatial Analysis – Traditional GIS focused on EE and RE o “Big Data” and Spatial Statistical Analysis o Spatiotemporal Data Visualization o Spatial Application Development What’s The Point? o Making sure the decisions we make with RE/EE are good ones National Renewable Energy Laboratory

5 NREL : Department of Energy National Laboratory o ~ 2400 People Focused on o Energy Efficiency o Renewable Energy Geospatial Data Science Team o ~ 14 People Focused on o Spatial Analysis – Traditional GIS focused on EE and RE o “Big Data” and Spatial Statistical Analysis o Spatiotemporal Data Visualization o Spatial Application Development What’s The Point? o Making sure the decisions we make with RE/EE are good ones National Renewable Energy Laboratory

6 What are we talking about here? o Looking at a progression from proprietary to open source software for analysis and application development o What were the drivers for this? o What has it meant for our team? o What are the Good and Bad outcomes of the decision? o What are we able to do with all of this anyway? National Renewable Energy Laboratory

7 Starting Point: Analysis (5 years back) Traditional Spatial Analysis Big Data Analysis (Solar/Wind Data) C/C++ TB of Data National Renewable Energy Laboratory

8 Starting Point: App Development Featur e Server National Renewable Energy Laboratory

9 Starting Point: App Development Featur e Server National Renewable Energy Laboratory

10 Current : Spatial Analysis Pathway National Renewable Energy Laboratory

11 Current : Spatial Analysis Pathway National Renewable Energy Laboratory

12 Current : Spatial Analysis Pathway National Renewable Energy Laboratory

13 Current : Spatial Analysis Pathway Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV National Renewable Energy Laboratory

14 Current : Spatial Analysis Pathway Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV National Renewable Energy Laboratory

15 Current : Spatial Analysis Pathway National Renewable Energy Laboratory

16 Analysis o R and libs o Python and libs o Postgres/PostGIS o Quantum Open Source Hit List: Application Development o Ruby on Rails o Python and libs o Node JS o Geoserver o Postgres/PostGIS o Tons of JS libs o Backbone and Ember o D3 o jQuery o Leaflet o OpenLayers General o Github o Linux o HDF National Renewable Energy Laboratory

17 Benefits: o Flexibility and Scalability o Laptop -to- Server -to- Servers -to- HPC o Ability to process multiple billions of data values o Costs and Ease of Use o Experimentation without having to purchase software o So we can use the right tools for the problem at hand o Promotes Modular Design o Design to support substitution of tools as needed o Promotes Integrated Architecture o Analysis uses the same data and design as applications Open Source Reflections National Renewable Energy Laboratory

18 Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP Open Source Reflections National Renewable Energy Laboratory

19 Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = Open Source Reflections National Renewable Energy Laboratory

20 Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554

21 Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = o Geospatial + Visualization + Advanced JS = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554

22 Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = o Geospatial + Visualization + Advanced JS = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554

23 Examples of Our Work National Renewable Energy Laboratory o Spatiotemporal Data Visualization o Renewable Electricity Futures Study Visualization o 40k possible maps in one visualization o Spatial Application Development o Open Carto – Web Based Mapping Platform o HyDRA – Hydrogen Vehicle Technology o Geothermal Prospector o Wind Prospector for Philippines o Spatial Analysis o Creation of large spatiotemporal resource datasets o Comparisons of large spatiotemporal resource datasets

24 Your Questions and Feedback are Welcome National Renewable Energy Laboratory Dan Getman – dan.getman@nrel.gov

25

26

27

28

29

30

31

32

33 33 Current Workflow

34 Your Questions and Feedback are Welcome National Renewable Energy Laboratory Dan Getman – dan.getman@nrel.gov


Download ppt "Creation, Analysis, Sharing, and Visualization of Complex Spatiotemporal Data FOSS4G North America Dan Getman | Match10, 2015 using Free and Open Source."

Similar presentations


Ads by Google