1. Sponsored by the National Science Foundation Nowcasting: UMass/CASA Weather Radar Demonstration Michael Zink CC-NIE Workshop January 7, 2013

2. Sponsored by the National Science Foundation 2 January 7, 2013 Problem CASA (an NSF ERC) is studying experimental networks of small controllable weather radars –Better data is the foundation of better hazardous weather detection and earlier warnings –Complex modeling to detect inclement weather requires many resources: sensors, bandwidth, storage, and computation Costly to dedicate resources for rare events –Cost of operation for weather for 75 days shows $50 of cloud usage vs. $4000 of dedicated hardware –How do we generate accurate, short-term “nowcasts” using these new distributed radar systems?

3. Sponsored by the National Science Foundation 3 January 7, 2013 Why more and smaller radars? gap - earth curvature prevents 72% of the troposphere below 1 km from being observed.

4. Sponsored by the National Science Foundation 4 January 7, 2013 Solution Today: only a few large NEXRAD radars (100s) Tomorrow: many (1000s) smaller, less expensive radars produce data close to the ground where weather happens Requires a flexible infrastructure for coordinated provisioning of shared sensing, networking, storage, and computing resources on-demand

5. Sponsored by the National Science Foundation 5 January 7, 2013 Example: Puerto Rico Testbed UPRM Student Testbed –Led by Jorge Trabal, Prof. Sandra Cruz-Pol, and Prof. Jose Colom –http://www.youtube.com/watch?v=7TR64BhwMlIhttp://www.youtube.com/watch?v=7TR64BhwMlI

6. Sponsored by the National Science Foundation 6 January 7, 2013 Demo Background Dynamic end-to-end Nowcasting on GENI –Use GENI/Orca Control Framework (RENCI/Duke) https://geni-orca.renci.org/trac/ http://geni-ben.renci.org:11080/orca/ –Reserve heterogeneous slice of resources Sensing Slice: UMass ViSE radars Networking Slice: NLR, BEN-RENCI Computation Slice: Amazon EC2 + UMass and ExoGENI VMs Storage Slice: Amazon S3

7. Sponsored by the National Science Foundation 7 January 7, 2013 What is a Nowcast? Up to 15 minute weather forecast Works only in the case of precipitation

8. Sponsored by the National Science Foundation 8 January 7, 2013 Demo Data Flow Dynamic end-to-end Nowcasting –Mapping Nowcast Workflows onto GENI Archival Storage Radar Nodes “raw” live data Upstream LDM feed archived netcdf data archived netcdf data Nowcast Processing aggregated multi-radar data aggregated multi-radar data Post to Web Nowcast images for display

9. Sponsored by the National Science Foundation 9 January 7, 2013 Multi-radar NetCDF Data Nowcast Processing 1.DiCloud Archival Service (S3) 2.LDM Data Feed (EC2) 1.DiCloud Archival Service (S3) 2.LDM Data Feed (EC2) “raw” live data Generate “raw” live data ViSE/CASA radar nodes Generate “raw” live data ViSE/CASA radar nodes http://stb.ece.uprm.edu/current.jsp Use proxy to track usage-based spending on Amazon and enforce quotas and limits http://geni.cs.umass.edu/vise/dicloud.php Use proxy to track usage-based spending on Amazon and enforce quotas and limits http://geni.cs.umass.edu/vise/dicloud.php 1.Ingest mulit-radar data feeds 2.Merge and grid multi-radar data 2.Generate 1min, 5min, and 10min Nowcasts 3.Send results over NLR to Umass 4.Repeat 1.Ingest mulit-radar data feeds 2.Merge and grid multi-radar data 2.Generate 1min, 5min, and 10min Nowcasts 3.Send results over NLR to Umass 4.Repeat ViSE views steerable radars as shared, virtualized resources http://geni.cs.umass.edu/vise ViSE views steerable radars as shared, virtualized resources http://geni.cs.umass.edu/vise Nowcast images for display Nowcast images for display

10. Sponsored by the National Science Foundation 10 January 7, 2013 Bigger Picture Analysis of Nowcast in the cloud –Compare networking and compute capabilities of different clouds

11. Sponsored by the National Science Foundation 11 January 7, 2013 Instance Type Memory (GB) Disk (GB) CPUCost/hr ($) Total Cost ($) Exec. Time (sec) Total Time (sec) Amzon EC27.585040.341.1374.3495.08 Rackspace8.032040.481.6396.53120.33 GENICloud8.0204--67.4578.60 ExoGENI8.0204--56.8372.07 Computation Time Analysis

12. Sponsored by the National Science Foundation 12 January 7, 2013 US Ignite – Ultra-high Bandwidth

13. Sponsored by the National Science Foundation 13 January 7, 2013 Future Experiments DFW UMass RENCI I2/NLR BBN LEARN UoH

14. Sponsored by the National Science Foundation 14 January 7, 2013 GENI/CASA Technologies and Credits UMass-Amherst –ViSE and DiCloud projects University of Puerto Rico, Mayaguez –Jorge Trabal, Prof. Cruz-Pol, and Prof. Colom –OTG Radars Colorado State University –Prof. V. Chandrasekar –Nowcasting Software RENCI/Duke –Orca Control Framework –BEN network Starlight

15. Sponsored by the National Science Foundation 15 January 7, 2013 Conclusion GENI is critical for next-generation applications –Enable nowcasting in experimental radar systems –GENI capabilities: “sliceability”/virtualization, federation, network programmability Provide domain scientists a new platform –Experiment with tightly integrated systems combining sensing, storage, networking, computing –Engage domain scientists in CASA and elsewhere Extend GENI network to Puerto Rico