CloudCast: Cloud Computing for Short-term Mobile Weather Forecasts Dilip Kumar Krishnappa, David Irwin, Eric Lyons and Michael Zink IPCCC 2012

2 ECE Department Outline  Introduction  Cloud Services  CloudCast Architecture  Cloud Measurements  Nowcasting  Conclusion

3 ECE Department Motivation In 75 days, 90 hours of convective precipitation Nowcast only useful during these events Cost for dedicated hardware $4000 (does not include power, AC, maintenance) Idle for 95% of the time At 45 cents/ hour, 90 hours of active use would cost $40 per product, plus $2 per user to stream the resultant data out of the cloud

4 ECE Department Nowcast Example

5 ECE Department Introduction  Dedicating high-end servers for intermittent requirement wastes resources.  Cloud services IaaS model suits such applications.  Short term weather forecasting is one such real-time application.  Cost of operation for weather predicted last year for 75 days shows 50$ of cloud usage vs. 4000$ of dedicated hardware.  Are cloud services well suited for such applications?

6 ECE Department Introduction (Contd..)  We consider real-time application of short-term weather forecasting called “Nowcasting”.  Nowcasting produces short- term forecasts for 10s of minutes in the future.  Nowcasting has strict time constraints.  Are cloud services capable of keeping up with such strict time constraints?

7 ECE Department Cloud Services  Commercial Cloud Services: Amazon EC2, Rackspace  Research Cloud Testbeds: GENICloud, ExoGENI  Commercial cloud services provide on-demand resources based on per hour pricing.  Pricing depends on the instance requested and the amount of I/O performed per hour.

8 ECE Department Cloud Services (Contd..)  GENICloud is an open source cloud platform using Eucalyptus as a base.  GENICloud provides slice-based architecture to acquire cloud instances.  ExoGENI is open source cloud platform provisioned by ORCA control framework.  ExoGENI provides flexibility to users to upload the image of their own and extend the size of the image.

9 ECE Department NEXRAD Radars

10 ECE Department Measurements  PlanetLab measurement to mimic the NEXRAD radars.  159 NEXRAD radar sites around the country. 103 PlanetLab nodes found near to those locations.  According to our own radar testbed setup (CASA), 5 Mbps is the minimum throughput of data transfer from radars.

11 ECE Department Measurements (Contd..)  Three types of data transfer experiment. - Serial Data Transfer - Parallel Data Transfer - Distributed Data Transfer  Parallel transfer helps to understand the bottleneck in the system.  Data is transferred to nearest instance based on the subset of nodes in distributed data ingest.

12 ECE Department Distributed Data transfer Radar2 Radar1Radar3 Radar4 Cloud Instance Cloud Instance1Cloud Instance2 Radar1 Radar2 Radar3 Radar4 Parallel Data Transfer Distributed Data Transfer

13 ECE Department Results – Serial, Commercial Cloud

14 ECE Department Results – Serial, Research Cloud

15 ECE Department Summary of Results Measurement Type EC2 East (Mbps) EC2 West (Mbps) Rackspace (Mbps) GENICloud (Mbps) ExoGENI (Mbps) Serial Parallel Distributed

16 ECE Department Computation Time Analysis Instance Type Memory (GB) Disk (GB) CPUCost/ hr ($) Total Cost ($) Exec. Time (sec) Total Time (sec) Amzon EC Rackspace GENICloud ExoGENI

17 ECE Department Nowcasting  Main objective of our work is the feasibility of Cloud Services for short-term weather forecasting.  We have found from our studies that, cloud services are feasible for Nowcasting.  We performed live measurement and analysis to determine the overall time taken to generate 15-minute Nowcasts in each of the cloud services considered.  We find that it takes just about 2 minutes max for the generation of 15-minute Nowcasts and sending the results to the end user.

18 ECE Department Data Aggregation

19 ECE Department DFW Radar Deployment NETWORK ROLLOUT: First phase in blue, Second phase in green UNT UTA Town of Addison City of Fort Worth

20 ECE Department Installation of UTA radar

21 ECE Department Future Scenario

22 ECE Department Conclusion  In our work, we look into the feasibility of cloud services for real time scientific applications.  Scientific application in our case is short-term weather forecasting called Nowcasting.  We perform a series of measurements to determine our objective.  From our measurements, we find that cloud services perform best when there is less number of radar nodes sending data to a single instance.  We conclude that, having a subset of radar nodes sending data to a single instance works best for our real-time application.

23 ECE Department US Ignite – Ultra-high Bandwidth Operations Control Center Short-term Forecast in the Cloud Short-term Forecast in the Cloud Long-term, Large-scale Forecast Long-term, Large-scale Forecast End Users: Public, NWS EM,Media Industry End Users: Public, NWS EM,Media Industry Up to 150 Mbps Up to n * 150 Mbps