1. Sponsored by the National Science Foundation University of Massachusetts Amherst November 2 nd, 2011 GENI DiCloud

2. Sponsored by the National Science Foundation 2 Project Summary DiCloud enables GENI experiments to incorporate cloud computing resources –Focuses on Amazon web services –EC2 (VMs), EBS (block storage), S3 (object storage) How are cloud resources different from other GENI resources? –They cost money! Requires active Amazon account and credit card –Require authentication with Amazon

3. Sponsored by the National Science Foundation 3 Project Summary DiCloud tracks per GENI-user Amazon costs –Amazon charges can be complicated Pay per VM time used Pay for data transfer in and out Pay for IOs to block devices –No current facilities for tracking per user costs Enables controlled access to cloud resources –Stand-alone: setup your own DiCloud instance to track your own costs –GENI CF-mode: place “underneath” GENI CF, where AM pays the bills, but gives GENI users access

4. Sponsored by the National Science Foundation 4 Elastic Compute Cloud (EC2) Rent VMs from Amazon “on-demand” –http://wwww.amazon.com/ec2http://wwww.amazon.com/ec2

5. Sponsored by the National Science Foundation 5 Elastic Compute Cloud (EC2) Rent VMs from Amazon “on-demand” –Pay per hour based on VM performance –Pay for data transfer in/out

6. Sponsored by the National Science Foundation 6 Elastic Block Store (EBS) Rent block storage from Amazon –Pay per GB-month (currently $0.10) –Also pay per 1 million IO requests (currently $0.10)

7. Sponsored by the National Science Foundation 7 Simple Storage Service (S3) Rent object storage from Amazon –Pay per GB-month –Pay for data transfer in/out

8. Sponsored by the National Science Foundation 8 Web Portal

9. Sponsored by the National Science Foundation 9 Web Portal

10. Sponsored by the National Science Foundation 10 Web Portal

11. Sponsored by the National Science Foundation 11 Tutorial Step 5: Web Portal

12. Sponsored by the National Science Foundation 12 Tutorial Step 5: Web Portal Now logout and log back in as user “geni” –Then click on “Request EC2 resources” –Use ami-013d6c44 as the AMI id

13. Sponsored by the National Science Foundation 13 Tutorial Step 5: Web Portal Now go back to your user page –Click on status next to the VM you just created

14. Sponsored by the National Science Foundation 14 Tutorial Step 5: Web Portal Now lets create an S3 bucket –Click on “Request new S3 buckets”

15. Sponsored by the National Science Foundation 15 Tutorial Step 5: Web Portal Once back at the user page, put something in your bucket –Select the s3_test file in ~/Tutorials/DiCloud/s3_test

16. Sponsored by the National Science Foundation 16 Tutorial Step 5: Web Portal Now get that same thing back from your bucket

17. Sponsored by the National Science Foundation 17 Tutorial Step 5: Web Portal Now lets create an EBS volume and attach it to our VM –Click request new EBS volumes –Make it 1GB, in us-west-1, in us-west-1

18. Sponsored by the National Science Foundation 18 Tutorial Step 5: Web Portal Attach the volume to our VM –Click on attach, enter instance id, and sdg for device

19. Sponsored by the National Science Foundation 19 Tutorial Step 5: Web Portal If we ssh into our VM we can see that the volume is now attached

20. Sponsored by the National Science Foundation 20 Demo Data Flow Dynamic end-to-end Nowcasting –Mapping Nowcast Workflows onto GENI Archival Storage Radar Nodes “raw” live data Upstream LDM feed archived netcdf data archived netcdf data Nowcast Processing aggregated multi-radar data aggregated multi-radar data Post to Web Nowcast images for display

21. Sponsored by the National Science Foundation 21 Multi-radar NetCDF Data Nowcast Processing 1.DiCloud Archival Service (S3) 2.LDM Data Feed (EC2) 1.DiCloud Archival Service (S3) 2.LDM Data Feed (EC2) “raw” live data Generate “raw” live data ViSE/CASA radar nodes Generate “raw” live data ViSE/CASA radar nodes http://stb.ece.uprm.edu/current.jsp Use proxy to track usage-based spending on Amazon and enforce quotas and limits http://geni.cs.umass.edu/vise/dicloud.php Use proxy to track usage-based spending on Amazon and enforce quotas and limits http://geni.cs.umass.edu/vise/dicloud.php 1.Ingest mulit-radar data feeds 2.Merge and grid multi-radar data 2.Generate 1min, 5min, and 10min Nowcasts 3.Send results over NLR to Umass 4.Repeat 1.Ingest mulit-radar data feeds 2.Merge and grid multi-radar data 2.Generate 1min, 5min, and 10min Nowcasts 3.Send results over NLR to Umass 4.Repeat ViSE views steerable radars as shared, virtualized resources http://geni.cs.umass.edu/vise ViSE views steerable radars as shared, virtualized resources http://geni.cs.umass.edu/vise Nowcast images for display Nowcast images for display

22. Sponsored by the National Science Foundation 22 Demo Overview Dynamic end-to-end Nowcasting on GENI –Slice of sensing, networking, computing, and storage Archival Storage Radar Nodes Upstream LDM feed Nowcast Processing Post to Web Generate “raw” data ViSE/CASA radars Archive radar data Amazon S3 “raw” live data Archived data available to downstream nodes archived netcdf data archived netcdf data aggregated multi-radar data aggregated multi-radar data Nowcast images for display Generate Nowcasts