1. WRF in the Cloud: An introduction to Big Compute on Windows Azure Wenming Ye Research Program Manager Microsoft Research Twitter: @wenmingye

3. What is Cloud Computing?

4. Software-as-a-Service SaaS Platform-as-a-Service Stateless, easy to scale, manage PaaS Infrastructure-as-a-Service with Persistent remote disks IaaS

6. Cloud servicesWeb sitesVirtual machines

7. DELL Compute/Storage Container Transformer & UPS Air Flow Vents

8. Inside a DELL Container Compute Rack 2 “Side by Side” Servers Disk Sled Rack On the truck At DELL’s Austin Plant

10. DLA Architecture (Old)Quantum10 Architecture (New) TOR Spine … … DCR BL Spine DC Routers BL 30,000 Gbps 120 Gbs

12. Internet of things Audio / Video Log Files Text/Image Social Sentiment Data Market Feeds eGov Feeds Weather Wikis / Blogs Click Stream Sensors / RFID / Devices Spatial & GPS Coordinates WEB 2.0 Mobile Advertisin g CollaborationeCommerce Digital Marketing Search Marketing Web Logs Recommendation s ERP / CRM Sales Pipeline Payables Payroll Inventory Contacts Deal Tracking Terabytes (10E12) Gigabytes (10E9) Exabytes (10E18) Petabytes (10E15) Velocity - Variety - variability Volume 1980 190,000$ 2010 0.07$ 1990 9,000$ 2000 15$ Storage/GB ERP / CRM WEB 2.0 Internet of things

13. Scenarios

14. Design Optimization Financial and insurance risk calculation Engineering modeling and simulation Computational life sciences Earth sciences Data analytics

15. Invisible devices Trillions of networked nodes Low bandwidth last- mile connection Mostly addressed by local schemes Machine-centricSensing-focus Global addressingUser-centric Communication- focus Laptops / tablets / smartphones Billions of networked devices High-bandwidth access

17. How do I optimize my services based on patterns of weather, traffic. How do I build a recommendation engine? What’s the social sentiment of my product? How do I better predict future outcomes?

18. Multiple/Any languages (Fortran too)  http://WindowsAzure.com

22. http://www.windowsazure.com/ en-us/develop/nodejs/how-to- guides/command-line-tools/

24. Distributed Storage (HDFS) Query (Hive) Distributed Processing (MapReduce) HDINSIGHT / HADOOP Eco-System Legend Red = Core Hadoop Blue = Data processing Purple = Microsoft integration points and value adds Orange = Data Movement Green = Packages

25. Front end Stream Layer Partition Layer Name Node de Data Node Front end HDFS API DFS (1 Data Node per Worker Role) and Compute Cluster Azure Storage (ASV) … Azure Blob Storage

26. A few Demos

27. Demo: Weather Forecast as a Service

28. http://weatherservice.cloudapp.net

30. Azure Table storage Front end Stream Layer Partition Layer Azure Storage Front end

36. HeatMap: 5 Compute Nodes running at 100%

41. Geogrid: Defines the model horizontal domain Horizontally interpolates static data to the model domain Output conforms to the WRF I/O API Ungrib: Decodes Grib Edition 1 and 2 data (Grib is the file format of the NOAA forecast data) Uses tables to decide which variables to extract from the data Supports isobaric and generalized vertical coordinates Output is in a non-WRF-I/O-API form, referred to as an intermediate format Metgrid: Ingest static data and raw meteorological fields Horizontally interpolate meteorological fields to the model domain Output conforms to WRF I/O API

42. Gdal_translate: Converts file from png to geospatial Tiff (geoTiff) format Assigns geo-referencing (“ground control points”) to specified pixel values Labels the output (geoTiff) file’s header with map projection information Gdalwarp: Geographically reprojects the output file from gdal_translate to the map projection of Bing Maps Gdal2tiles.py: A python script that generates tiles from the output file from gdalwarp Zoom levels are specified on the command line

43. Demo: Interactive computing and data analysis with IPython NotebookIPython Notebook

51. PaaS IaaS

53. Two MODIS satellites Terra, launched 12/1999 Aqua, launched 05/2002 Near polar orbits Global coverage two days Sensitive in 36 spectral bands

56. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.