Wen-ming Ye Sr. Technical Evangelist Microsoft Corporation 12/28/2018 10:26 AM AZR303 Developing High Performing Parallel Application Services on Windows Azure Wen-ming Ye Sr. Technical Evangelist Microsoft Corporation © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Session Goals and Agenda TechEd 2012 12/28/2018 10:26 AM Session Goals and Agenda Understand Compute Intensive Workload in Windows Azure Learn Build, Deploy, and Run Services using the HPC Scheduler Get ideas for teaching your old CI apps some new Cloud tricks! © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What’s the Problem? Some apps run too slowly on a single computer Splitting them up into chunks, then running those chunks in parallel on multiple computers can speed them up Doing this effectively requires creating a compute cluster, with: Tools to create and manage the cluster A batch scheduler for running apps on the cluster

How Does Azure Extend High Performance Computing to the Masses? Solve large computational problems HPC is best described as massively parallel Becoming a critical part of businesses

Example Workloads That Need a Cluster Digital media rendering and encoding Financial and insurance risk calculation Engineering modeling and simulation Computational life sciences Earth sciences Data analytics

Earthquake and Weather Simulations 12/28/2018 10:26 AM video Earthquake and Weather Simulations High resolution Computing © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

A 20,000 x 20,000 pixel Tobacco Mosaic Virus render 12/28/2018 10:26 AM demo A 20,000 x 20,000 pixel Tobacco Mosaic Virus render © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Embarrassingly Parallel Apps An illustration Cluster of Compute Nodes Compute Node Application Logic 1. Submit work 2. (Optionally) Interact with application 3. Get results

Tightly Coupled App (MPI) An illustration Cluster of Compute Nodes Compute Node Application Logic 1. Submit work 2. Get results

IO Intensive Applications An illustration Cluster of Compute Nodes Compute Node Application 1. Submit work 2. Get results

HPC Server Makes This Easy – offloading long running jobs Windows HPC Application Logic Client Logic Windows HPC Server Logic Job Scheduling Cluster Management Compute Nodes Windows 7 Windows HPC Server Windows Azure Head Node (Physical or VM)

Windows HPC Server 2008 R2 SP3

Cloud - Applications in Azure June 2012 Release Application in hosted entirely in Azure No on-premise Head Node Application is accessed from Rich client on premise Portal Web Application Multiple business models Packaged application is sold to a customer Application is available as a service in the cloud Scheduler and Runtimes Supported in Azure Workloads: Parametric Sweep, SOA, MPI No Head Node (Scheduler Service, no Admin Services)

Building a Scheduler Enabled Deployment Azure Subscription Your application Construct a base CS Package Customize and Deploy

Windows Azure Scheduler In Action A Windows Azure Deployment Front End Compute Worker Front End Application Front End (Web Page) Node Manager Application Windows Azure Scheduler Compute Worker Compute Worker Compute Worker Consistent HPC Job Scheduler API Compute Worker

Visual Studio Sample Solution – Easy first development experience for WAHS parallel applications in Azure Deployment Application Auto-create and initialize hosted service, storage, HPC database in same location Progress bar and status PSH Scripts available too Example: Cloud Numerics…

Deploy a cluster using Visual Studio 12/28/2018 10:26 AM demo Deploy a cluster using Visual Studio © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scenario: Full desktop client offloading long running tasks to the Azure HPC Scheduler

Simplicity of Microsoft HPC Compute Intensive Paradigms Run on: Client Cluster Cloud Transparently

Openfoam compute offload demo Openfoam compute offload Rich client application connecting to windows Azure scheduler for offloading a fluid dynamics model © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scenario: More code to run: Moving code form Linux/Unix platforms

OSS Eco-system To create Scalable TC Azure services Developer Tools Compilers: PGI Intel GCC Build: CMake Visual Studio Unix Env: Cygwin SUA (win8) MinGW Perf Tools Vampir TAU Libraries Solvers: PETSc Trilinos FFTW IO: HDF5MPI NetCDF Popular HPC Apps Chemistry Bio Science CFD Structure Climate Physics Data Mining Computer Science Analysis Tools Rich client Parallel Visualization: VisIt ParaView Python: HTML5 front Ipython NumPY SciPY

Scenario: Building a Service for sharing simulation engine and data using the Azure HPC Scheduler

demo Weather as a Service 12/28/2018 10:26 AM © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Roadmap Spring 2012 Hybrid Clusters Across Enterprise and Cloud All items considered general availability, unless noted. *Denotes Spring in the US, 1st half of 2012 Hybrid Clusters Across Enterprise and Cloud Support for Client->Job Scheduler .NET API tunneled over HTTP (i.e., same Job Scheduling API for on-premise and Windows Azure clusters) Building HPC and Parallel Applications in the Cloud Initializing HPC SQL Azure DB from HN in Azure Windows HPC Pack 2008 R2 Service Pack 4

Who would use it? Do it yourself cluster Packaged software Power user In house developer Packaged software Leverage additional compute resource in Windows Azure Software as a Service Vendor pays for the compute resources Vendor bills end user directly Who would use it?

9(-5) Tips for Best Practices Hide cloud from your users by providing a transparent experience Minimize data transfer between on-premises and the cloud CPU intensive jobs are best suited for pay as you go model Be creative and focus on what you can do on Azure today

The Matrix can not be told, try it yourself! Subscribe to my blog http://blogs.msdn.com/hpctrekker Read Application Models and Data Considerations Download SP4 from download.microsoft.com Get a free Trial of Azure for 30 days Download and try out the Azure Burst Samples Implement your own solution and let us know!

Thank You!

Track Resources @WindowsAzure @ms_teched DOWNLOAD Windows Azure Meetwindowsazure.com @WindowsAzure @ms_teched DOWNLOAD Windows Azure Windowsazure.com/ teched Hands-On Labs

Resources Learning TechNet http://europe.msteched.com Connect. Share. Discuss. http://europe.msteched.com Microsoft Certification & Training Resources www.microsoft.com/learning TechNet Resources for IT Professionals http://microsoft.com/technet Resources for Developers http://microsoft.com/msdn

Submit your evals online 12/28/2018 10:26 AM Evaluations Submit your evals online http://europe.msteched.com/sessions © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12/28/2018 10:26 AM © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Track Resources My Blog: http://blogs.msdn.com/hpctrekker Required Slide Track PMs will supply the content for this slide, which will be inserted during the final scrub. Tech Ed North America 2010 12/28/2018 10:26 AM Track Resources My Blog: http://blogs.msdn.com/hpctrekker The Azure Burst Samples and White paper Windows HPC ResourceKit Windows HPC Server TechCenter © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12/28/2018 10:26 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.