Grid Services For Virtual Organizations Andrea Valboni (andrea.valboni@microsoft.com) National Technology Officer Public Sector – Microsoft Italy Catania, 27-28 Ottobre 2006 EGEE2006 – Industry Days

We’re Going To Talk About… Driving Factors Current market trends Microsoft Compute Cluster Server Strategy © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Entry Price-Points For HPC Has Dropped 1991 1998 2005 Sample System Cray Y-MP C916 Sun HPC10000 Shuttle@ NewEgg.com Performance ~10 GFlops ~10 GFlops ~10 GFlops 15 years ago 10 GFlops WAS supercomputing .. And it cost 40 mill Now you can get that for under $5k Result – you now have a lot more folks thinking .. Hmm ..i should be using supercomputing to solve this problem I have. And these are folks in the business world Price $40,000,000 $1,000,000 (40x drop) $4,000 (250x drop) Customers Government Labs Large Enterprises Every Engineer and Scientist © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Distributed Memory Becoming The Norm Clusters over 70% So Supercomputing is growing… but it’s really growing in a single form factor - DM Look at the chart. Rapid growth in DM from 2000 to 2005 driven by growing use of compute clusters © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

COTS Components Gaining Ground 9/11/2018 8:37 PM COTS Components Gaining Ground GigE is gaining (50% of systems) You can also see this in the growing use of COTS components X86 is leading (Pentium 41%, EM64T 16%, Opteron 11% © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. 5 © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Commercial Adoption Is On The Rise Industry usage rising The end-result here is ….. Lots more people successfully making the case that HPC results in ROI Caveat – this is top500. But if it’s true here, it’s even more true for the segment below this © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Market Growth Driven By The Low End Worldwide HPC Systems Forecast Source: IDC 300,000 250,000 200,000 150,000 100,000 50,000 2003 2004 2005 2006 2007 2008 2009 >$1M >$250K >$50K <$50K Installations HPC system growth: 70% in 2004; 25% in 2005 (IDC) Growth being driven by <$250K segment (clusters) x86 server clusters growing faster than market (15%-20% for HPC clusters, 11.4% for x86 overall) So what does the overall market looking like? Explain the graph – total HPC shipments Look at overall growth Now look at growth by segment; can only see two bands; clusters are driving this 10% of WW x86 Growing faster than the overall market © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Compute Cluster Server Strategy Build a complete platform Operating System, Message Passing Interface (MPI), Job Scheduler, Cluster Management Develop the eco-system Hardware platform and interconnect support Application support from key players Raise awareness Evangelize parallel computing Partner with the community HPC Institutes for thought leadership Development of cross-platform standards Leverage (and give back) to the open-source community To repeat what I said earlier – this is a complete platform for cluster based HPC Apps drive the sale, so eco-system support is critical Some verticals are ahead of others in exploiting HPC. Academia and Govt are traditional (& large) strongholds Close-knit community that respects thought leadership and collaboration © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Windows Compute Cluster Server 2003 Head Node Job Mgmt Resource Mgmt Cluster Mgmt Scheduling Active Directory Desktop App Jobs Policy, reports User Input Admin Console Job Mgr UI Management Tasks Admin Cmd line Cmd line Data The compute cluster can run the high performance jobs in a secure, reliable and transparent fashion. Thru the integration with Active Directory, the jobs are running under the submission user’s security context, able to access the data residing on the corporate network in ways that the same application does on the users workstation. The Compute Cluster works as follows: User stages the data into a shared file system or the database The user submit jobs to the Job Scheduler via a wide spectrum of mechanisms: (1) command line interface, (2) Job Manager UI, or (3) Integrated desktop applications thru the COM API The scheduler allocates idle resources for the job, once allocated multiple tasks are dispatched to the allocated nodes so that the execution time are accelerated These tasks can use the built-in MPI stack or run independently to complete a parametric sweep These tasks are running inside the users’ security context validated against the Active Directory Domain\UserA High speed, low latency interconnect DB/FS Node Manager Job Execution User App MPI © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

HPC Grids And Web Services HPC Grid ~ Compute Grid + Data Grid Compute grid Forest of clusters and workstations within an organization Coordinated scheduling of resources Data grid Distributed storage facilities within an organization Coordinated management of data Web Services The means to achieve interoperable Internet-scale computing, including federation of organizations Loosely-coupled, service-oriented architecture

Grid Services for Virtual Organizations Application domain-specific services Security Data HPC Workflow Virtual Organizations Web Services Technologies

HPC Scenarios Tackling system complexity 9/11/2018 8:37 PM HPC Scenarios Tackling system complexity Scenario Focus Departmental Cluster Conventional scenario IT owns large clusters due to cost and complexity and allocates resources on per job basis Users submit batch jobs via scripts In-house and ISV apps, many based on MPI Scheduling multiple users’ applications onto scarce compute cycles Cluster systems administration Personal/Workgroup Cluster Emerging scenario Clusters are pre-packaged OEM appliances, purchased and managed by end-users Desktop HPC applications transparently and interactively make use of cluster resources Desktop development tools integration Interactive applications Workstation clusters, accelerator appliances Distributed, policy-based management and security HPC Application Integration Future scenario Multiple simulations and data sources integrated into a seamless application workflow Network topology and latency awareness for optimal distribution of computation Structured data storage with rich meta-data Applications and data potentially span organizational boundaries Data-centric, “whole-system” workflows Rapid prototyping of HPC applications Grids: Distributed application, systems, and data management Interoperability IT Mgr Manual, batch execution Interactive Computation and Visualization SQL © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Summary Parallel computing is here to stay Windows Compute Cluster Server 2003 is Microsoft’s first step towards bringing HPC into the computing mainstream CCS will serve existing scenarios… Datacenter-based Compute Clustering …and drive momentum around new ones Deskside Supercomputing © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Resources Microsoft HPC Web site CCS Community Site http://www.microsoft.com/hpc/ CCS Community Site http://www.windowshpc.net Windows Server x64 information http://www.microsoft.com/x64/ © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

© 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.