Download presentation
Presentation is loading. Please wait.
1
High-Performance Computing With Windows
4/17/2017 3:00 AM High-Performance Computing With Windows Ryan Waite General Program Manager Windows Server HPC Group Microsoft Corporation
2
Outline Part 1: Overview Part 2: Drill-down
4/17/2017 3:00 AM Outline Part 1: Overview Why Microsoft has gotten into HPC What our V1 product offers Some future directions Part 2: Drill-down A few representative V1 features (for those who are interested)
3
4/17/2017 3:00 AM Part 1 Overview
4
Evolving Tools Of The Scientific Process
4/17/2017 3:00 AM Evolving Tools Of The Scientific Process 1. Observation 2. Hypothesis Instruments Experiments done with a telescope by Galilei 400 years ago inaugurated the scientific method Microscope, laser, x-ray, collider, accelerator allowed peering further and deeper into matter HPC Automation and acceleration of the scientific and engineering process itself Digital instruments, data mining, simulation, experiment steering 4. Validation 3. Prediction
5
The Next Challenge Taking HPC Mainstream
Volume economics of industry standard hardware and commercial software applications are rapidly bringing HPC capabilities to a broader number of users But HPC is still only accessible to the few computational scientists who can master a domain science, program parallel, distributed algorithms, and use/manage a supercomputer Microsoft HPC Strategy – taking HPC to the mainstream Enabling broad HPC adoption and making HPC into a high volume market in which everyone can have their own personal supercomputer Enabling domain scientists who are not computer scientists to partake in the HPC revolution 5
6
Evidence Of Standardization And Commoditization
4/17/2017 3:00 AM Evidence Of Standardization And Commoditization Clusters over 70% Industry usage rising GigE is gaining (50% of systems) x86 is leading (Pentium 41%, EM64T 16%, Opteron 11%)
7
HPC Market Trends <$250K – 97% of systems, 55% of revenue
4/17/2017 3:00 AM HPC Market Trends <$250K – 97% of systems, 55% of revenue Capability, Enterprise $1M+ 2005 Systems 2005 Growth Divisional $250K-$1M Departmental $50-250K 981 -3% Workgroup <$50K 4,988 30% Report of the High-End Computing Revitalization Task Force, 2004 (Office of Science and Technology Policy, Executive Office of the President) “Make high-end computing easier and more productive to use. Emphasis should be placed on time to solution, the major metric of value to high-end computing users… A common software environment for scientific computation encompassing desktop to high-end systems will enhance productivity gains by promoting ease of use and manageability of systems.” 21,733 36% 163,441 33% Source: IDC, 2005 7
8
Even The Low End Is Powerful
4/17/2017 3:00 AM Even The Low End Is Powerful 1991 1998 2005 System Cray Y-MP C916 Sun HPC10000 Small Form Factor PCs Architecture 16 x Vector 4GB, Bus 24 x 333MHz Ultra-SPARCII, 24GB, SBus 4 x 2.2GHz Athlon64 4GB, GigE OS UNICOS Solaris 2.5.1 Windows Server 2003 SP1 GFlops ~10 Top500 # 1 500 N/A Price $40,000,000 $1,000,000 (40x drop) < $4,000 (250x drop) Customers Government Labs Large Enterprises Every Engineer and Scientist Applications Classified, Climate, Physics Research Manufacturing, Energy, Finance, Telecom Bioinformatics, Materials Sciences, Digital Media 8
9
Top Challenges Setup is painful Clusters are separate islands
4/17/2017 3:00 AM Top Challenges Setup is painful Takes a long time to get clusters up and running Clusters are separate islands Lack of integration into IT infrastructure Job management Lack of integration into end-user apps Application availability Limited eco-system of applications that can exploit parallel processing capabilities “Make high-end computing easier and more productive to use. Emphasis should be placed on time to solution, the major metric of value to high-end computing users… A common software environment for scientific computation encompassing desktop to high-end systems will enhance productivity gains by promoting ease of use and manageability of systems.” High-End Computing Revitalization Task Force, 2004 (Office of Science and Technology Policy, Executive Office of the President)
10
Windows Compute Cluster Server 2003
4/17/2017 3:00 AM Windows Compute Cluster Server 2003 Simplified cluster deployment, job submission and status monitoring Better integration with existing Windows infrastructure allowing customers to leverage existing technology and skill-sets Familiar development environment allows developers to write parallel applications from within the powerful Visual Studio IDE 10
11
Windows Compute Cluster Server 2003
4/17/2017 3:00 AM Windows Compute Cluster Server 2003 Desktop App Jobs Policy, reports User Admin Console Job Mgr UI Admin Cmd line Cmd line High speed, low latency interconnect DB/FS Node Manager 11
12
Leveraging Existing Windows Infrastructure
4/17/2017 3:00 AM Leveraging Existing Windows Infrastructure Active Directory Windows Security Integration with IT infrastructure Kerberos authentication Resource management Secure job execution Group policies Secure MPI Compute Cluster Built-in Tools Microsoft Enterprise Management Tools Operations manager Job scheduler Windows Update services Admin console Performance monitor Systems Management Server Command line interface Remote Installation services 12
13
CCS Key Features Node deployment and administration
4/17/2017 3:00 AM CCS Key Features Node deployment and administration Task-based configuration for head and compute nodes UI and command line-based node management Monitoring with Performance Monitor (Perfmon), Microsoft Operations Manager (MOM), Server Performance Advisor (SPA), and 3rd-party tools Integration with existing Windows and management infrastructure Integrates with Active Directory, Windows security technologies, management, and deployment tools Extensible job scheduler 3rd-party extensibility at job submission and/or job assignment Submit jobs from command line, UI, or directly from applications Simple job management, similar to print queue management Secure and performant MPI User credentials secured in job scheduler and compute nodes MPI stack based on MPICH2 reference implementation Support for high performance interconnects through Winsock Direct Integrated development environment OpenMP support in Visual Studio, Standard Edition Parallel debugger in Visual Studio, Professional Edition
14
4/17/2017 3:00 AM
15
4/17/2017 3:00 AM HPC Institutes Cornell Theory Center Ithaca, NY U.S.A. Southampton University Southampton, UK Nizhni Novgorod University Nizhni Novgorod, Russia National Center for Supercomputing Applications, IL U.S.A. University of Virginia Charlottesville, VA U.S.A. Tokyo Institute of Technology Tokyo, Japan University of Utah Salt Lake City, UT U.S.A. University of Tennessee Knoxville, TN U.S.A. HLRS – University of Stuttgart Stuttgart, Germany Shanghai Jiao Tong University Shanghai, PRC TACC – University of Texas Austin, TX U.S.A.
16
4/17/2017 3:00 AM An Example Of Porting To Windows Weather research and forecasting model Large collaborative effort, lead by NCAR, to develop next-generation community model with direct path to operations Applications Atmospheric research Numerical weather prediction Coupled modeling systems Current release WRFV2.1.2 ~1/3 million lines, Fortran 90 and some C using MPI, OpenMP Traditionally developed for Unix HPC systems Two dynamical cores Full range of physics options Rapid community growth – more than 3,000 registered users Operational capabilities U.S. Air Force Weather Agency National Centers for Environmental Prediction (NOAA) KMA (Korea), IMD (India), CWB (Taiwan), IAF (Israel), WSI (U.S.) .
17
WRF On Windows Motivation
4/17/2017 3:00 AM WRF On Windows Motivation Extend available systems available to WRF users Stability and consistency with respect to Linux Take advantage of Microsoft and 3rd party (e.g., Portland Group) development tools, environments WRF ported under SUA and running on development AMD64 clusters using Compute Cluster Pack Of 360k lines, fewer than 750 changed to compile and link under SUA Largest number of changes involved the WRF build mechanism (Makefiles, scripts) Level of effort and nature of tasks was not unlike porting to any new version of UNIX Details of porting experience described in a white paper available from Microsoft and at
18
An Example Of Application Integration With HPC Scaling Excel
Desktop Servers Clusters Excel Services Excel Services on Windows Compute Cluster Server 2003 Excel “12”
19
Open Spreadsheet/Snapshot
4/17/2017 3:00 AM Excel Services Browser 100% thin Excel “12” View and Interact Excel “12” client Author and Publish Spreadsheets Open Spreadsheet/Snapshot Web Services Access Custom applications
20
Excel And Windows CCS Customer requirements Example scenarios
Faster spreadsheet calculation Free-up client machines from long-running calculations Time/mission critical calculations that must run Parallel iterations on models Example scenarios Schedule overnight risk calculations Farm out analytical library calculations Scale-out Monte Carlo iterations, parametric sweeps
21
Evolution Of HPC Evolving Scenarios Key Factors
4/17/2017 3:00 AM Evolution Of HPC Evolving Scenarios Key Factors Batch computing on supercomputers Compute cycles are scarce and require careful partitioning and allocation Cluster systems administration major challenge Applications split into UI and compute parts Interactive computing on departmental clusters Compute cycles are cheap Interactive applications integrate UI/compute parts Emergence of turnkey personal clusters Complex workflow spanning applications Compute and data resources are diffused throughout the enterprise Distributed application, systems and data management is the key source of complexity Multiple applications are organized into complex workflows and data pipelines Focus on service orientation and web services IT Mgr Manual, batch execution Interactive Computation and Visualization SQL 21
22
Cheap Cycles And Personal Supercomputing
4/17/2017 3:00 AM Cheap Cycles And Personal Supercomputing IBM Cell processor 256 Gflops today 4 node personal cluster 1 Tflops 32 node personal cluster Top100 The key challenge How to program these things Concurrent programming will be an important area of investments for all of Microsoft (not just HPC) Microsoft Xbox 3 custom PowerPCs + ATI graphics processor 1 Tflops today $300 8 node personal cluster “Top100” for $2500 (ignoring all that you don’t get for $300) Intel many-core chips “100’s of cores on a chip in 2015” (Justin Rattner, Intel) “4 cores”/Tflop 25 Tflops/chip 22
23
The Continuing Trend Towards Decentralized, Networked Resources
Grids of personal and departmental clusters Personal workstations and departmental servers Minicomputers Mainframes 23 23
24
“Grid Computing” A catch-all marketing term Desktop cycle-stealing
4/17/2017 3:00 AM “Grid Computing” A catch-all marketing term Desktop cycle-stealing Managed HPC clusters Internet access to giant, distributed repositories Virtualization of data center IT resources Out-sourcing to “utility data centers” “Software as a service” Parallel databases
25
HPC Grids And Web Services
4/17/2017 3:00 AM HPC Grids And Web Services Compute grid Forest of clusters Coordinated scheduling of resources Data grid Distributed storage facilities Coordinated management of data Web Services Glue for heterogeneous platforms/applications/systems Cross- and intra- organization integration Standards-based distributed computing Interoperability and composability Microsoft is involved with GGF to help define the web service protocols and related specifications for an HPC Grid infrastructure.
26
Cluster-Based HPC 4/17/2017 3:00 AM 4/17/2017 3:00 AM
Let’s look at the traditional dedicated HPC compute cluster and see what a service-oriented version would look like. Existing web service protocols get us off the ground Need a directory services design and protocol Need a job scheduling protocol Need a data transfer protocol Need to do system management – WS-Management is Microsoft’s service-oriented approach to this topic Let’s cluster participate in the wider IT org’s management activities Is used within the cluster as well (i.e. uniform approach to both intra-cluster and whole-org sys mgmt) NOTE: internal to the cluster, services are only used for glue (at most), not for the actual applications 26
27
Intra-Organization HPC
4/17/2017 3:00 AM 4/17/2017 3:00 AM Intra-Organization HPC 27
28
Virtual Organizations
4/17/2017 3:00 AM 4/17/2017 3:00 AM Virtual Organizations 28
29
4/17/2017 3:00 AM Part 2 Drill-Down
30
Technologies Platform Administration Development
4/17/2017 3:00 AM Technologies Platform Windows Server 2003 SP1 64-bit Edition x64 processors (Intel EM64T and AMD Opteron) Ethernet, Ethernet over RDMA and Infiniband support Administration Prescriptive, simplified cluster setup and administration Scripted, image-based compute node management Active Directory based security Scalable job scheduling and resource management Development MPICH-2 from Argonne National Labs with performance and security enhancements Cluster scheduler programmable via Web Services and DCOM Visual Studio 2005 – OpenMP, Parallel Debugger Partner delivered Fortran compilers and numerical libraries
31
Head Node Installation
4/17/2017 3:00 AM Head Node Installation Head Node installs only on x64 Windows 2003 Compute Cluster Edition Windows 2003 SP1 Standard And Enterprise Windows 2003 R2 Installation Leverages appliance like functionality Scripted installation Warnings if system is misconfigured To Do list to assist with final configuration Walkthrough Windows Server 2003 is installed on the head node System may have been pre-installed using OPK User launches Compute Cluster Kit setup To Do list starts up, guiding User through next steps User joins Active Directory domain User installs IP over IB drivers for InfiniBand cards if not pre-installed Wizard assists with multi-NIC routing and configuration Remote Installation Service is configured for imaging compute nodes
32
Compute Node Installation
4/17/2017 3:00 AM Compute Node Installation Automated installation Remote Installation Service provides simple imaging solution May use third-party system imaging tools compute nodes Requires private network Walkthrough User racks up compute nodes Starts Add Node wizard Powers up a group of compute nodes Compute nodes PXE boot RIS and installation scripts will Install operating system: W2K3 SP1 Install drivers Join appropriate domain Install compute cluster software (CD2) Join cluster Exiting wizard turns off RIS
33
Node Management Not building a new systems management paradigm
4/17/2017 3:00 AM Node Management Not building a new systems management paradigm Leveraging Windows infrastructure for simple management MMC, Perfmon, Event Viewer, Remote Desktop Can integrate with enterprise management infrastructure, such as Microsoft Operations Manager Compute Cluster MMC snap-in Supports specific actions Pause Node Resume Node Open CD Drive Reboot Node Execute Command Remote Desktop Connection Start PerfMon Delete Properties Can operate on multiple nodes at once
34
Job/Task Conceptual Model
4/17/2017 3:00 AM Job/Task Conceptual Model Serial Job Task Proc Parallel MPI Job Task Proc IPC Parameter Sweep Job Task Proc Task Flow Job Task
35
Job Scheduler Stack Jobs/Tasks Admission Allocation Activation
4/17/2017 3:00 AM Job Scheduler Stack Jobs/Tasks Client Node Admission Head Node Allocation Activation Compute Node
36
4/17/2017 3:00 AM Job Scheduler Job scheduler provides two features: Ordering and allocation Job ordering Priority-based first-come, first-serve (FCFS) Backfill supported for jobs with time limits Resource allocation License-aware scheduling through plug-ins Parallel application node allocation policies Extensible Core engine based on embedded SQL engine Resource and job descriptions are based on XML 3rd parties can extend by plugging into submission and execution phases to implement queuing and licensing policies Job submission Jobs submitted via UI, API, command line, or web service Security Jobs on compute nodes execute in the security account of the submitting user, allowing secure access to networked resources Cleanup Jobs executed in Job Objects on compute nodes, facilitating cleanup
37
4/17/2017 3:00 AM Queue Management Job Management model similar to print queue management Leverage familiar user paradigm Queue management operations Delete Change properties Priority Run time # of CPUs Preferred nodes CPUs per node All in one License parameters Uniform attributes Notification
38
Networking Focusing on industry standard interconnect technologies
4/17/2017 3:00 AM Networking Focusing on industry standard interconnect technologies MPI implementation tuned to Winsock Automatic RDMA support through Winsock Direct (SAN provider required from IHV) Gigabit Ethernet Expect to be the mainstream choice RDMA + GigE offers compelling latency Infiniband Emerging as a leading high end solution Engaged with all IB vendors OpenIB group developing a Windows IB stack Planning to support IB in WHQL
39
Resources Microsoft HPC web site (evaluation copies available)
4/17/2017 3:00 AM Resources Microsoft HPC web site (evaluation copies available) Microsoft Windows Compute Cluster Server 2003 community site Windows Server x64 information Windows Server System information 39
40
© 2006 Microsoft Corporation. All rights reserved.
4/17/2017 3:00 AM © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
41
4/17/2017 3:00 AM
Similar presentations
