1. 1 Windows Compute Cluster Server 2003 Guilherme Carvalhal Gerente Acadêmico Microsoft Brasil gcarva@microsoft.com www.microsoft.com/hpc

2. www.microsoft.com/hpc 2  The Definition of HPC  WCCS2003 Launch!  Architecture  Security  User Interface  Task Manager  Developing using Visual Studio 2005  Microsoft Academic Programs Agenda

3. 3 Windows Compute Cluster Server 2003 The Definition of HPC

4. www.microsoft.com/hpc 4  HPC is broadly defined as the technology that is used to provide solutions to problems that need:  Significant computational power  To quickly access and process large amounts of data  To operate interactively across a geographically distributed network  The goal of HPC is to reduce the execution time of a compute intensive or data intensive application  The market for HPC was small & niche. In the past 15 years, this market grew significantly  HPC has changed in the past 15 years due to improvement in:  Hardware performance  Hardware price  Software performance and usability Definition and History of HPC

5. www.microsoft.com/hpc Definition and History of HPC 199119982005 SystemCray Y-MP C916Sun HPC10000Shuttle @ NewEgg.com Architecture 16 x Vector 4GB, Bus 24 x 333MHz Ultra-SPARCII, 24GB, SBus 4 x 2.2GHz x64 4GB, GigE OS UNICOSSolaris 2.5.1Windows Server 2003 SP1 Performance~10 GFlops Top500 # 1500N/A Price$40,000,000$1,000,000 (40x drop)< $4,000 (250x drop) Customers Government LabsLarge EnterprisesEvery Engineer & Scientist Applications Classified, Climate, Physics Research Manufacturing, Energy, Finance, Telecom Bioinformatics, Materials Sciences, Digital Media

6. www.microsoft.com/hpc 6 Areas where HPC is used  A few of the areas where HPC is used are:  Aerospace and Automotive  Earth and Space Science  Seismology  Bioinformatics  Chemistry  Rendering

7. 7 Windows Compute Cluster Server 2003 Launched on June 2006 !!!

8. www.microsoft.com/hpc Microsoft Compute Cluster Server  Vision  Solution for aplications that uses intensive compute tasks.  To help scalate using a cluster of computers.  Mission Statement  Empowering end users by allowing them to easily harness distributed computing resources to solve complex problems.  Platform  Based on Windows Server 2003 SP1 64 bit Edition.  Suport for Ethernet, Infiniband and others (better than Winsock Direct).  Administration  Setup and administration simplified.  Administration based on images + scripts.  Security based on Active Directory.  Job scheduling and resources administration.  Development  Cluster scheduler via.NET and DCOM.  MPI2 stack with a better performance and security for parallel applications.  Visual Studio 2005 – OpenMP, Parallel Debugger.

9. www.microsoft.com/hpc 9 Hardware Requirements  CPU (x64 bit processor architecture)  Intel Pentium or Xeon family (EM64T)  AMD Opteron and Athlon family  Memory  512 MB minimum, 32 GB RAM maximum  Multiprocessor support  Up to 4 processors per server  Disk capacity  4GB disk capacity for setup  Disk Volumes  Two volumes on head node (if using RIS) and single volume on nodes.  Network Interface Cards  At least one network interface card required on all the nodes.

10. www.microsoft.com/hpc 10 Software Requirements  Operating Systems (Cluster Nodes)  Microsoft Windows Server 2003 Compute Cluster Edition  Microsoft Windows Server 2003 Standard x64 Edition  Microsoft Windows Server 2003 Enterprise x64 Edition  Microsoft Windows Server 2003 R2 Standard x64 Edition  Microsoft Windows Server 2003 R2 Enterprise x64 Edition  Operating Systems (Remote Workstation)  Microsoft Windows XP Professional with Service Pack 2  Microsoft Windows XP Professional x64 Edition  Microsoft Windows Server 2003 with Service Pack 1, Standard Edition  Microsoft Windows Server 2003, Standard x64 Edition  Microsoft Windows Server 2003 with Service Pack 1, Enterprise Edition  Microsoft Windows Server 2003, Enterprise x64 Edition  Microsoft Windows Server 2003 R2, Standard Edition  Microsoft Windows Server 2003 R2, Standard x64 Edition  Microsoft Windows Server 2003 R2, Enterprise Edition  Microsoft Windows Server 2003 R2, Enterprise x64 Edition

11. www.microsoft.com/hpc 11  Remote Installation Service  Active Directory  Software Development Kit  Visual Studio 2005 (recommended)  Hotfixes  ICS hotfix Q897616  RIS Beta Hotfix for Compute Cluster Server  Additional Software Requirements .NET Framework 2.0  Microsoft Management Console (MMC) 3.0 Pre-Release Software Requirements

12. www.microsoft.com/hpc 12 Software Components  Compilers  GNU Compiler Collection  PGI Compilers  Intel Compilers  Microsoft Visual C++  UNIX Environment for Windows  Interix and Windows Services for UNIX  Cygwin  MKS Toolkit  Optimization and Tuning Software  Intel VTune  Intel Thread Checker  MSTI SeeWithin/Pro  Microsoft Tools  Spy++  PView  Communication Libraries  Threading Libraries  OpenMP API  Native Windows Threading  Message Passing Libraries  MPI  MPI/Pro  MPICH  MSMPI

13. 13 Windows Compute Cluster Server 2003 Architecture

14. www.microsoft.com/hpc 14 Topology of WCCS

15. www.microsoft.com/hpc 15 Winsock Direct Topology

16. www.microsoft.com/hpc 16 Communication Components  Computers in a cluster can be connected in one of the six communication topologies:  Star  Crossbar  Ring  2D Hypercube  Fully Connected  Mesh / Grid

17. www.microsoft.com/hpc 17 Communication Hardware  Bus  Buses communicate between the various components in a computer  Technologies that enable distributed computing  Hubs  Switches  Routers  Interconnects  Fast Ethernet  Giganet  Myrinet  InfiniBand Bus CPU 1 CPU 2Memory 1Memory 2

18. 18 Windows Compute Cluster Server 2003 Security

19. www.microsoft.com/hpc 19 Some Details about Security  Permissions on files and folders on the file server that is connected to both the head nodes and the compute nodes.  Secure movement of files from personal computers back and forth to the secure file server.  Authentication of users on compute nodes so that jobs can be run remotely on these computers.  User management  Human and programming interfaces  Program run levels  User level, kernel, Admin mode  Dynamic access to resources

20. www.microsoft.com/hpc 20 General Checklist for Security Testing  Check the physical security of the setup.  Permissible personnel are aware of the security standards and rules.  Security settings for the applications are set and documented.  Security logging has been implemented and its impact on the system has been assessed.  Appropriate level of directory security is set.  Check whether the team has procedures for contacting key decision makers in the event of a major problem or security threat.

21. www.microsoft.com/hpc 21 Security in Compute Cluster Server  Integration with production Active Directory.  Job execution credentials.  Security  Job executes based on submitters credentials  Credentials cache  DPAPI (credentials storage)  Exception handling  Re-queue

22. 22 Windows Compute Cluster Server 2003 User Interface – Control Panel

23. www.microsoft.com/hpc 23 WCCS Components  Head Node  Compute Node  Job Scheduler  Management Infrastructure  Compute Cluster Administrator and Job Manager  Command Line Interface

24. www.microsoft.com/hpc 24 Installing and Configuring Head Node Head Node Node

25. www.microsoft.com/hpc 25  Configuring the Cluster Installing and Configuring Head Node

26. www.microsoft.com/hpc 26  Selecting Network Topology Installing and Configuring Head Node

27. www.microsoft.com/hpc 27 Services on Nodes  Head Node  Compute Cluster Management Service  Compute Cluster Scheduler Service  Compute Cluster SDM Store Service  Compute Cluster MPI Service  Compute Cluster Node Manager Service  Compute Nodes  Compute Cluster Management Service  Compute Cluster MPI Service  Compute Cluster Node Manager Service

28. www.microsoft.com/hpc 28 Cluster Control

29. www.microsoft.com/hpc 29 Run a sample code on the Cluster

30. www.microsoft.com/hpc 30 Management of WCCS  Remote Desktop Sessions

31. www.microsoft.com/hpc 31 Management of WCCS  System Monitor  This page displays performance monitoring data for the cluster

32. 32 Windows Compute Cluster Server 2003 Task Manager

33. www.microsoft.com/hpc 33 Defining a Job  Job: “A collection of tasks”  A task represents the execution of a program on specified compute nodes

34. www.microsoft.com/hpc Job Activation  State transition during job execution on compute node

35. www.microsoft.com/hpc 35 Job life cycle in WCCS

36. www.microsoft.com/hpc 36 Create a new Job

37. www.microsoft.com/hpc 37 CCPAPI with XML  Submission and serialization of jobs using CCPAPI and XML structures known as Templates.  Templates are XML structures that capture the essence and relationship of Tasks to Jobs, Jobs to Nodes and Nodes to Cluster.  The job and task XML schemas definitions (XSD) are provided in the SDK.  CCPAPI provides methods to create and schedule jobs based on these templates <Job xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" AllocatedNodes="1 NODE1 1 " Id="50" Name=“CLUSTER\user1:May 29 2006 1:34PM "SubmittedBy="CLUSTER\user1" Status="Finished" User="CLUSTER\user1"> <Task AllocatedNodes="1 NODE1 1 " CommandLine="cmd.exe /c echo helloworld" Id="1" ParentJobId="50" Status="Finished">

38. 38 Windows Compute Cluster Server 2003 Developing using Visual Studio 2005

39. www.microsoft.com/hpc 39 Tuning Area  Five areas of focus to improve performance

40. www.microsoft.com/hpc OpenMP  For Multi-CPU with shared memory  Based on pragmas on C/C++ (VS2005) #pragma omp parallel for for (i=1; i<n; i++) b[i] = (a[i] + a[i-1]) / 2.0; #pragma omp parallel sections { #pragma omp section xaxis(); #pragma omp section yaxis(); #pragma omp section zaxis(); }

41. www.microsoft.com/hpc 41 Message Passing Interface – MPI  Has involved about 60 people from 40 organizations  A specification for message passing routines  Provides portability and ease-of-use

42. www.microsoft.com/hpc 42 Message Passing Interface – MPI  Goals of MPI  Design a compiler independent API.  Allow efficient and reliable communication.  Scope for implementations to support heterogeneous environment.  Interface design to be language independent  Convenient C and Fortran bindings for the interface.  When not to use MPI  Cases where “data parallel” is more efficient.  Cases where library functions of parallel code are already available for specific domain problems.  No scope for parallel programming.

43. www.microsoft.com/hpc 43 MSMPI  MSMPI is an implementation of MPI2 by Microsoft.  Compatible with MPICH2 from Argonne National Lab.  Security feature in MSMPI is the differentiator.  Parallel debugging of MPI application through Microsoft Visual Studio 2005.

44. www.microsoft.com/hpc #include "mpi.h" #include int main(int argc, char *argv[]) { int done = 0, n, myid, numprocs, i, rc; double PI25DT = 3.141592653589793238462643; double mypi, pi, h, sum, x, a; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); while (!done) { if (myid == 0) { printf("Enter the number of intervals: (0 quits) "); scanf("%d",&n); } MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD); if (n == 0) break; Example: Calculating PI Initialize “Correct” pi Start MPI Get # procs assigned to this job Get proc # of this proc On proc 0, ask user for number of intervals Compute Send # of intervals to all procs

45. www.microsoft.com/hpc h = 1.0 / (double) n; sum = 0.0; for (i = myid + 1; i <= n; i += numprocs) { x = h * ((double)i - 0.5); sum += 4.0 / (1.0 + x*x); } mypi = h * sum; MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); if (myid == 0) printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); } MPI_Finalize(); return 0; } Sum this proc’s share of the intervals Sum all proc intervals Report On proc 0, print the estimated value of pi and the deviation from the “correct” value Example: Calculating PI

46. 46 Microsoft Academic Programs WCCS 2003 Access to Academia

47. www.microsoft.com/hpc 47 Academic Programs Running Together  Main Value Proposition: To get jobs for students. www.microsoft.com/brasil/comunidadeacademica

48. www.microsoft.com/hpc 48 Academic Framework  Access to Content  Access to Microsoft Software www.microsoft.com/brasil/comunidadeacademica

49. www.microsoft.com/hpc 49 Academic Programs Website Academic Website http://www.microsoft.com/brasil/comunidadeacademica Academic Forum http://forums.microsoft.com/ComunidadeAcademica Informativo Acadêmico Subscribe now!

50. 50 Windows Compute Cluster Server 2003 Thank you!!! Guilherme Carvalhal Gerente Acadêmico Microsoft Brasil gcarva@microsoft.com www.microsoft.com/hpc Microsoft HPC website http://www.microsoft.com/hpc Public Newsgroup nntp://microsoft.public.windows.hpc Comunidade Acadêmica | Brasil http://www.microsoft.com/brasil/comunidadeacademica