1 SIAC 2000 Program

2 SIAC 2000 at a Glance AMLunchPMDinner SunCondor MonNOWHPCGlobusClusters TuePVMMPIClustersHPVM WedCondorHPVM

3 SIAC 2000 Materials: Book 1 David Spector, “Building Linux Clusters: Scaling Linux for Scientific and Enterprise Applications,” August 2000 CD-ROM RedHat Linux, PVM, … O'Reilly & Associates, ISBN:

4 SIAC 2000 Materials: Book 2 Rajkumar Buyya (Editor), “High Performance Cluster Computing: Programming and Applications” Volume 2, June 1999 Prentice Hall; ISBN:

5 SIAC 2000 Materials: Book 3 Selection of papers –Linux clusters –Beowulf HOWTO –PVM, MPI FAQs Handouts from speakers

6 Introduction to Cluster Computing Prabhaker Mateti Wright State University Dayton, Ohio

7 Overview High performance computing High throughput computing NOW, HPC, and HTC Parallel algorithms Software technologies

8 “High Performance” Computing CPU clock frequency Parallel computers Alternate technologies –Optical –Bio –Molecular

9 “Parallel” Computing Traditional supercomputers –SIMD, MIMD, pipelines –Tightly coupled shared memory –Bus level connections –Expensive to buy and to maintain Cooperating networks of computers

10 “NOW” Computing Workstation Network Operating System Cooperation Distributed (Application) Programs

11 Traditional Supercomputers Very high starting cost –Expensive hardware –Expensive software High maintenance Expensive to upgrade

12 Traditional Supercomputers No one is predicting their demise, but …

13 Computational Grids are the future

14 Computational Grids “Grids are persistent environments that enable software applications to integrate instruments, displays, computational and information resources that are managed by diverse organizations in widespread locations.”

15 Computational Grids Individual nodes can be supercomputers, or NOW High availability Accommodate peak usage LAN : Internet :: NOW : Grid

16 Globus: A Computational Grid Lee Liming, Argonne Monday, Aug 21, 2000, 1:00 – 5:30 PM

17 “NOW” Computing Workstation Network Operating System Cooperation Distributed+Parallel Programs

18 Workstation? PC? Mac?

19 “Workstation Operating System” Authenticated users Protection of resources Multiple processes Preemptive scheduling Virtual Memory Hierarchical file systems Network centric

20 Network Ethernet –10 Mbps obsolete –100 Mbps common –1000 Mbps desired Protocols –TCP/IP

21 Cooperation Workstations are “personal” Others use slows you down … Willing to share Willing to trust

22 Distributed Programs Spatially distributed programs –A part here, a part there, … –Parallel –Synergy Temporally distributed programs –Compute half today, half tomorrow –Combine the results at the end Migratory programs –Have computation, will travel

23 Technological Bases of Distributed+Parallel Programs Spatially distributed programs –Message passing Temporally distributed programs –Shared memory Migratory programs –Serialization of data and programs

24 Distributed Shared Memory “Simultaneous” read/write access by spatially distributed processors Abstraction layer of an implementation built from message passing primitives Semantics not so clean

25 Technological Bases for Migratory programs Same CPU architecture –X86, PowerPC, MIPS, SPARC, …, JVM Same OS + environment Be able to “checkpoint” –suspend, and –then resume computation –without loss of progress

26 Development of Distributed+Parallel Programs New code + algorithms Old programs rewritten in new languages that have distributed and parallel primitives Parallelize legacy code

27 New Programming Languages With distributed and parallel primitives Functional languages Logic languages Data flow languages

28 Parallel Programming Languages based on the shared-memory model based on the distributed-memory model parallel object-oriented languages parallel functional programming languages concurrent logic languages

29 PVM, and MPI Message passing primitives Can be embedded in many existing programming languages Architecturally portable Open-sourced implementations

30 PVM, and MPI Prabhaker Mateti, WSU Tuesday Aug 22, :30 – 12:00

31 OpenMP Distributed shared memory API Implementations: Real soon now

32 SPMD Single program, multiple data Contrast with SIMD Same program runs on multiple nodes May or may not be lock-step Nodes may be of different speeds Barrier synchronization

33 Condor Cooperating workstations Migratory programs –Checkpointing –Remote IO Resource matching

34 Condor Prabhaker Mateti, WSU Wednesday Aug 22, :30 – 12:00

35 Clusters of Workstations Inexpensive alternative to traditional supercomputers High availability –Lower down time –Easier access Development platform with production runs on traditional supercomputers

36 Linux Clusters Kumaran Kalyanasundaram, SGI Monday Aug 21, :30 – 9:00 PM Tuesday Aug 22, :00 – 5:30 PM

37 HPVM Clusters Mario Lauria Wednesday Aug 23, :00 – 4:00 PM