NSF Site Visit HYDRA Using Windows Desktop Systems in Distributed Parallel Computing

Introduction… Windows desktop systems at IUB student labs –2300 systems, 3 year replacement cycle –Pentium IV (>=1.6 GHz), 256/512/1024 MB memory, 10/100 Mbps/GigE, Windows XP –More than 1.5 TF NSF Site Visit

Possibly Utilize Idle Cycles? Red: total owner Blue: total idle Green: total Condor NSF Site Visit

Problem Description Once again... Windows desktop systems at IUB student labs: – As a scientific resource – Harvest idle cycles NSF Site Visit

Constraints Systems dedicated to students using desktop office applications — not parallel scientific computing – making their availability unpredictable and sporadic Microsoft Windows environment Daily software rebuild (updates) NSF Site Visit

What could these systems be used for? Many small computations and a few small messages –Foreman-worker –Parameter studies –Monte Carlo Goal: High Throughput Computing (not HPC) –Parallel runs of the aforementioned small computations to make better use of resource –Parallel libraries – MPI, PVM, etc. – have constraints if availability of resources is ephemeral i.e. not predictable NSF Site Visit

Solution Simple Message Brokering Library (SMBL) –Limited replacement for MPI Both server and client library based on TCP socket abstraction –Porting from MPI is fairly straight forward Process and Port Manager (PPM) Plus … –Condor for job management, file transfer, no checkpointing or parallelism –Web portal for job submission NSF Site Visit

The Big Picture We’ll discuss each part in more detail next… The shaded box indicates components hosted on multiple desktop computers NSF Site Visit

SMBL (Server) SMBL server maintains a dynamic pool of client process connections Worker job manager hides details of ephemeral workers at the application level SMBL RankCondor Assigned Node 0 (Foreman) Wrubel Computing Center, sacramento 1Chemistry Student Lab, computer_14 2CS Student Lab, computer_8 3Library, computer_6 SMBL Server Process Table for 4 CPU parallel session NSF Site Visit

SMBL (Server) SMBL server maintains a dynamic pool of client process connections Worker job manager hides details of ephemeral workers at the application level SMBL RankCondor Assigned Node 0 (Foreman) Wrubel Computing Center, sacramento 1Chemistry Student Lab, computer_14 2Physics Student Lab, computer_11 3Library, computer_6 SMBL Server Process Table for 4 CPU parallel session NSF Site Visit

SMBL (Client) Client library implements selected MPI-like calls –MPI_Send ()  SMBL_Send () –MPI_Recv ()  SMBL_Recv () In charge of message delivery for each parallel process NSF Site Visit

Process and Port Manager (PPM) Starts the SMBL server and application processes on demand Assigns port/host to each parallel session Directs workers to their servers NSF Site Visit

PPM with two SMBL servers (two parallel sessions) SMBL RankCondor Assigned Node 0 (Foreman)Wrubel Computing Center, sacramento 1Chemistry Student Lab, computer_14 2CS Student Lab, computer_8 3Wells Library, computer_6 0 (Foreman)Wrubel Computing Center, sacramento 1Wells Library, computer_27 2Biology Student Lab, computer_4 3CS Student Lab, computer_2 PPM (cont’d...) NSF Site Visit Parallel Session 1 Parallel Session 2

Once again … the big picture The shaded box indicates components hosted on multiple desktop computers NSF Site Visit

Recent Development Hydra cluster Teragrid enabled! (Nov 2005) –Allow TG users to use resource –Virtual Host based solution – two different URLs for IU and Teragrid users –Teragrid users authenticate against PSC’s Kerberos server NSF Site Visit

PPM, SMBL server, Condor and web portal running on Linux server –Dual Intel Xeon 3.0 GHz, 4 GB memory, GigE Second Linux server running Samba to serve BLAST database System Layout NSF Site Visit

Portal Creates and submits Condor files, handles data files Apache/PHP based Kerberos authentication URLs: – (IU users) – (Teragrid users) NSF Site Visit

Utilization of Idle Cycles Red: total owner Blue: total idle Green: total Condor NSF Site Visit

Summary Large parallel computing facility created at a low cost –SMBL parallel message passing library that can deal with ephemeral resources –PPM port broker that can handle multiple parallel sessions SMBL Homepage – (Open Source) NSF Site Visit

Links and References Hydra Portal – (IU users) – (Teragrid users) SMBL home page: Condor home page: IU Teragrid home page – NSF Site Visit

Links and References (cont’d..) Parallel FastDNAml: Blast: Meme: NSF Site Visit