DOSAR Workshop VI April 17, 2008 Louisiana Tech Site Report Michael Bryant Louisiana Tech University

4/17/2008DOSAR Workshop VI 2 Louisiana Tech University and LONI COMPUTING IN LOUISIANA

Computing locally at LTU At the Center for Applied Physics Studies (CAPS), ▫ Small eight node cluster with 28 processors  Used by our local researchers and the Open Science Grid  Dedicated Condor Pool of both 32-bit and 64-bit (w/ compat) machines running RHEL5 Additional resources at LTU through the Louisiana Optical Network Initiative (LONI) ▫ Dell 5TF Intel Linux cluster (not yet ready)  128 nodes (512 CPUs total)  Two 2.33 GHz dual-core Xeon 64-bit processors  4 GB RAM per node ▫ IBM Power5 AIX cluster  13 nodes (104 CPUs) 4/17/2008DOSAR Workshop VI 3

Focused on High Energy Physics, High Availability (HA), and Grid computing ▫ Fermilab (DØ), CERN (ATLAS), and ILC:  Dick Greenwood (DOSAR Institutional Rep.), Lee Sawyer, Markus Wobisch ▫ High Availability (HA) and Grid computing:  Box Leangsuksun  Thanadech “Noon” Thanakornworakij  Michael Bryant Researchers at Louisiana Tech 4/17/2008DOSAR Workshop VI 4

Louisiana Optical Network Initiative A 40Gb/sec fiber optics network linking Louisiana and Mississippi research universities Connected to the National LambdaRail (10Gb/sec) and Internet2 in Baton Rouge Provides 12 high-performance computing clusters around the state (9 of which are online) LONI provides Louisiana researchers with one of the most advanced optical networks in the country and the most powerful distributed supercomputer resources available to any academic community with over 85 teraflops of computational capacity. 4/17/2008DOSAR Workshop VI 5 -

LONI Computing Resources 1 x Dell 50 TF Intel Linux cluster ▫ “Queen Bee” named after Governor Kathleen Blanco who pledged $40 million over ten years for the development and support of LONI. ▫ 668 compute nodes (5,344 CPUs), RHEL4  Two quad-core 2.33 GHz Intel Xeon 64-bit processors  8 GB RAM per node ▫ Measured 50.7 TF peak performance ▫ According to the June, 2007 Top500 listing*, Queen Bee ranked the 23rd fastest supercomputer in the world. ▫ Half of Queen Bee's computational cycles is contributed to TeraGrid 6 x Dell 5 TF Intel Linux clusters housed at 6 LONI member institutions ▫ 128 compute nodes (512 CPUs), RHEL4  Two dual-core 2.33 GHz Xeon 64-bit processors  4 GB RAM per node ▫ Measured TF peak performance 5 x IBM Power5 575 AIX clusters housed at 5 LONI member institutions ▫ 13 nodes (104 CPUs), AIX 5.3  Eight 1.9 GHz IBM Power5 processors  16 GB RAM per node ▫ Measured TF peak performance 4/17/2008DOSAR Workshop VI 6 *

National Lambda Rail Louisiana Optical Network IBM P5 Supercomputers LONI Members Dell 80 TF Cluster NEXT ??? 4/17/2008DOSAR Workshop VI 7 LONI: The big picture… by Chris Womack

4/17/2008DOSAR Workshop VI 8 LONI and the Open Science Grid ACCESSING RESOURCES ON THE GRID

OSG Compute Elements LONI_OSG1 LTU_OSG Official LONI CE (osg1.loni.org) ▫Located at LSU OSG production site Connected to Dell 5TF cluster ▫Opportunistic PBS queue ▫64 CPUs out of 512 CPUs  The 16 nodes are shared with other PBS queues. Replaced LTU_CCT ▫Reason: aging hardware caps10.phys.LaTech.edu ▫Located at Louisiana Tech OSG production site Connected to our small cluster ▫Dedicated Condor Pool ▫8 nodes, 20 of the 28 CPUs  Other 8 CPUs used by local researchers. Waiting to be replaced… ▫LONI_OSG2 → Dell 5TF cluster 4/17/ DOSAR Workshop VI

Installed OSG on node setup by Sam White at LSU at the end of February Registered with the OSG and recently validated ▫ LTU_CCT was removed from OSG and VORS (3/26/08) As of last night, the LONI cluster at LSU (Eric) and LONI_OSG1 are now operational. Thanks to Sam White! Simple globus-job-run tests are successful! We will begin running DZero and ATLAS production jobs again very soon. ▫ Need to first modify jobmanager-pbs to submit jobs under our LONI allocation (using PBS -A project option) Current Status of LONI_OSG1 4/17/2008DOSAR Workshop VI 10

Current Status of LTU_OSG Upgraded to OSG Upgraded compute nodes to RHEL5 Added two new Dell Precision Workstations (16 CPUs, two quad- core 2.0GHz Xeon 64-bit processors, 16GB and 8GB) Connected to LONI 40Gbps network in June 2007 Running DZero MC production jobs (sent using Joel’s AutoMC daemon) Installed standalone Athena on caps10 for testing ATLAS analysis 4/17/2008DOSAR Workshop VI 11

4/17/2008DOSAR Workshop VI 12 Highly available CEs and PetaShare LOOKING AHEAD

Establish three LONI OSG CEs around the state: OSG Roadmap 4/17/2008DOSAR Workshop VI 13

In planning phase ▫ Located at Louisiana Tech ▫ Dual headnode using HA techniques HA framework using: ▫ Linux-HA, or HA-OSCAR ▫ DRDB (Distributed Replicated Block Device)  Think “network RAID1” LONI_OSG2 4/17/2008DOSAR Workshop VI 14

Provides 200TB disk and 400TB tape storage ▫ dCache/SRM and DQ2 services Initial 10TB at each OSG site for ATLAS production More details tomorrow in Tevfik Kosar’s talk… PetaShare storage 4/17/2008DOSAR Workshop VI 15

4/17/2008DOSAR Workshop VI 16 QUESTIONS / COMMENTS?