1. Jefferson Lab Site Report Sandy Philpott HEPiX Fall 07 Genome Sequencing Center Washington University at St. Louis

2. Organizational Updates Top priority is 12GeV upgrade; receiving CD-2 Lab Director, Christoph Leemann, retiring –Offer is out to a new director IT Division, including Scientific Computing group merger, one year anniversary –Operations and Development for experimental and theoretical (HPC) physics computing combined CyberSecurity Reviews, Audits, Tests ongoing, increasing –Cisco Firewall Services Modules –Enclave ACLs –Desktop network segmentation –Security clearances now required

3. Cluster Updates New HPC Cluster: “7n – 2007 Infiniband” –396 AMD dual processor nodes; 4GB RAM, 4x IB, IPMI Initially 2.6GHz dual core; 2.6.18 kernel 20% of procurement held for upgrade –add more dual-dual nodes or upgrade CPUs to dual-quads Upgraded to 1.9GHz quad core Barcelonas in October; required new kernel: 2.6.22 –IB: OFED 1.2.5 –PBS/Maui/Torque –~3200 processors 13.46 Teraflops from 380 nodes running HPL –Top 100?

4. Cluster Updates (cont) Move old HPC nodes (“3g, 4g”) into batch farm –Easier to maintain old machines as individual nodes, rather than part of a parallel system –Update Auger (JLab batch software) to support PBS Not enough LSF licenses; cost-prohibitive –Cost of power and cooling requirements of old machines v. buying more energy-efficient new ones? 384 2.8GHz P4s, 512MB RAM (from 2004 “4g”) Move ExpPhy batch farm to 2.6.22 kernel –ease sysadmin and configuration management

5. Storage Updates Online Disk –Decommissioned 49TB Panasas (ExpPhy) – too expensive –Added 6 Thumpers (3 ExpPhy, 3 HPC) Solaris & ZFS NFS – mostly v3 clients IB configuration for HPC? –Solaris »Linux incompatibilities –Linux »no zfs, but port is underway… –pNFS, NFSv4.1? Mass Storage – at almost 90% of 2PB capacity (STK w/ 9940Bs) –Postpone next silo purchase until 2010 – funding –Eject data in 2009?

6. IT Updates Network Appliance upgrade –FAS3040: 21TB SATA –Central fileserver: home, group, apps, site, mail, … 2 factor authentication - SysAdmins on service hosts –Windows: SmartCard –Linux/Solaris: CryptoCard tokens Virtualization –VMware Windows Vista – initial support phase RHEL5 for desktops

7. Challenges IPMI? RPC timeouts? –Cisco FWSM; UDP problems? WAN data transfers NOT 10GbE - not even 1GbE! Revive Grid efforts? –OSG? –ILDG? Obtaining meaningful physics computing and storage requirements …