1. Jefferson Lab Site Report Kelvin Edwards Thomas Jefferson National Accelerator Facility Newport News, Virginia USA Kelvin.Edwards@jlab.org 757-269-7770 http://cc.jlab.org HEPiX – October, 2004

2. Central Computing Email –Distracted by SPAM problem –Evaluated and purchased MXLogic Offsite solution Filters virus/spam before getting to Lab –Upgraded our email hardware Windows builds –Purchased MS Enterprise Agreement –Developed an automatic build process –Upgrading all of our systems to Windows XP –Still evaluating SP2, problems with CAD, etc.

3. File Server Storage Adaptec 2200S Raid and Linux XFS –Linux kernel 2.6 and Adaptec firmware (build 7244) It doesn’t work (I/O errors, etc.) –RedHat EL3 WS kernel works fine, but no XFS support –Tested ext3 performance unacceptable (20MB/s read, 34MB/s write) XFS performance (approx 100MB/s read/write) –Dropped back to prior Adaptec BIOS and 2.6 kernel works fine

4. File Server Storage (cont) Purchased 2 StorageTek B280 systems –14 TB of disk space –4 Sun V210 head units –Stable, but slow, NFS performance Aggregate -- 6MB/s write, 63MB/s read Each node -- 0.13 MB/s write, 1.4MB/s read average

5. File Server Storage (cont) Evaluating 10TB Panasas system –Tested 2 protocols (directFLOW and NFS) –No directFLOW problems –NFS finally stable at version 2.1.4c –Good performance with either Aggregate -- 160-185MB/s write, 100-180MB/s read Each node – 3.5 - 5MB/s write, 2.5 - 4.5 MB/s read

6. Jasmine Changes Jasmine is Jlab’s mass storage system (disk+tape) stores ~1PB and can routinely move 20TB/day. Disk cache system recently rewritten for performance and reliability –I/O load spread out over pool of many disk servers –Files belong to file groups (per experiment) with quotas –Quotas may be exceeded if there is enough disk space; allows more flexible use of disk –Files deleted from servers in a modified LRU fashion –Files may be pinned until used by the batch farm

7. Jasmine changes (2) New programmatic interfaces for –Batch Farm (Auger) –Other services that need to move files (SRM, DAQ, LQCD disk cache) More reliance on MySQL database; concurrency and load are challenging Writing 9940B tapes Experiment data rates now ~30MB/sec

8. Auger Changes Auger is Jlab’s Batch farm management system. Uses LSF to run jobs, keeps accounting in a database for web or command line presentation. Users can submit thousands of jobs using a compact job description that includes file retrieval and storage. Interfaces with Jasmine to stage files to disk before the job runs on the farm to keep CPUs busy

9. Jasmine & Auger Web Interface Java Server Pages

10. Projects Email upgrade –Still evaluating software/hardware Desktop systems –MacOS-X –Linux, Unix –Windows Power/Cooling issues –Reached limit of current Computer Room –New Computer Center to open in Jan 2006 –Increased power requirements for 800 MHz FSB systems 1.3A to 2.1A (single CPU) 1.6A to 2.8A (dual CPU) –Shutdown problems with non-ACPI enabled systems