Computing Infrastructure Hardware & Software GrADS Project Athena Technical Issues Computing infustructure computing facilities grads project athena technical issues
IGES Computing Requirements Large model runs done offsite Requires high bandwidth connectivity to remote sites In house focus on data analysis Must provide shared access to data Requires large disk space Compute servers for analysis
Network Public Internet Internet 2 Firewall for all networking T-3 ~ 20Mb/s Internet 2 1Gb/s Firewall for all networking Internal Gigabit Ethernet Infiniband
Servers Dell hardware (linux) Public servers www.iges.org www.wxmaps.org www.monsoondata.org Front end servers (cola##) 8 cpu 32GB ram Public network Compute servers (cpu#) 24-32GB ram Private network
Data storage Dell Intel servers (linux) with direct attached disk Disks in RAID 6 Dual drive redundancy Gluster file-system High speed clustered file-system Open-source Data is stored on standard Linux file-systems (XFS) Shared over Infiniband network
Data access Gluster 6 servers No metadata server 100% scaling 24 disk bricks No metadata server 100% scaling ~2 GB/s aggregate throughput ~400 MB/s single stream
IGES DATA accessibility Single name space for storage, 191TB Mounted to every server All users have access to all data Saves on duplicate data sets Users on any server /homes /data Single namespace
Reliability / Backups No single point of failure Bacula for backups Dual drive redundancy 4 hour or NBD support contracts Spare parts onsite Bacula for backups All data backed up weekly 500 Slot Quantum i2000 tape loader 3 LTO-4 tape drives 2 copies of everything Up to 3000 slots, 96 drives
Future plans - hardware Data archiving 2 copies of data to tape Increased disk storage CPU servers with more cpu cores and large amounts of memory analysis 24 cores / 128 GB memory
Software stack Open source software whenever possible Servers run identical software Example open source packages R Flint On each server Operating system Compilers GrADS Home directory Data directory R Flint
Questions…