HEPIX - Spring 2015 Tony Wong (BNL)
Yearly purchase cycle of hardware for RACF timed with U.S. gov’t fiscal year (October to September) Aim for delivery of new equipment by end of fiscal year to maximize value for the $ Purchase cycle has taken longer to complete in recent years 12 months in FY14 and delivery spilled into FY15 All three components of the purchase cycle are to blame (evaluation, purchase and delivery)
As late as FY 2011, the cycle was ~3 months long The FY 2014 experience Evaluation Dec to Jun ~ 7 months Purchase Jul. to Oct ~ 3 months Delivery Dec ~ 2 months Multiple hardware options to evaluate (Intel/AMD cpu’s, drive options, RAID controllers, IB, 10 GbE, etc) Long lead times with vendor (~1-2 months) to obtain parts to change hardware configuration
Onerous internal purchase procedure Multi-level approval for large $ purchase Multiple bidding cycles RACF-external technical approval required Frustrating back and forth with procurement dept. Delivery delays by supplier Required to be within 6 weeks of awarding order Actual delivery took 11 weeks Since FY11, delays are the norm rather than the exception
Of the three components (evaluation, purchase and delivery), RACF only fully controls evaluation Proposed strategy to minimize delays Shorten evaluation process by relying on remote access Address purchase delays internally at BNL Address delivery delays with supplier Purchase and delivery delays may be self- inflicted wounds – large work load near end of U.S. fiscal year
Previous experience with remote access to special purpose servers (ie, HP’s Moonshot system – results presented at Annecy meeting) Discussed remote evaluation goals with Dell and HP formal agreements signed BNL legal dept. involved Split evaluation process in two phases Initial (wide range of configurations – weed out uninteresting technologies) DONE Final (focus on specific configurations – may include physical access to systems) in progress
Use established VPN’s provided by supplier Temporary account allows access over a defined period (~2 weeks) Supplier and/or RACF installs system software RACF installs benchmark and executes Occasional re-configuration of hardware, re- installation of software and re-execution of benchmark
Dell PowerEdge 430 Dual 2.6 GHz (40 logical cores) 10 x 8 GB DDR MHz PERC H330 4 x 2 TB Seagate 7200 rpm 6.0 Gbps SATA drives HS06 Dell PowerEdge R730xd Dual E5-2690v3 2.6 GHz (48 logical cores) 8 x 8 GB DDR MHz PERC H730P 5 x 2 TB Seagate 7200 rpm 6.0 Gpbs SAS drives HS06 (462 Watts at peak load) Dell PowerEdge R730xd Dual E5-2660v3 2.6 GHz (40 logical cores) 16 x 6 GB DDR MHz PERC H730 12 x 6 TB Seagate 7200 rpm 6.0 Gpbs SATA drives HS06 (434 Watts at peak load)
HP DL160 Dual 2.6 GHz (40 logical cores) 12 x 8 GB DDR MHz SmartArray H240 4 x 4 TB 7200 rpm 6.0 Gbps SATA drives HS06 HP DL180 Dual 2.6 GHz (40 logical cores) 12 x 8 GB DDR MHz SmartArray P840 12 x 6 TB 7200 rpm 6.0 Gbps SATA drives HS06
Conclude hardware evaluation by March 30 th Purchase process: April 1 st - May 31 st Delivery of hardware by August 1 st Seek remote evaluation agreements with other suppliers (ie, Lenovo, etc) Make remote access a permanent feature of the evaluation process Use remote access to complement on-site evaluation of hardware
Do other institutions have remote access agreements? Is it possible to leverage the effort? Provide access to multiple institutions Motivate suppliers with bigger demand Legal considerations What agreements are possible between institutions and suppliers? Laws not uniform among countries Institutional rules vary within a country
In 2014, clusters were priced in July and delivered in December Standard prices for Intel Ivybridge in July Discounts after Haswell became available in October Delays affected HEPN programs at BNL Need to minimize delays to maximize purchasing power and meet deadline for deliverables Follow-up at Fall meeting