Platform Disaggregation Lightening talk Openlab Major review 16 th Octobre 2014

Background: the CERN *aaS catalog Platform disaggregation- 2 OpenStack LSF s/w build … Interactive AFS Ceph CASTOR EOS Hadoop … Netapp Exchange DFS TS AD … Lync JIRA Drupal Twiki Indico CATIA Ansys … Oracle CDS Kibana ElasticSearch TSM Puppet Foreman Git BOINC CERNBOX CVMFS Grid Services LQCD

Background: CERN IT assets Platform disaggregation- 3 Server & StorageMeyrinWigner Number of Cores93,93720,544 Number of Drives65,71610,921 Number of Memory Modules66,16710,247 Number of 10G NIC3,7081,211 Number of 1G NIC18,7762,292 Number of Processors18,0182,570 Number of Servers9,8081,288 Total Disk Space (TB)99,32932,584 Total Memory Capacity (TB)34483 Tape & networkMeyrin&Wigner Tape Drives141 Tape Cartridges50,622 Data Volume on Tape (TB)98,552 Free Space on Tape (TB)13,582 Routers (GPN)135 Routers (TN)24 Routers (Others)101 Star Points631 Switches3,225

Background: Platform customizations Platforms –Base node (Cloud, batch worker, web services, …) 2x CPU (e.g. Intel E5-26xx v2,3) 64GB RAM 2x 2TB HDD On-board 1GbE + dedicated IPMI –Disk storage (JBOD) front-end Base node + SAS HBA (LSI e) + 10GbE (SFP+) –Tape server Base node + 10GbE + 8Gbs FC –TSM front-end Base node + SAS HBA + 10GbE + 8Gbs FC (tape) –Oracle DB server Base node + 64GB RAM + 2x dual port 10GbE (SFP+) + RHEL certification –HPC Base node + 64GB (or 192GB) + low-latency 10GbE –Windows server Base node + RAID (int or int/ext) –“Fat” cloud node Base node + 64GB RAM + 10GbE (SFP+ or RJ45) + SSD –…–… Challenge: achieve all those customizations starting from monolithic base platforms Platform disaggregation- 4

Open Compute Project (OCP) is an interesting new direction with –Potential far-reaching impact for industry and data centres –A constantly growing provider community and private customer space Encouraging results from our small-scale tests with two twin systems –Sufficiently interesting to motivate launching a project for larger deployment Platform is still monolithic –Except rack level power distribution Could Open Compute help? Open Compute at CERN 21/05/2014

Breaking Up the Monolith? Platform disaggregation- 6 Intriguing paragraph from an Open Compute announcement (OCP summit January 2013): ( ) … “But most exciting of all are a series of new developments that will enable us to take some big steps forward toward better utilization of these technologies. One of the challenges we face as an industry is that much of the hardware we build and consume is highly monolithic -- our processors are inextricably linked to our motherboards, which are in turn linked to specific networking technology, and so on. This leads to poorly configured systems that can't keep up with rapidly evolving software and waste lots of energy and material. To fix this, we need to break up some of these monolithic designs -- to disaggregate some of the components of these technologies from each other so we can build systems that truly fit the workloads they run and whose components can be replaced or updated independently of each other. Several members of the Open Compute Project have come together today to take the first steps toward this kind of disaggregation:” … (*) (*) More technical details in “Design guide for photonic architecture” contributed by Intel to Open Compute project in 2013:

Openlab-V scope Disaggregation for enabling provisioning of customized hardware platforms? Vision: Software Defined Platform Flexible provisioning / sustainment –Commissioning –Repurpose –Connectivity and routing domains Flexible component lifecycle management –Trays of network components (NICs, switches, fabrics, …) –Trays of storage (HDD, SSD, NVMe, …) –Trays of memory (RAM, NVRAM, …) –Trays of processors sockets (Xeon, Xeon Phi, Atom, ARM, …) Scalable, Open and Affordable –Scalable performance and manageability –Open competitive manufacturer / supplier ecosystem –Affordable also underneath the hyper-scale pedestal Platform disaggregation- 7 Long-term

Objectives and timescale Phase 1, ~PM12:Disaggregated ToR Model –OCP rack enhanced with a disaggregated Top of the Rack switch Phase 2, ~PM24: Disaggregated storage –Prototype enhanced with SSD storage sleds are added to the switch fabric Phase 3, ~PM36 or beyond: Disaggregated system memory –Add second level memory hierarchy on NVRAM sleds? Would other research organisations want to participate? Platform disaggregation- 8