Challenges of Storage in an Elastic Infrastructure. May 9, 2014 Farid Yavari, Storage Solutions Architect and Technologist

The eBay Inc. Portfolio University of Minnesota Storage Workshop

The eBay Inc. Portfolio University of Minnesota Storage Workshop

The eBay Inc. Portfolio University of Minnesota Storage Workshop

The eBay Inc. Portfolio University of Minnesota Storage Workshop

The eBay Inc. Portfolio 2014 University of Minnesota Storage Workshop 6

Size of the Managed Infrastructure 7 SAN ~ FC OLTP environment 16PB NAS ~ 6PB Cloud Object Store ~1EB by 2015 Analytics environment ~270PB ~130 enterprise SAN/NAS/FLASH storage arrays in 3 DCs ~1.4mil peak hour IOPS in SAN environment Thousands of servers with external storage 2014 University of Minnesota Storage Workshop

Foundation of an Elastic Infrastructure 8 Automated Control Plane Resource Pool –Compute –Memory –Storage –Low latency, High bandwidth interconnect Traffic Management –PCI Compliance –Security –QoS Definition: An infrastructure that can spawn, destroy, grow, shrink and move processes dynamically and efficiently within and across data centers University of Minnesota Storage Workshop

Key Technologies to Enable an Elastic Infrastructure 9 Control Plane –Virtualization / Containers/ Hardware –Orchestration of infrastructure resources –Normalization of resources Resource Pool –High Speed Networking (10Gbe, 40Gbe, 100Gbe, beyond) RDMA enabled (routable layer3) Lossless flow control –Memory Class Storage –New Media beyond Virtical Nand Traffic –Virtual Lan –Access Control Image Credit: Open StackOpen Stack 2014 University of Minnesota Storage Workshop

Key Initiatives to Enable an Elastic Infrastructure 10 Separation of Storage and Compute -Hadoop use case Software defined storage, software defined network Cloud, SLA, OLA based services –Standardization –Automation –Show/Chargeback –Self Service 2014 University of Minnesota Storage Workshop

More importantly 11 Simplicity Simplicity Scales Simplicity is the ultimate sophistication 2014 University of Minnesota Storage Workshop

Shifting Paradigm of Storage 12 TechBusBWLat1234 HDD (LFF)SAS/SATA600MB/s 1.2GB/s 3-12ms5-6TB7-8TB10TB? SSDSAS/SATA600MB/s 1.2GB/s ms4-8TB8-16TB24TB36-48TB FlashPCIE2GB/s 3GB/s 2μs - 150μs 1-16TB16TB+ Beyond Nand PCIE2+GB/s1μs -40μs 100’s ns 8-24TB24TB+ Sectors/Block/Cells -> unbound (bytes/ pages) Devices becoming consumables (cell failures vs platter failures, shrinking capacities on failure) Endurance is temporary (next 2-8 years). Interim – controllers will shield users from endurance. long term – media becomes tolerant (no read/ program disturb) 2014 University of Minnesota Storage Workshop

What about Block, File, Object and Databases? 13 Legacy support will keep them around for a long time (Is Cobol/RPG Dead yet?) Block is still the basis of storage, until Software catches up to media (2-4 years delayed after new tech introduced) Object Store is the new Block, File and Database –No Attachments required (Simplicity) –Enforceable Access Control –Only difference between Block, File and Database in an object store is the richness of metadata Maps to new technology better than Block, File and Databases today years: We’re back to the future. Processes have compute and a work area (memory) which may be persistent. Processed data stored in an object store and handed off to further processing or tiered to archival is based on policy and flow control. Scale out across a data center 2014 University of Minnesota Storage Workshop

The Challenge 14 Near Term (3-5 years) –Paradigm shift is starting. –Storage Arrays are understanding flash and flash are becoming flash not disk emulation (slow evolution) –Next steps is to model filesystems, kernels, application to understand flash Failure domains, Performance, Data placement/movement vs RAID Strategic 5+ years –Server model/ Resource model changing Disaggregation of resources brings new challenges Persistent Memory replacing standard memory requires system bring up and fault remediation challenges Object storage evolution and metadata handling – long ways to go 2014 University of Minnesota Storage Workshop

2016 Infrastructure Goals 15 Flash Everywhere: Local, Shared, Cold, Hadoop, Media etc. No FC or Infiniband. ISER/IP, 40Gbps+ethernet Fully REST API driven storage management and automation Hyperscale cloud across all classes of service (QA, Dev, Prod, ec.) Extremely high densities, 140TB+ per flash device, 54PB+ racks Two tiers of flash: High performance, and Archival flash 2014 University of Minnesota Storage Workshop