Download presentation
Presentation is loading. Please wait.
Published byKelly Hancock Modified over 8 years ago
1
Data storage innovations demonstrated in collaboration with CERN openlab Dan Chester, Seagate Systems
2
2 Big Science drives BIG data High Energy Physics 1 PB of raw data/sec Biology : 8PB for 100M Neurons Medicine : Exascale to tackle mysteries of Protein folding Astronomy SKA: ~35 PB/yr 10 – 20 TB/s I/O Industrial Visualizations and Volume rendering (eg:Oil and gas) Climate change effects 1 – 2 EB/year
3
3 End-to-end cloud solutions: mobile > desktop > data center, 25,000 recoveries/mo 98%+ satisfied customers World-class integrated supply chain in 100+ locations Supplier to 8 of the 10 world’s largest cloud service providers Complete hard drive portfolio: HDD, SAS, SATA, SSD, hybrid Nearly 1 million hard drives a day of the world’s digital information $14B in Annual Revenues Technology leader: 1st in SMR*, 14,000+ patents, >$900M R&D Seagate Key Facts – leading in HDD and Storage Systems
4
4 Seagate ClusterStor Systems for HPC and Big Data L300 The Fastest, most efficient and Complete Lustre HPC Solution SG200 The Fastest, most efficient & complete Spectrum Scale HPC Solution SL200 The Only, most secure PFS solution for the MLS Ecosystem A200 The most efficient HPC and scalable Active Archive Solution World Class Line-Up – Each a clear leader in its class Active Archive Object Storage Multi-Level Security
5
5 1.6 TB/sec Lustre File System 1 TB/sec Lustre File System 500+ GB/s Lustre File System 55 PB Lustre File System 130+ GB/s Lustre File System 140+ GB/s Lustre File System 100+ PB Lustre File System
6
Kinetic Open Storage Platform
7
Kinetic: freeing the HDD from legacy storage architectures Traditional HDD: Has 1M lines of code With 100+ tunables Custom code for specific areas Video surveillance Archiving (SMR) Extended life Synchronized RAID HPC … Uses 56 periodic table elements MTBF: 1 - 2.5M hours But still uses same low-level SATA/SCSI (SAS) command set
8
8 Kinetic Technology Overview Storage now fully disaggregated from compute Key/Value Interface Ethernet Connectivity Standard Device Recording
9
9 Device Innovation up to 2 years faster For Scale Out, Object Storage Architectures as compared to current alternatives * Performance characterization still in progress Kinetic Value Proposition Performance up to 4x* faster Rack Density up to 1.5x increase Scalability Infinite Storage Servers up to 100% reduction CAPEX up to 15% reduction OPEX (Power & Staffing) up to 70% reduction TCO up to 40% reduction $$$
10
Kinetic project @CERN openlab
11
11 Phase 1 Yr 1 (started March 2015) Complete Access to Kinetic hardware has been integrated into the EOS storage system. The integration is functionally complete and can be used transparently by any EOS client. High performance erasure coding with striping Phase 2 Yr 2 (underway) Achieve production deployment with TCO gains using a multi-stage deployment Find and Resolve performance and stability issues, especially under failure scenarios. Kinetic EOS integration project at CERN 1 PB CERN Kinetic Installation
12
12 Stage 1: Dedicated Storage Servers with Dedicated Clusters The only addition to the existing deployment is the use of the kineticio library on the storage server to expose non-local hard drives to the EOS storage system. Kinetic Progress at CERN – Stage 1 Overview: No TCO gains, ready now Motivation: Verify performance, functionality and stability of core IO library Milestones: Keep deployment active until project completion. Gather statistics about uptime, data reliability
13
13 Stage 2: Dedicated Storage Servers with Shared Clusters Each kinetic cluster will be mounted by multiple storage servers. Reduces the overall number of required storage servers while maintaining bandwidth. Load balancing logic will have to be employed to decide which storage server to use for an access to a kinetic cluster. Again this stage does not impact the existing infrastructure. Kinetic Progress at CERN – Stage 2 Overview: Small TCO gains, approximately 3 months development required Motivation: Prepare multi-path functionality for stage 3 Milestones: Implementation Build distributed test-suite for multi-path functionality & multiple writer concurrency resolution.
14
14 Stage 3: Shared Storage Servers with Shared Clusters Use existing storage servers in the current EOS deployment to provide kinetic access instead of relying on dedicated servers. The pool of storage servers mounting a specific kinetic cluster likely has to increase compared to stage two in order to prevent bottlenecks when io to locally attached hard drives happens concurrently with io to kinetic drives. Kinetic Progress at CERN – Stage 3 Overview Maximum TCO gains, ready with stage 2 Motivation: Gain cost effectiveness by applying stage 2 to existing storage servers and remove dedicated servers. This allows 10x+ expansion of storage capacity while maintaining current server infrastructure. Milestones: Stage 3 deployment for full production system
15
15 Using Kinetic-enabled EOS instance to provide support for another openlab project: Evaluate usefulness of "root" physics analytic framework for non- physics data in collaboration with Kings College London Storing 300 TB of genome data Erasure coded and deployed on 4x 25 drive (100TB) Kinetic Clusters Further collaboration initiated through CERN openlab
16
Platinum Members Silver Members Industry Ecosystem DIY Software Chassis Drive Seagate Products 5U-84 Bay 672 TB/chassis 5.3 PB/rack Mid 2016 Kinetic 8TB Transparent Solution to SMR Technology! Kinetic OneStor Mid 2016 Dual 1 or 2.5- Gbps Ethernet Collaborating around Kinetic Ecosystem Seagate Confidential
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.