Data storage innovations demonstrated in collaboration with CERN openlab Dan Chester, Seagate Systems.

Slides:



Advertisements
Similar presentations
PowerEdge T20 Customer Presentation. Product overview Customer benefits Use cases Summary PowerEdge T20 Overview 2 PowerEdge T20 mini tower server.
Advertisements

PowerEdge T20 Channel NDA presentation Dell Confidential – NDA Required.
Joey Yep Technical Marketing, Seagate CSS Creating a Competitive Advantage with Cloud.
MARCO TECHNOLOGY SEMINAR Sponsored By:. 2:15-2:30 p.m. Registration and Welcome 2:30-4:30 p.m.Cisco Networking, Video and Data Center 4:30-5:15 p.m. EMC.
Seagate Hybrid Systems & Solutions
Desktop Computing Strategic Project Sandia National Labs May 2, 2009 Jeremy Allison Andy Ambabo James Mcdonald Sandia is a multiprogram laboratory operated.
NWfs A ubiquitous, scalable content management system with grid enabled cross site data replication and active storage. R. Scott Studham.
Technology Partner Nexenta and Dell #1 Software-Defined Storage and the #1 Server Platform combine to Power your Cloud Lower TCO. SSD/HDD/Hybrid. Any Protocol.
1 Exploring Data Reliability Tradeoffs in Replicated Storage Systems NetSysLab The University of British Columbia Abdullah Gharaibeh Matei Ripeanu.
1 AppliedMicro X-Gene ® ARM Processors Optimized Scale-Out Solutions for Supercomputing.
11 Establishing the Framework for Datacenter of the Future Richard Curran Director Product Marketing, Intel EMEA.
MaxTronic International Co., Ltd. May 2013 Focused on Data Protection.
© 2013 Mellanox Technologies 1 NoSQL DB Benchmarking with high performance Networking solutions WBDB, Xian, July 2013.
1 Exploring Data Reliability Tradeoffs in Replicated Storage Systems NetSysLab The University of British Columbia Abdullah Gharaibeh Advisor: Professor.
© Copyright 2010 Hewlett-Packard Development Company, L.P. 1 HP + DDN = A WINNING PARTNERSHIP Systems architected by HP and DDN Full storage hardware and.
Cisco Confidential 1 © 2010 Cisco and/or its affiliates. All rights reserved. Data Center Solutions Marketing Data Center Business Advantage Customer Proof.
Copyright 2009 Fujitsu America, Inc. 0 Fujitsu PRIMERGY Servers “Next Generation HPC and Cloud Architecture” PRIMERGY CX1000 Tom Donnelly April
SOLUTIONS FOR THE EFFICIENT ENTERPRISE Sameer Garde Country GM,India.
Meeting the Data Protection Demands of a 24x7 Economy Steve Morihiro VP, Programs & Technology Quantum Storage Solutions Group
Huawei IT : Make IT Simple, Make Business Agile
FlashSystem family 2014 © 2014 IBM Corporation IBM® FlashSystem™ V840 Product Overview.
Presented by Leadership Computing Facility (LCF) Roadmap Buddy Bland Center for Computational Sciences Leadership Computing Facility Project.
Bosch DSA Storage (based on NetApp E2700)
CERN IT Department CH-1211 Genève 23 Switzerland t Frédéric Hemmer IT Department Head - CERN 23 rd August 2010 Status of LHC Computing from.
11 CLUSTERING AND AVAILABILITY Chapter 11. Chapter 11: CLUSTERING AND AVAILABILITY2 OVERVIEW  Describe the clustering capabilities of Microsoft Windows.
Evaluating the performance of Seagate Kinetic Drives Technology and its integration with the CERN EOS storage system Ivana Pejeva openlab Summer Student.
Tackling I/O Issues 1 David Race 16 March 2010.
PHD Virtual Technologies “Reader’s Choice” Preferred product.
Extreme Scale Infrastructure
Barracuda Backup Easy Cloud-Connected Backup Version 5.4 | July 2014.
Parallel Virtual File System (PVFS) a.k.a. OrangeFS
AuraPortal Cloud Helps Empower Organizations to Organize and Control Their Business Processes via Applications on the Microsoft Azure Cloud Platform MICROSOFT.
EMC: Redefining ERP and ROI with a Virtualized SAP HANA® Deployment
Univa Grid Engine Makes Work Management Automatic and Efficient, Accelerates Deployment of Cloud Services with Power of Microsoft Azure MICROSOFT AZURE.
MERANTI Caused More Than 1.5 B$ Damage
Video Security Design Workshop:
Organizations Are Embracing New Opportunities
What is it ? …all via a single, proven Platform-as-a-Service.
OpenStack Swift Where do big data go? Eben van Zyl
WP18, High-speed data recording Krzysztof Wrona, European XFEL
DSS-G Configuration Bill Luken – April 10th , 2017
The demonstration of Lustre in EAST data system
Storage Options & Trends Mark Leedy- Director of Sales
CERN Lustre Evaluation and Storage Outlook
Andrew McCombs March 10th, 2011
OCP: High Performance Computing Project
What’s going on next door? The 2017 HPSS User Forum!
Traditional Enterprise Business Challenges
Introduction to Networks
The INFN Tier-1 Storage Implementation
Large Scale Test of a storage solution based on an Industry Standard
Maximize the value of your cloud
The Brocade Cloud Manageability Vision
Built on the Powerful Microsoft Azure Platform, Lievestro Delivers Care Information, Capacity Management Solutions to Hospitals, Medical Field MICROSOFT.
Low Latency Analytics HPC Clusters
© 2016 Global Market Insights, Inc. USA. All Rights Reserved Network Attached Storage Market to reach $20bn by 2024: Global Market Insights.
Buy Amazon AWS-Solution-Architect-Associate Questions Answers - AWS-Solution-Architect-Associate Exam Dumps PDF Dumpsprofessor.com
Dynamics 365 for Wholesale Ditribution Readiness Assessment
Intelledox Infiniti Helps Organizations Digitally Transform Paper and Manual Business Processes into Intuitive, Guided User Experiences on Azure MICROSOFT.
Scalable SoftNAS Cloud Protects Customers’ Mission-Critical Data in the Cloud with a Highly Available, Flexible Solution for Microsoft Azure MICROSOFT.
On-Premises, or Deployed in a Hybrid Environment
Datacastle RED Delivers a Proven, Enterprise-Class Endpoint Data Protection Solution that Is Scalable to Millions of Devices on the Microsoft Azure Platform.
Blockchain technology at Change Healthcare
Dell Data Protection | Rapid Recovery: Simple, Quick, Configurable, and Affordable Cloud-Based Backup, Retention, and Archiving Powered by Microsoft Azure.
One-Stop Shop Manages All Technical Vendor Data and Documentation and is Globally Deployed Using Microsoft Azure to Support Asset Owners/Operators MICROSOFT.
IBM Power Systems.
Last.Backend is a Continuous Delivery Platform for Developers and Dev Teams, Allowing Them to Manage and Deploy Applications Easier and Faster MICROSOFT.
Guarantee Hyper-V, System Center Performance and Autoscale to Microsoft Azure with Application Performance Control System from VMTurbo MICROSOFT AZURE.
The Cambridge Research Computing Service
Presentation transcript:

Data storage innovations demonstrated in collaboration with CERN openlab Dan Chester, Seagate Systems

2 Big Science drives BIG data High Energy Physics 1 PB of raw data/sec Biology : 8PB for 100M Neurons Medicine : Exascale to tackle mysteries of Protein folding Astronomy SKA: ~35 PB/yr 10 – 20 TB/s I/O Industrial Visualizations and Volume rendering (eg:Oil and gas) Climate change effects 1 – 2 EB/year

3 End-to-end cloud solutions: mobile > desktop > data center, 25,000 recoveries/mo 98%+ satisfied customers World-class integrated supply chain in 100+ locations Supplier to 8 of the 10 world’s largest cloud service providers Complete hard drive portfolio: HDD, SAS, SATA, SSD, hybrid Nearly 1 million hard drives a day of the world’s digital information $14B in Annual Revenues Technology leader: 1st in SMR*, 14,000+ patents, >$900M R&D Seagate Key Facts – leading in HDD and Storage Systems

4 Seagate ClusterStor Systems for HPC and Big Data L300 The Fastest, most efficient and Complete Lustre HPC Solution SG200 The Fastest, most efficient & complete Spectrum Scale HPC Solution SL200 The Only, most secure PFS solution for the MLS Ecosystem A200 The most efficient HPC and scalable Active Archive Solution World Class Line-Up – Each a clear leader in its class Active Archive Object Storage Multi-Level Security

5 1.6 TB/sec Lustre File System 1 TB/sec Lustre File System 500+ GB/s Lustre File System 55 PB Lustre File System 130+ GB/s Lustre File System 140+ GB/s Lustre File System 100+ PB Lustre File System

Kinetic Open Storage Platform

Kinetic: freeing the HDD from legacy storage architectures Traditional HDD: Has 1M lines of code With 100+ tunables Custom code for specific areas Video surveillance Archiving (SMR) Extended life Synchronized RAID HPC … Uses 56 periodic table elements MTBF: M hours But still uses same low-level SATA/SCSI (SAS) command set

8 Kinetic Technology Overview Storage now fully disaggregated from compute Key/Value Interface Ethernet Connectivity Standard Device Recording

9 Device Innovation up to 2 years faster For Scale Out, Object Storage Architectures as compared to current alternatives * Performance characterization still in progress Kinetic Value Proposition Performance up to 4x* faster Rack Density up to 1.5x increase Scalability Infinite Storage Servers up to 100% reduction CAPEX up to 15% reduction OPEX (Power & Staffing) up to 70% reduction TCO up to 40% reduction $$$

Kinetic openlab

11 Phase 1 Yr 1 (started March 2015) Complete Access to Kinetic hardware has been integrated into the EOS storage system. The integration is functionally complete and can be used transparently by any EOS client. High performance erasure coding with striping Phase 2 Yr 2 (underway) Achieve production deployment with TCO gains using a multi-stage deployment Find and Resolve performance and stability issues, especially under failure scenarios. Kinetic EOS integration project at CERN 1 PB CERN Kinetic Installation

12 Stage 1: Dedicated Storage Servers with Dedicated Clusters The only addition to the existing deployment is the use of the kineticio library on the storage server to expose non-local hard drives to the EOS storage system. Kinetic Progress at CERN – Stage 1 Overview: No TCO gains, ready now Motivation: Verify performance, functionality and stability of core IO library Milestones: Keep deployment active until project completion. Gather statistics about uptime, data reliability

13 Stage 2: Dedicated Storage Servers with Shared Clusters Each kinetic cluster will be mounted by multiple storage servers. Reduces the overall number of required storage servers while maintaining bandwidth. Load balancing logic will have to be employed to decide which storage server to use for an access to a kinetic cluster. Again this stage does not impact the existing infrastructure. Kinetic Progress at CERN – Stage 2 Overview: Small TCO gains, approximately 3 months development required Motivation: Prepare multi-path functionality for stage 3 Milestones: Implementation Build distributed test-suite for multi-path functionality & multiple writer concurrency resolution.

14 Stage 3: Shared Storage Servers with Shared Clusters Use existing storage servers in the current EOS deployment to provide kinetic access instead of relying on dedicated servers. The pool of storage servers mounting a specific kinetic cluster likely has to increase compared to stage two in order to prevent bottlenecks when io to locally attached hard drives happens concurrently with io to kinetic drives. Kinetic Progress at CERN – Stage 3 Overview Maximum TCO gains, ready with stage 2 Motivation: Gain cost effectiveness by applying stage 2 to existing storage servers and remove dedicated servers. This allows 10x+ expansion of storage capacity while maintaining current server infrastructure. Milestones: Stage 3 deployment for full production system

15 Using Kinetic-enabled EOS instance to provide support for another openlab project: Evaluate usefulness of "root" physics analytic framework for non- physics data in collaboration with Kings College London Storing 300 TB of genome data Erasure coded and deployed on 4x 25 drive (100TB) Kinetic Clusters Further collaboration initiated through CERN openlab

Platinum Members Silver Members Industry Ecosystem DIY Software Chassis Drive Seagate Products 5U-84 Bay 672 TB/chassis 5.3 PB/rack Mid 2016 Kinetic 8TB Transparent Solution to SMR Technology! Kinetic OneStor Mid 2016 Dual 1 or 2.5- Gbps Ethernet Collaborating around Kinetic Ecosystem Seagate Confidential