Scale-out Storage, Simplified Meteorological data Lifecycle Management Laijunchen Market manager Cloud Storage Domain Huawei IT product line

MERANTI Caused More Than 1.5 B$ Damage 96 People be Killed /wounded /disappear 2.56 B$ Property has been damaged

Could "Tornado Hunter" Become A Reality? <Globe lightning> Could "Tornado Hunter" Become A Reality? At least 5 days in advance to predict whether the typhoon will cause serious disaster Sufficient condition: Global Data Collection Base on Decades data 刘慈欣 2015

How to improve the efficiency of Hundreds PB’s data processing? Could The Existing Meteorological Data Processing Architecture Achieve That Goal? Hot Data (Online) Data in Silos too difficult to use the data Cold data (Offline) Archive FC-SAN Tap only 2-4 Weeks data Retrieve More than 20 years data Expensive Hard to scale Poor performance Hard to manage How to improve the efficiency of Hundreds PB’s data processing? Is that be affordable?

The perfect Data Management Architecture Unified Data Pool A unified namespace can directly provide data storage and data sharing service; Easy to expand the capacity, and Store huge amounts of data with lower TCO; All data can been accessed directly at whole lifecycle, so the business systems may access to any range of spatial and temporal data as input as they need. Hot Data (SSD) Automatic data tiering Warm Data (HDD: Last 3 Years data ) Be Accessed Directly Cold Data (High density, energy saving? Other data） New Meteorological data lifecycle Model

Retrieve before access Scale-out Storage, Simplified Meteorological Data Lifecycle Management (Phase 1) Hot Data (Online) Warm Data(Online) Cold data(Offline) Archive Scale-out Storage FC-SAN … Tap only 2-4 Weeks data Last 3 Years data More than 20 years data Retrieve before access Middle range TCO Easy to use, Scale and manage Poor performance Hard to manage Expensive Hard to scale

Hundreds PB Unified Namespace Data on demand: Hundreds PB pool, Data Always Online Media CCTV HPC Web Disk Archive Behavior Analysis Precision marketing Business promotion Apps File Object Big data NFS CIFS FTP Swift S3 HDFS File Service Object Service Big data Service Software Hundreds PB Unified Namespace Hardware Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node 100 PB unified namespace can provide data storage and data sharing service

Flexible Network and Diversify Hardware Servers Front end network Nodes of scale out storage Throughput Cost per TB C72 C36 P36 P12 P25 Front end network P25 2U 25Bay C72 4U 72Bay 10GE/Infiniband supported both front-end and back-end network Diversify Hardware balance performance, capacity, and cost

Unified Namespace, Let Data Management Easier Capacity quota File quantity quota 100TB Soft quota Hard quota Numerical forecast Pressure analysis User quota Directory quota Cloud chart User group quota 40 TB 5 PB 200 TB Varied quota policies makes space Management easier NIS, AD, LDAP domain control, translate to Business centric data management. Thin provisioning. Temperature 50 TB 6TB 10TB 8TB 5TB User a User b User c User d Business centric data management, all data can be accessed form Standard interface

Distributed RAID (EC) Provide High Availability and Excellent Performance 400GB/s +1 +3 +1 +2 +4 +2 Capacity:100 PB +3 4 node failure tolerance, 1TB/Hr Rebuild speed 3 to 288 nodes, up to 400GB/s throughput

Data Tiering, Balance The Performance And Cost SSD Creation time Access time File size I/O popularity File name I/O count SAS P Serial Hardware Modification time User-defined SATA The value of data determines its storage location, achieving optimal resource configuration. Free and transparent data flowing Ample policies precisely divide tiers. C Serial Hardware

Success Story At NMSC(China) FY3C 99 Nodes FY3D 5.5 PB Capacity 20% Improvement

Scale-out Storage, Simplified Meteorological Data Lifecycle Management (Phase 2) Using High Performance Node to store Hot data to replace the High-end FC-SAN Huawei will release All Flash Node for Scale-out Storage with 5GB/s Bandwidth and 30K OPS soon Automatic data tiering between hot-data-layer and warm-data-layer Automatic tiering Hot Data (SSD) Warm Data (HDD: Last 3 Years data ) Scale-out Storage Archive Retrieve Cold data (More than 20 years data) Tape

Scale-out Storage, Simplified Meteorological Data Lifecycle Management (Phase 3) Automatic tiering Hot Data (SSD) Warm Data (HDD: Last 3 Years data ) Scale-out Storage All data managed by Scale-out Storage Data could archive to Public cloud and transparent to Applications Cold Data (High density） Automatic Archive/Retrieve Public Cloud

Huawei Scale-out Storage Architecture Evolution, Makes Data Manage Easier 2016 Block/File/Object as a Service (Control plane) 2015 + + Distributed Block Converged Distributed Block/File/Object Storage Distributed File Distributed Object (Data plane) Commodity Server 2012 2013 (Hardware) 2014 Decouple Control and Data Plane Decouple Software and Hardware Commodity Server

Huawei Scale-out Storage Technology Evolution World first NAS with 25Gb Ethernet Fully Converged Cloud Storage All Flash Node for Scale-out Storage 100Gb Ethernet,32TB SSD Release 2015 2016 2017