1. Huawei Big Data Storage N9000
Jan Stibor
Storage & Server Product Manager

2. 1 2 3 Definition of Big Data HUAWEI Big Data Storage Solution
Application Scenarios of HUAWEI Solution

3. Era of Big Data Is Coming
250 million pictures per day
Videos of 28,800 hours per day
2.9 million s per second globally
24 PB data processed per day
200 PB per safe city
6.3 million orders per day
More than 1 PB per 3D movie
50 million messages per day
250 million pictures are uploaded to Facebook per day on average.
There are 200 million video playbacks on Youku per day.
The capacity of a 3D movie is more than 1 PB.
The storage capacity of a safe city is 200 PB.
Starting from 2012, data amount doubles every two years. By 2020, data amount is 44 times that of 2009.
In the first quarter of 2012 alone, the website and mobile client of gained over 0.8 billion views monthly.
During the period when Euro CUP is held, the daily average number of users that visit v.qq.com/ is 1.76 million.
If you learn about the big data market composed of servers, storage devices, networks, software, and services defined by IDC, you will discover a market that has a value of 3.2 billion dollars in 2010 and is expected to grow with the compound annual growth rate of 39.4% for five years, reaching 16.9 billion dollars in IDC said that such a growth rate is seven times that of the entire IT market within the same five years.
By 2015, data amount will reach 8 ZB with the valued of $16.9 billion.

4. What Is Big Data? Mass data Volume Velocity
Exponential growth of unstructured data
Quick data flow and dynamic data system
Real-time analysis, not post analysis
Veracity
Variety
Various data types
Unstructured data such as files, s, and videos accounting for 90% of the data amount generated in the next 10 years
 Mass data (volume), quick data flow and dynamic data system (velocity), various data types (variety), and great data value (value)
Veracity of Great data
Prediction and analysis of the future and the behavior mode
IDC: four characteristics of big data

5. Driving Force of Big Data1
Storage — open architecture, gradual scale out, and integration of near-line systems and offline systems
Big Data
Collection 2
Analysis — compound data, unified storage of multiple data sources, unified query, and sharing of storage resources
Storage 3
Archiving — lifecycle management of mass data at a low cost and high energy efficiency
Big data analysis is a technology-driven strategy that aims to obtain comprehensive, accurate, and in-depth insights into customers, partners, and business, gaining competitive strength. By processing stable real-time data flows, an enterprise is able to promptly make time-sensitive decisions, monitor the latest trend, and adjust its strategy to seize new business opportunities.
Big data is a also a big challenge to traditional data storage in terms of data collection > storage > management > cleanup > analysis.
Big data requires the following:
Big storage: distributed storage and flexible management of mass data
High performance: quick access and search
Efficient data analysis: real-time analysis, predictive and prescriptive analysis
Big analýza dat je technologie-řízený strategie, která si klade za cíl získat komplexní, přesné a do hloubky pohled na zákazníky, partnery a podniky získávají konkurenční sílu. Zpracováním stabilní real-time datové toky, je podnik schopen okamžitě provést časově citlivé rozhodnutí, sleduje nejnovější trendy a přizpůsobit svou strategii využívat nové obchodní příležitosti.
Big data jsou také velkou výzvou pro tradiční ukládání dat z hlediska sběru dat> Úložiště> Správa> čištění> analýza.
Big dat vyžaduje následující:
  Big skladování: distribuované ukládání a flexibilní řízení hromadných dat
  Vysoký výkon: rychlý přístup a vyhledávání
  Efektivní analýza dat: analýza v reálném čase, prediktivní a normativní analýza
Management 4
Maintenance — easy to use
Analysis

6. Contents 1 2 3 Definition of Big Data HUAWEI Big Data Storage Solution
Application Scenarios of HUAWEI Solution

7. HUAWEI Big Data Storage Solution
CSS/CSE
HUAWEI Big Data Storage Solution
Analysis
Storage
Archiving
Large-capacity solution
High-performance solution
Big
Data
Mass data query and analysis solution
High-performance solution
Large-capacity solution
Mass data query solution

8. HUAWEI OceanStor N9000—X in 1
Quick data import
Support for JDBC and ODBC
SQL 92/2003
Data compression
Linear expansion from 3 to 288 nodes
More than 3,000,000 OPS
Capacity up to 100 PB
NFS/CIFS interfaces
HDFS interface
Global online deduplication
Support for LTFS
Scale-out NAS
Scale-out DB
Scale-out Backup

9. HUAWEI OceanStor N9000 Architecture
Application
layer
Media asset
Internet
HPC …
Behavior prediction, real-time analysis, …
Apple ……
Unstructured data, object data, structured data
Big data sharing BI
Big data analysis
NAS interface
Object interface
Archiving interface
Database interface
Map-reduce
HDFS
Unified management and namespace
Lifecycle policy
Index of quick file search
Remote replication and snapshot
Global deduplication
Periodic data inspection
HPC: High-Performance Computing
Storage
Analysis
Archiving
Storage
layer
10GE/Infiniband
Node
Node
Node
Node
Node
Node

10. N9000 Hardware 1 U high High-performance node：
25 disk slots
2-channel hexa-core CPU
48 GB memory at least
SSDs for storing metadata
High-performance node：
The scene of frequent read and write small files
Front view
Rear view
Front view
4 U high
36 disk slots
2-channel hexa-core CPU
48 GB memory at least
High-bandwidth node：
The scene of large file sequential read and write
Rear view
4 U high
36 disk slots
2-channel quad-core CPU
16 GB memory at least
Front view
Large-capacity node：
Near-line storage
Rear view
Acceleration node
1 U high

11. N9000 Software — Wise Series
WushanFS
Self-developed distributed file system
Mandatory, one-off purchase
Wise Data Protection (WiseDP) 1
WiseLink
Cluster load balancing software
Support for multiple load balancing policies
Optional, allowing for separate purchase 2
WushanFS
WiseQuota
Quota management software that sets space usage limits on directories or users.
Optional, allowing for separate purchase 3
WushshanFS: a suite of enterprise software program used to provide a single volume, single file system, and a single namespace for forming a base for an enterprise DC.
WushanFS: Wushan
File System
WiseTier
DST software
Support for policy-based automatic migration
Optional, allowing for separate purchase 4

12. Extensive Flexibility: WiseDP
Number of Nodes
ODC
RDC
RDN
Max. No. of Allowed Faulty Disks
Max. No. of Allowed Faulty Nodes
Utilization 3 4 2 1
67% 6
75% 5 8
80% 10 14 12
86%
71% 20
Switch +1
ECC +2
ECC
Original data counts (ODC)
Redundant data counts (RDC)
Redundant data nodes (RDN) +3
ECC +4
ECC
N+1 to N+4 data protection
File-level data protection
1 hr/TB data recovery

13. Outstanding Performance: WiseLink
Automatic load balancing among existent and newly added nodes
...
Single node expansion within 60s
Without human intervention and manual modifications

14. Outstanding Performance: WiseTier
Enables DST to work among diverse types of nodes.
Adopts tiering policies based on I/O access popularity.
Adopts hotspot anti-shake policy to prevent unnecessary data migration.
Seamlessly adapts to service changes without interrupting ongoing applications.
Consolidates multiple storage systems into a single storage system, eliminating the possibilities of frequent data migration and simplifying management.
High-OPS nodes
Large-capacity nodes
Hot data
Cold data

15. Simplified Management: Refined Space Management
WiseQuota: sets quotas for subdirectories to facilitate space management.
100 TB
10 TB
20 TB
40 TB
30 TB
Fast automatic deployment
Comprehensive performance monitoring
User
User
User group
User group
Unified topology display
20 TB
10 TB
Centralized management of a single file system of 100 PB
User
User

16. HUAWEI 9000 — Adapting to Changes and Converging for Easy Management
Extensive Flexibility
Outstanding Performance
Optimal Integration
Streamlined Management
Scaling from 3 to 288 nodes
Support up to 100 PB of capacity
Linear scaling of performance and capacity
World's leading file system access performance of more than 3 million OPS
Distributed scale-out architecture
Integration of scale-out NAS, scale-out DB, and scale-out backup
Unified management of all nodes on a single interface
LCM
Adapting to changes and converging for easy management is the core design concept of the N9000. The N9000 provides a powerful scalability from 3 to 288 nodes capable of delivering up 10 PB of storage capacity. The file system access performance is expected to exceed 3 million OPS. The linear scaling of capacity and performance contributes to the unprecedented ultra-large capacity and ultra-high performance. The N9000 innovatively incorporates scale-out NAS, DB, and backup for bearing multiple types of applications and centrally manages all nodes on a single interface for providing complete LCM.

17. Contents 1 2 3 Definition of Big Data HUAWEI Big Data Storage Solution
Application Scenarios of HUAWEI Solution

18. HUAWEI OceanStor N9000 Scenario— Media Asset Management
Focusing on five aspects
Animation rendering …
Non-linear editing system
HD editing
SD editing
Cataloging
TV station non-linear editing system solution
Stable bit steams and low single-stream bandwidth, with a strong need for data sharing
Animation rendering solution
Frame image file storage, with a strong need for high performance in sequential read of small-sized I/Os
VOD solution
Concurrent reads by multiple users, with a strong need for high bandwidth
Media asset management solution
Large capacity, high bandwidth, and data sharing
Movie post-production solution
Stable bid streams and high-bandwidth throughput, providing data sharing
...
HPC
N9000 high-performance nodes
N9000 high-bandwidth nodes and large-capacity nodes
VOD server cluster
TV set + set-top box PC
Smart phone
VOD system
LAN
The solution provides over 2.2 GB/s access throughput to meet the need for design and production of 10 channels.
The storage system supports high-speed access to hotspot files.
The storage system provides the capability to manage the space and rights of production projects, allowing easy project-level and directory- level space allocation and management of project team members' data access rights.
3.6 GB/s production and rendering performance is provided. Three-level performance acceleration improves hotspot data access performance, allowing 120 rendering computing nodes to access the storage system concurrently.
Highly reliable storage is provided, with the support for 7x14 service continuity.
The scale-out architecture supports online smooth expansion.
Comprehensive capability to manage user rights and space quotas.
Snapshot, mirror, and backup provide multi-level continuous protection of critical data.
Editing
Audio
Effect …
Query/Cataloging
Post-production system
Media asset management system