Upload
eros
View
251
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Huawei Big Data Storage N9000. Jan Stibor Storage & Server Product Manager. 1. Definition of Big Data. 2. HUAWEI Big Data Storage Solution. 3. Application Scenarios of HUAWEI Solution. Era of Big Data Is Coming. 250 million pictures per day. Videos of 28,800 hours per day. - PowerPoint PPT Presentation
Citation preview
Jan StiborStorage & Server Product Manager
Huawei Big Data Storage N9000
2
1
2
3
Definition of Big Data
HUAWEI Big Data Storage SolutionApplication Scenarios of HUAWEI Solution
Page 3 Page 3
Era of Big Data Is Coming
By 2015, data amount will reach 8 ZB with the valued of $16.9 billion.
250 million pictures per day
Videos of 28,800 hours per day
More than 1 PB per 3D movie
200 PB per safe city6.3 million orders per day
50 million messages per day
2.9 million emails per second globally
24 PB data processed per day
Page 4 Page 4
What Is Big Data?
Veracity
Volume
Variety
Mass dataExponential growth of unstructured data
Velocity
Quick data flow and dynamic data systemReal-time analysis, not post analysis
Various data typesUnstructured data such as files, emails, and videos accounting for 90% of the data amount generated in the next 10 years
Veracity of Great dataPrediction and analysis of the future and the behavior mode
IDC: four characteristics of big data
Page 5 Page 5
Driving Force of Big Data
• Storage — open architecture, gradual scale out, and integration of near-line systems and offline systems
• Archiving — lifecycle management of mass data at a low cost and high energy efficiency
Big Data
• Maintenance — easy to use
• Analysis — compound data, unified storage of multiple data sources, unified query, and sharing of storage resources
1
2
3
4
Collection
Storage
Management
Analysis
6
Contents
1
2
3
Definition of Big Data
HUAWEI Big Data Storage Solution
Application Scenarios of HUAWEI Solution
Page 7 Page 7
HUAWEI Big Data Storage Solution
Large-capacity solutionHigh-performance solution
BigData
Mass data query and analysis solution
CSS/CSE
Analysis
Storage
Archiving
Page 8 Page 8
HUAWEI OceanStor N9000—X in 1
Scale-out DB
Scale-out NAS
Scale-out Backup
• Quick data import
• Support for JDBC and ODBC
• SQL 92/2003• Data
compression
• Linear expansion from 3 to 288 nodes
• More than 3,000,000 OPS• Capacity up to 100 PB• NFS/CIFS interfaces• HDFS interface
• Global online deduplication
• Support for LTFS
Page 9 Page 9
HUAWEI OceanStor N9000 Architecture
……
Unstructured data, object data, structured data
Unified management and namespaceLifecycle policy Index of quick file
searchRemote replication and
snapshot Global deduplication Periodic data inspection
NAS interface Database interfaceObject interface Archiving interface HDFS
Storage Analysis Archiving
Map-reduce
Apple
Big data sharing BI Big data analysis
Behavior prediction, real-time analysis, …HPCInternet …
10GE/Infiniband
Node Node Node Node Node Node
Application layer
Storage layer
Media asset
Page 10 Page 10
N9000 HardwareHigh-performance node :The scene of frequent read
and write small files
2 U high 25 disk slots2-channel hexa-core CPU48 GB memory at leastSSDs for storing metadata
High-bandwidth node :The scene of large file
sequential read and write
Large-capacity node :Near-line storage
Acceleration node
4 U high36 disk slots2-channel hexa-core CPU48 GB memory at least
4 U high36 disk slots2-channel quad-core CPU16 GB memory at least
1 U high
Front view
Rear view
Front view
Front view
Rear view
Rear view
Page 11 Page 11
N9000 Software — Wise Series
WiseLink
WiseQuota
WiseTier
1
2
3
Cluster load balancing software Support for multiple load balancing policies Optional, allowing for separate purchase
Quota management software that sets space usage limits on directories or users.
Optional, allowing for separate purchase
DST software Support for policy-based automatic migration Optional, allowing for separate purchase
WushanFS
WushanFS: Wushan File System
WushanFS Self-developed distributed file system Mandatory, one-off purchase Wise Data Protection (WiseDP)
4
Page 12 Page 12
Switch
Extensive Flexibility: WiseDP
N+1 to N+4 data protection File-level data protection1 hr/TB data recovery
+1
ECC
+3
ECC
+2
ECC
+4
ECC
Number of Nodes
ODC RDC RDN Max. No. of Allowed Faulty Disks
Max. No. of Allowed Faulty Nodes
Utilization
3 4 2 1 2 1 67%4 6 2 1 2 1 75%5 8 2 1 2 1 80%6 4 2 2 2 2 67%8 6 2 2 2 2 75%
10 8 2 2 2 2 80%14 12 2 2 2 2 86%14 10 4 4 4 4 71%20 12 4 4 4 4 75%
Original data counts (ODC)Redundant data counts (RDC)Redundant data nodes (RDN)
Page 13 Page 13
Outstanding Performance: WiseLink
...Automatic load balancing among existent and newly added nodes
Single node expansion within 60s
Without human intervention and manual modifications
Page 14 Page 14
Outstanding Performance: WiseTier
High-OPS nodes
Large-capacity nodes
Hot data Cold data
• Enables DST to work among diverse types of nodes.
• Adopts tiering policies based on I/O access popularity.
• Adopts hotspot anti-shake policy to prevent unnecessary data migration.
• Seamlessly adapts to service changes without interrupting ongoing applications.
• Consolidates multiple storage systems into a single storage system, eliminating the possibilities of frequent data migration and simplifying management.
Page 15 Page 15
Simplified Management: Refined Space Management
100 TB
20 TB
User
30 TB
User group
Centralized management of a single file system of 100 PB
Unified topology display
Comprehensive performance monitoring
Fast automatic deployment
40 TB
User group
10 TB
User
WiseQuota: sets quotas for subdirectories to facilitate space management.
20 TB
User
10 TB
User
Page 16 Page 16
HUAWEI 9000 — Adapting to Changes and Converging for Easy Management
Extensive Flexibility
Scaling from 3 to 288 nodes
Support up to 100 PB of capacity
Linear scaling of performance and capacity
Outstanding Performance
World's leading file system access performance of more than 3 million OPS
Optimal Integration
Distributed scale-out architecture
Integration of scale-out NAS, scale-out DB, and scale-out backup
Streamlined Management
Unified management of all nodes on a single interface
LCM
17
Contents
1
2
3
Definition of Big Data
HUAWEI Big Data Storage Solution
Application Scenarios of HUAWEI Solution
Page 18 Page 18
HUAWEI OceanStor N9000 Scenario— Media Asset Management
...HPC
VOD server cluster
TV set + set-top box PC PC Smart phone
VOD system
… ……
Non-linear editing system
HD editingSD editing Cataloging
EditingAudio Effect
…Query/Cataloging
Post-production system
Media asset management system
LAN
Animation rendering
N9000 high-performance nodes N9000 high-
bandwidth nodes and large-capacity nodes
TV station non-linear editing system solution
• Stable bit steams and low single-stream bandwidth, with
a strong need for data sharing
Animation rendering solution
• Frame image file storage, with a strong need for high
performance in sequential read of small-sized I/Os
VOD solution
• Concurrent reads by multiple users, with a strong need
for high bandwidth
Media asset management solution
• Large capacity, high bandwidth, and data sharing
Movie post-production solution
• Stable bid streams and high-bandwidth throughput,
providing data sharing
Focusing on five aspects
Copyright©2012 Huawei Technologies Co., Ltd. All Rights Reserved.The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice.
HUAWEI ENTERPRISE ICT SOLUTIONS A BETTER WAY