32
Accelerating Time to Results KC ZHANG Panasas Technical and Business Development Manager [email protected] Leader in Parallel Storage Systems

Accelerating Time to Results KC ZHANG Panasas Technical and Business Development Manager [email protected] Leader in Parallel Storage Systems

Embed Size (px)

Citation preview

Accelerating Time to Results

KC ZHANGPanasas Technical and Business Development Manager

[email protected]

Leader in Parallel Storage Systems

Slide 2 Panasas, Inc.

Agenda

Panasas introduction

Customer successes

Panasas solutions

Per

form

ance

Manageability

Availability

Panasas

Slide 3 Panasas, Inc.

Panasas

Primary Investors:

HQ – Silicon Valley

Market Focus:o Energy o Academiao Government o Life Scienceso Manufacturing o Finance

Technologies: parallel file system and parallel storage appliance

World wide support with over 25 global resellers

Founded by Garth Gibson in 1999. First Customer Ship in 2003

The fastest supercomputer in the world runs Panasas

Slide 4 Panasas, Inc.

Partnering to meet customer needs

Application ISVsResellers

Standards Development

Slide 5 Panasas, Inc.

Recognized Product Innovation and Excellence

10 Disruptive New Storage Technologies Promise Big Changes

6 Panasas Customers Win HPCWire Awards in 2007!

NAS Magic Quadrant

“Visionary” Top Supercomputing Achievement Roadrunner,

Los Alamos National Laboratory

Top Supercomputing Achievement Roadrunner,

Los Alamos National Laboratory

Top Collaboration Between Government and Industry Roadrunner,

Best HPC Storage Product

Top 5 Vendors to Watch in 2009

8 Panasas Customers Win HPCWire Awards in 2008!

Slide 6 Panasas, Inc.

Panasas Powers RoadRunner

Slide 7 Panasas, Inc.

RoadRunner at a Glance

Slide 8 Panasas, Inc.

NFS and

other network services,

WAN

Secure Core switches

Archive

Scalable to

600 GB/sec before adding

Lanes

IB4X

FatTree

Myr inet

Myr inet

Site wide Shared Global Parallel

File System

(Panasas)

Roadruner Phase 3

1.026 PF

1GE

IONODES

10GE

IONODES

Roadrunner Phase 1

70TF

Lightning/Bolt35 TF

FTA’sNx10GE

NxGENx10GE

4 GE per 5-8

TB

Compute Unit

Compute Unit

IO Unit

IO Unit

CU

CU

CU

CU

fail over

10GE

IONODES

Petascale Red Infrastructure Diagram with Roadrunner Accelerated FY08

Slide 9 Panasas, Inc.

Leaders in HPC choose Panasas

SWIFT ENERGYCOMPANY

Slide 10 Panasas, Inc.

The Common Themes

A. Very complex problems and simulations

B. Very large number of files being used concurrently

C. Very large number of concurrent users/servers

D. Consolidating Users and Clusters on one storage system

E. Any or all of the above

Panasas solves the most difficult storage problems while delivering very high reliability in an easy to use

appliance-like package.

Slide 11 Panasas, Inc.

Breaking Through the Bottleneck

Clusters = Parallel Compute Parallel Compute needs Parallel IO

Monolithic Monolithic Storage Storage

(NFS (NFS servers)servers)

Linux Linux Compute Compute ClusterCluster

Single data Single data path to path to storagestorage

IssuesComplex ScalingLimited BW & I/OIslands of storageInflexibleExpensive

Linear ScalingExtreme BW & I/OSingle storage poolEase of MgmtLower Cost

Parallel Parallel data data

pathspaths

Panasas Panasas Parallel Parallel Storage Storage ClustersClusters

Benefits

Linux Linux Compute Compute ClusterCluster

Slide 12 Panasas, Inc.

What is Parallel Storage?

The architecture for scale-out file storage

File Server

NFSClustered

NFS

File Server

File Server

NAS:Network Attached

Storage

Clustered Storage:Multiple NAS file servers managed

as one. Good aggregate performance.

Parallel Clustered Storage:

File server not in data path. Performance bottleneck

eliminated.

Parallel NFS

Slide 13 Panasas, Inc.

Panasas Storage Cluster:Built on Industry-Standard Components

DirectorBlade StorageBlade

Integrated 10GE Switch

Shelf Front1 DB, 10 SB

Shelf Rear

Midplane routes GE, power

Battery Module(2 Power units)

Slide 14 Panasas, Inc.

Performance and Scaling

DirectFLOW client

o Standard installable file systemo Supports all common Linux flavorso Support up to 12K clients

DirectorBlade clustero Divides namespace into virtual

volumeso Allows metadata to scale (no

bottleneck)

Demonstrated scalable performanceo 30+ GB/sec of sustained

throughput from a single filesystem

Panasas DirectFLOW® data path

Slide 15 Panasas, Inc.

Scalable NAS - NFS/CIFS

Scalable NFS/CIFS server o Load automatically distributed across

scalable DirectorBlade moduleso Scale to satisfy growing number of clientso Any DirectorBlade module can access

any fileo Slide in a new DB, instantly get more

NFS ops/sec into the same data

Access same data from any protocolo Integrates non-Linux devices into systemo 2+9 configuration typically best for NFS.

Balances CPU ops/sec with disk ops/sec

Total Time in Hours to complete the job

• 23 Million Traces

• 139GB input dataset

• 234GB output depth migrated image gathers

• 247MB per depth slice, 970 depth slices

0

50

100

150

200

250

300

350

400

Panasas Other Vendor A Other Vendor B

Ho

urs

Data Set

Throughput of Reads & Writes (MB/sec)

0

10

20

30

40

50

60

Panasas Other Vendor A Other Vendor B

MB

/ S

EC

Read Rate Write Rate

• 23 Million Traces

• 139GB input dataset

• 234GB output depth migrated image gathers

• 247MB per depth slice, 970 depth slices

Data Set

Chart Legend

Aggregate Throughput for 24 Nodes

0

200

400

600

800

1000

1200

1400

Panasas Other Vendor A Other Vendor B

MB

/ S

EC

Data Set

• 23 Million Traces

• 139GB input dataset

• 234GB output depth migrated image gathers

• 247MB per depth slice, 970 depth slices

Aggregate Read Throughput

Aggregate Write Throughput

Chart Legend

Job Time Activity

Panasas Other Vendor A Other Vendor B

Data Set

• 23 Million Traces

• 139GB input dataset

• 234GB output depth migrated image gathers

• 247MB per depth slice, 970 depth slices

Processor Waiting on Data

Computation

Chart Legend

Slide 20 Panasas, Inc.

ActiveScale Operating System

DirectFLOW® Protocolo Provides parallel data paths for maximum

performance

PanFS™ Parallel File Systemo Distributed and parallel file systemo Block management hidden behind object

storage interfaceo File management distributed across metadata

managers Designed to be managed by non-storage professionals

ActiveScan Predictive Media Managemento Continuous sweeps of all data and disk media in the StorageBladeo If discrepancies are detected the system proactively corrects the media defects

Predictive Disk Managemento Anticipates disk problems with automated, predictive failure analysis; data is moved

prior to failure, to avoid reconstruction

Real-time monitoring of client load generationo Identify performance bottlenecks among storage users

Slide 21 Panasas, Inc.

Horizontal Parity: Panasas ObjectRAID

Parity calculated and written to disk(s)

o Any failed disk can be reconstructed from the remaining disks

Panasas ObjectRAID is faster

o Uses multiple RAID controllers to run in parallel (“Parallel Reconstruction”)

Panasas ObjectRAID is more efficient

o Reconstructs only user data versus every sector on disk

800GB Blade reconstructed in 31 minutes at Los Alamos National Laboratory!

Horizontal Parity

Slide 22 Panasas, Inc.

Unique: Vertical Parity

Solves media error problem regardless of drive density

“RAID” within an individual drive

Improves on internal ECC capabilities

Independent of horizontal array-based parity schemes

Seamless recovery from media errors by applying RAID schemes across disk sectors

Vertical Parity

Horizontal Parity

Vertical Parity

Slide 23 Panasas, Inc.

Unique: Network Parity

Horizontal Parity

Vertical Parity

Network Parity

Extends parity capability across the data path to the client or server node

Enables end-to-end data integrity validation

o Protects from errors introduced by disks, firmware, server hardware, server software, network components and transmission

o Client either receives valid data or an error notification

Slide 24 Panasas, Inc.

Manageability: Single Global Namespace

Panasas removes artificial, physical and logical boundarieso Eliminates need to maintain mount scripts or move data

Single Global Namespace

Traditional Storage NetworksTraditional Storage Networks Panasas Storage ClusterPanasas Storage Cluster

Cluster 1Cluster 1 Cluster 3Cluster 3

Cluster 2Cluster 2

Archived Files

Cluster 1Cluster 1 Cluster 3Cluster 3

Cluster 2Cluster 2

Cluster 2 Results

Cluster 3 Results

Cluster 1 Results

Slide 25 Panasas, Inc.

Automatic provisioning for easy growth

Online Provisioning

o Configure One DirectorBlade and all others obtain their configuration via DHCP on private port

o New Storage is seamlessly integrated into the system

Growth without limitations

o Terabytes to Petabytes

o Single seamless namespace

Single Seamless Namespace!

DHCP on DHCP on Private PortPrivate Port

Reading ConfigReading ConfigSetting IP AddrsSetting IP Addrs

Matching VersionsMatching Versions

Slide 26 Panasas, Inc.

Manageability: Automatic RAID configuration

Per File RAID

o RAID Layout is an Attribute Stored within the Object

System assigns RAID level based on file size

o < 64 KB RAID 1 for efficient space allocation

o > 64 KB RAID 5 for optimum system performance

Automatic transition from RAID 1 to 5

o No re-striping

Two level RAID MAP, Stripe width and depth

o Automatically optimizes stripe size

Small File

RAID 1 Mirroring

RAID 5 Striping

Large File

Enables optimum system growth and reconstruction

Slide 27 Panasas, Inc.

Manageability: Dynamic Load Balancing

StorageBlade Capacity

StorageBlade Performance

DirectorBlade PerformanceBiases new data objects to new blades

Dynamically moves data objects from filled blades as needed Data objects striped

broadly for performance

Dynamically moves objects from “hot” blades

Cluster design assigns new clients to least utilized DirectorBlades

11

22

33

Slide 28 Panasas, Inc.

Proven Panasas Scalability

Storage Cluster Sizes Today (e.g.)

o Boeing, 50 DirectorBlades, 500 StorageBlades in one system. (plus 25 DirectorBlades and 250 StorageBlades each in two other smaller systems.)

o LANL RoadRunner.100 DirectorBlades, 1000 StorageBlades in one system today, planning to increase to 144 shelves next year.

o Intel has 5,000 active DF clients against 10-shelf systems, with even more clients mounting DirectorBlades via NFS. Release 3.2 will allow them to deploy up to 12,000 clients against a single system.

o BP uses 200 StorageBlade storage pools as their building block

o Most customers run systems in the 100 to 200 blade size range

Slide 29 Panasas, Inc.

Fast Deployment

Panasas Appliance Model

o Deploy solutions in hours and days vs. weeks and months

o Ireland's most powerful computer (#117 in the world) was installed in three hours and powered up in just one day, thanks to a rapidly deployable computing platform from Silicon Graphics and Panasas.

http://biz.yahoo.com/prnews/090205/sf67219.html?.v=1

Slide 30 Panasas, Inc.

ActiveScale 3.2 Released Sept 2008

Performance

10 GE switch => 50% improvement in shelf performance

Multi-core client performance tuning

Infiniband connectivity

RAID-10 volumes to optimize N-1 workloads

Reliability

Complete HA feature set with addition of NFS/CIFS Fail over

Industry leading data integrity with Vertical Parity and Network Parity

Manageability

Snapshots

NDMP support for easy backups

Slide 31 Panasas, Inc.

Summary

Parallel storage provides high performance for faster survey turnaround and more complex algorithms

o 10s of GB/s in production seismic processing data centers

o 50% performance increase per shelf with 10Gb Ethernet

Scalability to support more complex data acquisition and larger clusters

o Deployed on a single shelf on survey vessels

o 12,000 core clusters in production today

o 4PB+ systems in production today

Proven across the E&P industry

o All major ISVs: Landmark, Paradigm, Schlumberger

o Operating on 6 continents for Service Cos., NOCs, Majors and IndependentsPanasas is proven to cost effectively increase

processing throughput!

Slide 32 Panasas, Inc.

Thank You

张克诚13701026265

For more information,call Panasas at:

1-888-PANASAS (US & Canada)

00 (800) PANASAS2(UK & France)

00 (800) 787-702(Italy)

+001 (510) 608-7790(All Other Countries)