Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
INTEL High Performance Data Division 1
HPC Technologies for Big Data
Brent Gorda GM High Performance Data Division March 2013
INTEL High Performance Data Division 2
Introduction
One year ago:
- Community
- Roadmap
- Current Development
- Looking Forward
Whamcloud’s goal was to preserve Lustre for the community.
INTEL High Performance Data Division 3
Along the way – Intel Agreed
July 13, 2012
The move to Intel completes the journey started by Eric Barton, Robert Read and myself in 2010
Lustre is on stable ground again
INTEL High Performance Data Division 4
Intel and Lustre
• Intel continues the Whamcloud business model: • Multi-vendor Lustre development
• Open source, community friendly, single source tree
• Staff retention at 98% since acquisition (lost 1)
• Intel supports all Whamcloud business (incl. Fastforward)
• A member of EOFS (Whamcloud’s position)
• Recently joined OpenSFS as a promoter ($500K)
“Intel bought Whamcloud to be Whamcloud” – Boyd Davis
INTEL High Performance Data Division 5
Dir, Datacenter Computing Paresh Pattani
Deb Christensen, Administrative Assistant
Datacenter Software Division (DSD) Boyd Davis VP, GM DSD
HR Lea Ann Hansen
Finance Fawwad Qureshi
Legal TBD
GM., Enterprise Software Strategy Pauline Nist
GM, Intel Hybrid Cloud Bridget Karlin
CTO, DSD & GM, Big Data Software & Services Girish Juneja
Dir., Datacenter Management Jeff Klaus
GM, Attached Platform Storage Software
Steve Dalton
Dir., Manageability Software Ben Vrvilo
Curtis McKee, Technical Assistant
GM., High Performance Data Brent Gorda
Dir, Solutions and Technologies Lynn Comp
CTO, HPC Ecosystem Mark Seager
INTEL High Performance Data Division 6
Evolving Lustre Community
An increasing number of organizations contribute
http://git.whamcloud.com/?p=fs/lustre-release.git;a=summary
0 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000
[2.1]
2.1
2.2
2.3
Bull
CEA
Cray
DDN
EMC
Fujitsu
LLNL
NICS
ORNL
TACC
Ultrascale
UVT
Whamcloud/Intel
Xyratex
Sun/Oracle
INTEL High Performance Data Division 7
INTEL High Performance Data Division 8
Department of Energy - Fast Forward Challenge
• FastForward RFP provided US Government funding for Exascale research and development
• Sponsored by 7 leading US national labs
• Aims to solve the currently intractable problems of Exascale to meet the 2020 goal of an Exascale machine
• RFP elements were Processor, Memory and Storage
• Whamcloud won the Storage (filesystem) component – HDF Group – HDF5 modifications and extensions – EMC – Burst Buffer manager and I/O Dispatcher – Cray - Test
• Contract renegotiated on Intel acquisition of Whamcloud – Intel - Arbitrary Connected Graph Computation – DDN - Versioning OSD
INTEL High Performance Data Division 9
Jitter Bulk synchronous programming
– Simplifies application development – Susceptible to jitter – Makes strong scaling harder
Asynchronous programming – Loosen coupling between
processes – No waiting at barriers – Closes “gaps” provided jitter
balances out over time
INTEL High Performance Data Division 10
Transactions
Consistency and Integrity • Guarantee required on any and all failures
– Foundational component of system resilience • Required at all levels of the I/O stack
– Metadata at one level is data to the level below
No blocking protocols • Non-blocking on each OSD • Non-blocking across OSDs
I/O Epochs demark globally consistent snapshots • Guarantee all updates in one epoch are atomic • Recovery == roll back to last globally persistent epoch
– Roll forward using client replay logs for transparent fault handling • Cull old epochs when next epoch persistent on all OSDs
Time
Upd
ates
I/O Epochs
INTEL High Performance Data Division 11
Exascale I/O Architecture
Compute Nodes
I/O Nodes
Burst buffer NVRAM @ 200 TB/s
Disk Metadata NVRAM
Storage Servers
Site Storage Network @20 TB/s
Exascale Machine Shared Storage
Exascale Fabric
INTEL High Performance Data Division 12
Project Goals
Make storage a tool of the Scientist • Manage the explosive growth
and complexity of application data and metadata at Exascale
• Support complex / flexible analysis to enable scientists to engage with their datasets
• Provide storage performance and capacity required for Exascale science
Overcome today’s filesystem scaling limits • Share nothing • Move compute to data or data to compute as appropriate
Provide unprecedented fault tolerance • Design ground-up to handle failure as the norm rather than the exception • Guarantee data and application metadata consistency and integrity
INTEL High Performance Data Division 13
Exascale Filesystem
Integrated I/O Stack • Epoch transaction model • Non-blocking scalable object I/O
HDF5 • High level application object I/O model • I/O forwarding
I/O Dispatcher • Burst Buffer management • Impedance match application I/O performance to
storage system capabilities
DAOS • Conventional namespace for administration, security & accounting • DAOS container files for transactional, scalable, object I/O
/projects
/Legacy /HPC /BigData
Simulation data OODB metadata
data data data data data data data data data data data data data data data
OODB metadata OODB metadata OODB metadata OODB metadata
Posix striped file
a b c a b c a b c a
MapReduce data
data data data
data data data
data data data
Blocksequence
INTEL High Performance Data Division 14
Hadoop on Lustre
14
70%
• Hadoop on Lustre: extend Hadoop analytics to HPC environments o Exploit performance and scalability of shared storage o Scale storage and compute nodes separately
• Initial work demonstrates Lustre performance advantage • In plan to enable/optimize Hadoop analytics stack on Lustre in 2013 • Early results indicate existing HPC sites do well changing HDFS->Lustre
• No need to put disk back in the compute nodes • Scale compute and I/O separately to balance work
InfiniBand Interconnect
Hadoop Cluster/Compute Nodes
Lustre Storage
INTEL High Performance Data Division 15
Intel @ intersection of forces behind big data
Enabling exascale computing on massive data sets
Helping enterprises build open interoperable clouds
Commi_ed and contribu`ng code, process, and capital
HPC Cloud Open Source
Intel® TrueScale Infiniband
* Other names and brands may be claimed as the property of others.
INTEL High Performance Data Division 18
Today: A Vibrant Open Solution Ecosystem
Advocacy Storage Compute Integra=on Development
INTEL High Performance Data Division 19
Another view of the code effort
LLNL 23
ORNL 5 TACC 2
Whamcloud 250
Xyratex 5 Bull 1 CEA 3 Cray 5
DDN 3 LLNL 10 NICS 1
ORNL 15
TACC 3
Whamcloud 258
Xyratex 19
Bull 4 CEA 3 Cray 4 DDN 1
EMC 38
Fujitsu 1
Intel 435
LLNL 19 ORNL 17
TACC 5
Ultrascale 2
UVT 1 Xyratex 28