Embed Size (px)
DESCRIPTION
Fast partial access to objects from very large files in the SDSC Storage Resource Broker (SRB5) can be extremely challenging, even when those objects are small. The HDF-SRB project integrates the SRB and NCSA Hierarchical Data Format (HDF5), to create an access mechanism within the SRB that is can be orders of magnitude more efficient than current methods for accessing object-based file formats. The project provides interactive and efficient access to datasets or subsets of datasets in large files without bringing entire files into local machines. A new set of data structures and APIs have been implemented to the SRB support such object-level data access. A working prototype of the HDF5-SRB data system has been developed and tested. The SRB support is implemented in HDFView as a client application.
Citation preview
December 1, 2005 HDF & HDF-EOS Workshop IX
Integrating HDF5 with SRBObject-level Access to Remote Files
Peter Cao, NCSADecember 1, 2005
Sponsored by NLADR, NFS PACI Project in Support of NCSA-SDSC Collaboration
December 1, 2005 HDF & HDF-EOS Workshop IX 2/22
Outline
Introduction to the HDF-SRB project
The HDF-SRB model
SRB Support in HDFView
December 1, 2005 HDF & HDF-EOS Workshop IX 3/22
Overview of SRBWhat is
SRB
The SDSC Storage Resource Broker (SRB) is client-server middleware that provides a uniform interface for connecting to distributed data storage in multiple types of storage resources.
December 1, 2005 HDF & HDF-EOS Workshop IX 4/22
Overview of SRB
Architecture
SRB Client
SRB Server
MCAT
HPSS UnitreeDB2 ObjStore HDF5 FTP
Distributed Storage Resources: database system, archival storage system, file system, ftp
December 1, 2005 HDF & HDF-EOS Workshop IX 5/22
Project DescriptionMotivation
SRB HDF5
Indexing and searchingDistributed data systemAccess control
Large and diverse dataHigh performance accessInteractive and subsetting
High performance distributed data system
December 1, 2005 HDF & HDF-EOS Workshop IX 6/22
Remote Data Access on SRBMethods
Normal ways to access SRB:Get the whole file: large files (100TB SCEC)Use POSIX low level calls: low performance
New way: Implement proxy operations to access objects or parts of objects in one request
December 1, 2005 HDF & HDF-EOS Workshop IX 7/22
Project DescriptionGoals
Working prototype of client/server system forobject-level access to HDF5 stored in the SRB
Use SRB as middleware to transfer data between the server and client Use Object-level access for interactive and efficient access to part of the file
December 1, 2005 HDF & HDF-EOS Workshop IX 8/22
Normal SRB File AccessArchitecture
SRB Server
HDF5HDF5
MCAT
HDF5 File(whole file or a
sequence of bytes)
clientclient
December 1, 2005 HDF & HDF-EOS Workshop IX 9/22
Object-level File AccessArchitecture
SRB Server
HDF5HDF5
MCAT
HDF5 LibraryHDF5 Library
HDF5-SRB Module(pack/unpack messages)
HDF5 Object(File, Group, Dataset,
Subset, Attribute)HDF5 Object(File, Group, Dataset,
Subset, Attribute)
HDF5-SRB Module(pack/unpack messages)
clientclient serverserver
December 1, 2005 HDF & HDF-EOS Workshop IX 10/22
Examples of File Access
I need to see the eye of Hurricane Bob!
HDF5HDF5
December 1, 2005 HDF & HDF-EOS Workshop IX 11/22
Examples of File AccessWhole file transfer
HDF5HDF5
Get the file
December 1, 2005 HDF & HDF-EOS Workshop IX 12/22
Examples of File AccessWhole file
transfer
HDF5HDF5
Transfer large image – slow!
HDF5HDF5HDF5HDF5HDF5HDF5
December 1, 2005 HDF & HDF-EOS Workshop IX 13/22
Examples of File AccessSRB POSIX
API
HDF5HDF5
I need to see the eye of Hurricane Bob!
December 1, 2005 HDF & HDF-EOS Workshop IX 14/22
Examples of File AccessSRB POSIX
API
HDF5HDF5
image foundimage open
open image
find imagefile’s open
Open file
Many small messages – slow and complex!
December 1, 2005 HDF & HDF-EOS Workshop IX 15/22
Examples of File AccessObject level
HDF5HDF5
Get me the eye of hurricane Bob
December 1, 2005 HDF & HDF-EOS Workshop IX 16/22
Examples of File AccessObject level
HDF5HDF5
Get me the eye of hurricane Bob
One request & small image – fast & simple!
December 1, 2005 HDF & HDF-EOS Workshop IX 17/22
HDF5-SRB Model New objects/APIs
A new set data objectsH5File, H5Group, H5Dataset, H5Datatype, etcEncapsulated client requests and server results
Enhanced SRB APIsPack/Unpack routines (exchange data between byte stream and structure) to handle complicated struct – string, pointers, pointers to arrays, arrays of pointers, etcNew srbGenProxyFunct (general Proxy Function) handles other types of request besides HDF5
December 1, 2005 HDF & HDF-EOS Workshop IX 18/22
HDF5-SRB Model Data Flow
Client APIsrbObjRequest(void *obj, int objID)
Server APIsrbObjProcess(void *obj, int objID)
srbGenProxyFunct
1. packMsg()
2. u
npac
kMsg
()
HDF5 Library
HDF5 file
3. H5Obj::op()
4. Access file
5. H5Object
6. p
ackM
sg()
7. unpackMsg()
SRB Server
December 1, 2005 HDF & HDF-EOS Workshop IX 19/22
Runing Server/Client
A SRB server that supports HDF5HDF5 library and other external libraries (SZIP, ZLIB)A SRB version 3.4 or later from http://www.sdsc.edu/srb/Follow instruction on how to run SRB server from UG packed with SRB source release or online at http://hdf.ncsa.uiuc.edu/hdf-srb-html/HDF-SRB-UG.html
Any client application that implements HDF5-SRB Objects
No HDF5 library is required on the clientExample client application: HDFView 2.3 or above
December 1, 2005 HDF & HDF-EOS Workshop IX 20/22
Short DemoHDFView
Support Windows and LinuxMay support SGI, AIX if there is new funding
December 1, 2005 HDF & HDF-EOS Workshop IX 21/22
Future Work?
Writing capabilities: the current HDF-SRB implementation only supports read-only operations. We propose to add write functionality so that users will be able to create new files and data objects, and to modify data content and attributes.Better support for complex datatypes such as compound datatypeand variable length datatypeSupport for HDF5 indexing and ingesting HDF5 metadata, which will enable users to access HDF5 objects directly through MCATSupport for files across different servers (or SRB federation)More features for HDFView: compared to its local file-handling capabilities, the current version of HDFView has very limited features for remote file access on an SRB server.
December 1, 2005 HDF & HDF-EOS Workshop IX 22/22
Question / Comments?
