Scientific Data Management Group LBNL

1

Scientific Data Management Group LBNL

SRM related demos SC 2002

• Demos Demos

• Robust File Replication of Massive Datasets on the Grid

• GridFTP-HPSS access Provided through HRM

• HRM enables NCAR’s MSS to be accessed on the Grid

• Uniform Grid Access to Different Mass Storage Systems

• ProjectsProjects

• Earth Science Grid: Use of HRMs in ESG Demo, new HRM-NCAR included

• Particle Physics Data Grid: File replication for STAR project – Demo

2

Junmin Gu, Vijaya Natarayan, Alex Sim, Arie ShoshaniJunmin Gu, Vijaya Natarayan, Alex Sim, Arie ShoshaniLBNLLBNL

http://sdm.lbl.gov/srmhttp://sdm.lbl.gov/srm

DemoDemo: Robust File Replication of : Robust File Replication of Massive Datasets on the GridMassive Datasets on the Grid

This demo illustrates:

Replicating Thousands of Files Robustly between Two remote HPSS installations

• Using a single command for a request to replicate all files

• Using an intermediate middleware storage management (called Hierarchical Storage Manager - HRM)

• Using a client HRM command-line module

• Without any modifications to the MSS

• Synchronizing pre-staging, file transfer, and archiving

• Robustness: recovers from MSS and network transient failures

3

SRMs in Action : PPDGSRMs in Action : PPDG

DiskCache

DiskCache

HRM-COPY(thousands of files)

HRM-GET (one file at a time)

HRM-ClientCommand-line Interface

HRM(performs writes)

HRM(performs reads)

LBNL BNL

GridFTP GET (pull mode)

Anywhere

stage filesarchive files

Network transfer

4

Web-Based File Monitoring ToolWeb-Based File Monitoring Tool

Shows:-Files already transferred- Files during transfer- Files to be transferred

Also shows foreach file:-Source URL-Target URL-Transfer rate

5

Recent Measurements Recent Measurements of large multi-file replicationof large multi-file replication

Shows that the network is the bottleneck

6

Alex Romosan,Alex Romosan, Alex Sim, Junmin Gu,Alex Sim, Junmin Gu, Arie ShoshaniArie Shoshani


DemoDemo: GridFTP-HPSS: GridFTP-HPSSAccess Provided through HRMAccess Provided through HRM

This demo illustrates the capability of:

Providing GridFTP access to Mass Storage Systems (MSSs)

• Using an intermediate middleware storage management (called Hierarchical Storage Manager - HRM)

• Without any modifications to the MSS

• Managing queues of multiple requests to the MSS

• Minimizing tape mounts

• Recovers from MSS transient failures

7

Where does this service belongWhere does this service belongin the Grid architecture?in the Grid architecture?

ComputeSystems

Networks

OtherStorage

systems

Hierarchical

ResourceManager

ComputeResource

Management

General DataDiscoveryServices

CommunityAuthorization

Services

Application-Specific Data

Discovery Services

StorageManagement(Brokering)

ComputeScheduling(Brokering)

Data Filtering orTransformation

Services

DatabaseManagement

Services

RequestInterpretationand Planning

Services

File TransferService(GridFTP)

DataTransportServices

Monitoring/AuditingServices

Workflow orRequest

ManagementServices

Consistency Services(e.g., Update Subscription,Versioning, Master Copies)

DataFederationServices

RE

SO

UR

CE

:S

HA

RIN

G S

ING

LER

ES

OU

RC

ES

CO

LLE

CT

I VE

1:

GE

NE

RA

LS

ER

VIC

ES

FO

RC

OO

RD

INA

TIN

GM

ULT

I PLE

RE

SO

UR

CE

S

CO

LLE

CT

IVE

2:

SE

RV

ICE

SS

PE

CIF

IC T

OA

PP

LIC

AT

ION

DO

MA

IN O

RV

IRTU

AL

OR

G.

ResourceMonitoring/

Auditing

FA

BR

ICC

ON

NE

CTI

VIT

Y

CommunicationProtocols (e.g.,TCP/IP stack)

Authentication andAuthorization

Protocols (e.g., GSI)


Services

CO

LL

EC

TI V

E

This figure based on theGrid Architecture paper by Globus Team

Mass StorageSystem(HPSS)

8

HRMs and GridFTPHRMs and GridFTP

HRM

GridFTP

SRM-API

GridFTP-API

Client

HRM

GridFTP move

SRM-API

GridFTP-API

Client

Using HRM protocol New: GridFTP-HPSSthrough HRM

GridFTP entry

9

GridFTP-HRM-LayerGridFTP-HRM-Layerimplementation detailimplementation detail

HRM

GridFTP-API

Client

GridFTP entry

GridFTP move

GridFTP exit

FTP-HRMLayer

Sharedmemory Corba

1a 1b

2a 2b

3a 3b

1a: stor/retv1b: hrm_get/hrm_put

2b: call_back2a: unblock semaphore

3a: success_code3b: hrm_release

10

Alex Romosan,Alex Romosan, Junmin Gu,Junmin Gu, Alex Sim, Arie ShoshaniAlex Sim, Arie ShoshaniLBNLLBNL


DemoDemo: HRM enables NCAR’s MSS to : HRM enables NCAR’s MSS to be accessed on the Gridbe accessed on the Grid

This demo illustrates:

The application of Storage Resource Management (SRM) technology to make NCAR’s MSS Grid-enabled

•NCAR’s MSS is a unique system developed in house

• Making HRM-HPSS grid-enabled is a big task

• HRM is a specialized SRM designed for Mass Storage Systems (MSS). HRM-HPSS already exists

• Adapting HRM-HPSS to work with HRM-HPSS is a small task

• Demo illustrates success and ease of this approach

11

How Was It Done?How Was It Done?only MSS-specific module modifiedonly MSS-specific module modified

Disk ResourceManager (DRM)

Tape ResourceManager (TRM)

HPSS-specificAccess Module

DiskCache

HRM-HPSS

Disk ResourceManager (DRM)

Tape ResourceManager (TRM)

NCAR-specificAccess Module

DiskCache

HRM-HPSS

Specialize for NCAR-MSS

12

Demo details: File Replication from NCAR MSS Demo details: File Replication from NCAR MSS to NERSC HPSS controlled at NCAR to NERSC HPSS controlled at NCAR

LBNL NCAR

HRM HRM

1. Request to Get

2. file request

3. Stage the file

4. notifythe caller

5. gridftp from NCARto LBNL

6. release the file

7. migrate the fileto HPSS

HRM ClientCommand

LineInterface

Client

DISPLAY:* Files Requested

0. HRM-COPY

HRM ClientCommand

LineInterface

HRM-STATUS

DISPLAY:* Files Status

Files Status

HRM FMTon the web

browser

MSS

13

SRMs in Action : ESGSRMs in Action : ESG

Tomcat servlet engine

Tomcat servlet engine

MCSMetadata Cataloguing Services

MCSMetadata Cataloguing Services

RLSReplica Location Services

RLSReplica Location Services

SOAP

RMI

MyProxyserver

MyProxyserver

MCS client

RLS client

MyProxy client

GRAMgatekeeper

GRAMgatekeeper

CASCommunity Authorization Services

CASCommunity Authorization Services

CAS client

disk MSSMass Storage System

HPSSHigh PerformanceStorage System

disk

HPSSHigh PerformanceStorage System

disk

disk

SRMStorage Resource

Management

SRMStorage Resource

Management

SRMStorage Resource

Management

SRMStorage Resource

Management

SRMStorage Resource

Management

SRMStorage Resource

Management

SRMStorage Resource

Management

SRMStorage Resource

Management

gridFTP

gridFTP

gridFTPserver

gridFTPserver

gridFTPserver

gridFTPserver

gridFTPserver

gridFTPserver

gridFTPserver

gridFTPserver

openDAPgserver

openDAPgserver

gridFTPStripedserver

gridFTPStripedserver

LBNL

LLNL

ISI

NCAR

ORNL

ANL

14

Contributors:Contributors:JLABJLAB: Bryan Hess, Andy Kowalski: Bryan Hess, Andy KowalskiFNALFNAL: Don Petravick, Timur Perelmutov, Rich Wellner: Don Petravick, Timur Perelmutov, Rich WellnerLBNLLBNL: Junmin Gu, Vijaya Natarayan,: Junmin Gu, Vijaya Natarayan, Ekow Otoo, Ekow Otoo,

Alex Romosan,Alex Romosan, Alex Sim, Arie Alex Sim, Arie ShoshaniShoshani

DemoDemo: Uniform Grid Access: Uniform Grid Accessto Different Mass Storage Systemsto Different Mass Storage Systems

This demo illustrates: Uniform Grid Access to:

three MSSs: HPSS, Enstore, JASMine

• Using the Storage Resource Manager (SRM) interface

• SRM modules were developed independently for the 3 MSSs

• Shows the power of standard SRM functionality

15

High Level View of Demo SetupHigh Level View of Demo Setup

SRM

SRM

SRM

Enstore

JASMine

Client(USER/APPLICATIONS)

16

Where do SRMs belongWhere do SRMs belongin the Grid architecture?in the Grid architecture?

ComputeSystems

Networks

OtherStorage

systems

StorageResourceManager

ComputeResource

Management

General DataDiscoveryServices

CommunityAuthorization

Services

Application-Specific Data

Discovery Services

StorageManagement(Brokering)

ComputeScheduling(Brokering)


Services

DatabaseManagement

Services

RequestInterpretationand Planning

Services

File TransferService(GridFTP)

DataTransportServices

Monitoring/AuditingServices

Workflow orRequest

ManagementServices

Consistency Services(e.g., Update Subscription,Versioning, Master Copies)

DataFederationServices

RE

SO

UR

CE

:S

HA

RIN

G S

ING

LER

ES

OU

RC

ES

CO

LLE

CT

I VE

1:

GE

NE

RA

LS

ER

VIC

ES

FO

RC

OO

RD

INA

TIN

GM

ULT

I PLE

RE

SO

UR

CE

S

CO

LLE

CT

IVE

2:

SE

RV

ICE

SS

PE

CIF

IC T

OA

PP

LIC

AT

ION

DO

MA

IN O

RV

IRTU

AL

OR

G.

ResourceMonitoring/

Auditing

FA

BR

ICC

ON

NE

CTI

VIT

Y

CommunicationProtocols (e.g.,TCP/IP stack)

Authentication andAuthorization

Protocols (e.g., GSI)


Services

CO

LL

EC

TI V

E

This figure based on theGrid Architecture paper by Globus Team

Mass StorageSystem(HPSS)

17

Main advantages of using SRMs Main advantages of using SRMs

• Same uniform interface to all types of storage systemsSame uniform interface to all types of storage systems• Disk Storage System• Mass Storage System

• For Mass Storage Systems it provides:For Mass Storage Systems it provides:• Queuing and pre-staging

• Queued multi-file “get” requests (avoid flooding MSS)• Pre-staging of files (concurrent with transfer)• Queued archiving of “put” requests (avoid flooding MSS)

• Robustness and efficiency• Recover in case of transient failures from MSS• Reorders pre-staging requests to minimize tape mounts

• Recovery from failed GridFTP transfers• Re-issues requests in case of failure

Documents

Scientific Data Management Group LBNL