March 18, 2004 Eric Hjort

HRM use in STARUS Atlas/RHIC Grid Workshop

March 18, 2004

Eric Hjort, LBNL(STAR/PPDG Collaborations)

-SRM’s in general-HRM’s in STAR-CLI examples

March 18, 2004 Eric Hjort

What are SRMs?• Grid middleware developed by the Scientific

Data Management (SDM) Resource Group, LBNL• Examples of SRMs (Storage Resource Managers) Disk Resource Managers (DRMs) Tape Resource Managers (TRMs) Hierarchical Resource Managers (HRM = TRM+DRM)

• API and CLI are supported• For STAR: HRMs make the complicated 3-step

transfer: HPSS – disk – disk - HPSS look like a simple copy. Details of cache

management, request queuing, fault recovery, etc. are hidden.

March 18, 2004 Eric Hjort

March 18, 2004 Eric Hjort

Main advantages of using Main advantages of using SRMsSRMs

(courtesy SDM group)(courtesy SDM group)

• Same uniform interface to all types of storage systems– Disk Storage System– Mass Storage System

• For Mass Storage Systems it provides:– Queuing and pre-staging

• Queued multi-file “get” requests (avoid flooding MSS)• Pre-staging of files (concurrent with transfer)• Queued archiving of “put” requests (avoid flooding MSS)

– Robustness and efficiency• Recover in case of transient failures from MSS• Reorders pre-staging requests to minimize tape mounts

– Recovery from failed GridFTP transfers• Re-issues requests in case of failure

March 18, 2004 Eric Hjort

Star File Replication: Overview

BNL LBL

BNLFile Catalog

BNL FCMirror

LBLFC

LBL FCMirror

Files Files

mysql

mysql

HRM

metadata data

March 18, 2004 Eric Hjort

BNL LBL

BNLFile Catalog

BNL FCMirror

LBLFC

LBL FCMirror

Files Files

mysql

mysql

HRM

RRS

Registration Replica Service

March 18, 2004 Eric Hjort

LBNL-PDSF BNL

Client

HRM HRM

1. request toreplicate

2. file request

3. Stage the file

4. notify the caller

5. gridftp from BNLto PDSF

6. release the file

8. migrate the fileto HPSS

9. notify the client(file in HPSS)

7. notify the client(file in HRM)

Architecture(from Alex Sim)

Not shown - step 10: notify RRS server of completed transfer

March 18, 2004 Eric Hjort

Request Submitted

Space Reserved

PFTP->HPSS done

PFTP->HPSS start

HPSS request

GridFTP end

GridFTP start

File Released

Time (seconds)

- As seen from the HRM at LBL- (525 MB/file)(56 files)/7000 sec = 4.2 MB/s

Transfer Schematic

March 18, 2004 Eric Hjort

CLI example 1

srm-copy.linux -conf hrm.rc -sd "srm://stargrid03.rcf.bnl.gov:XXXX /home/starsink/raw/daq/2004/011/st_physics_adc_5011048_raw_1* ?remoteobj=HRMServerBNL&msshost=hpss.rcf.bnl.gov&mssport=XXXX" -td "srm://garchive.nersc.gov/nersc/projects/starofl/raw/daq/2004/011" -at PLAIN -et GSI -al starpftp -ap “fakePWD"

Copy the entire contents of a directory, subject to a wild-card search:

- Recursive and conditional transfers also supported

March 18, 2004 Eric Hjort

CLI example 2

srm-copy.linux -d -f daq.get -c /auto/u/hjort/hrm2/hrm.rc -l daq.log -w -at PLAIN -et GSI -al starpftp -ap “fakePWD"

daq.get: (sourceURL, size, targetURL)

srm://stargrid03.rcf.bnl.gov:XXXX /home/starsink/raw/daq/2004/015/st_physics_adc_5015002_raw_1030001.daq ?remoteobj=HRMServerBNL&msshost=hpss.rcf.bnl.gov&mssport=XXXX 525349883srm://garchive.nersc.gov/nersc/projects/starofl /raw/daq/2004/015/st_physics_raw_1030001.daq

Supply SRM with a file list:

- File lists created by comparing BNL mirror catalog and HPSS - Useful for getting missing files, completing crashed transfers, etc.

March 18, 2004 Eric Hjort

CLI example 3

srm-copy.linux -d -f disk_ppLong-1FullFieldP03ih.get -c /auto/u/hjort/hrm2/hrm.rc -l disk_ppLong-1FullFieldP03ih.log

gsiftp://stargrid03.rcf.bnl.gov/star/data16 /reco/ppLong-1/FullField/P03ih/2003/150 /st_physics_4150010_raw_0020078.MuDst.root 23315827 srm://garchive.nersc.gov/nersc/projects/starofl /reco/ppLong-1/FullField/P03ih/2003/150 /st_physics_4150010_raw_0020078.MuDst.root

Advantage: NFS access typically faster than HPSS

Get files off of NFS disks instead of HPSS:

disk_ppLong-1FullFieldP03ih.get:

March 18, 2004 Eric Hjort

Final Notes

- For more info, see the SDM site: http://sdm.lbl.gov/ and in particular the SRM pages: http://sdm.lbl.gov/projectindividual.php?ProjectID=SRM

- SRMs have served STAR well for ~2 years – transferred many 10’s of TB. SRMs allow improved efficiency, robustness and automation.

- SRM V2 due out later this year (web services)