e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

1

Distributed Storage management using SRB on UK National Grid Service

Manandhar A, Haines K, Brown G.D, Kleese K, Berrisford P.

CCLRC

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

2

Overview

• SDSC SRB overview• SRB Features of interest• Issues focussed for this deployment• Deployment Structure• Expected usage scenario• Future development• Questions

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

3

National Grid Service

The National Grid Service is a new initiative to provide computing resources to the e-Science community

Services• Computational service• File management service• Database service

For providing file management services on the National Grid Service, Storage Resource Broker is being used.

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

4

SRB for Data Grid

The SDSC Storage Resource Broker is a middleware that virtualises storage space by providing a unified view to multiple heterogeneous storage Resources over the network.

It is a software that sits in between users and resources and provides a storage service by managing users, file locations, storage resources and metadata information and enables collaboration in an Grid environment

Data Grid

Disk space @ loc y

Disk space @ loc x

MSS Space @ loc z

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

5

SRB Logical Structure

DB

SRBView

User/Process

User/Process

User/Process

Disk Farm

Disk Farm

Tape Farm

RDBMS

The logical directory structure and the physical

file locations are maintained in a Database.

Also other persistent information such as User accounts, Participating resources and file meta

data are stored in it.

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

6

Features of interest

• Logical abstraction to multiple heterogeneous storage resources.

• Ability to inter-operate with Grid applications with GSI authentication mechanisms

• Fine grained access control mechanism for file access

• Device driver interface to disk arrays, and to mass storage systems such as Atlas and HPSS.

• Bulk file and parallel transfers for performance enhancements while transferring multiple small files

• Lightweight clients for access to data from Unix, Windows, Macintosh and browser based interface.

• C, Java, Python and Webservices API for closer interaction with programs

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

7

Issues focussed for this deployment

• ReliabilityAs data is a very important part of a project, reliability of the system has been

given a very high priority. The system is made reliable by having online replication at many different layers.

• Inter-operabilityEasy inter-operability with the computational service and other Grid

applications by the use of common authentication mechanism and enabling easy data sharing between projects.

• Multi project hostingHosting of multiple domains and enabling access control of certain resources

to only specific projects in the SRB network for supporting multi project hosting.

• Future ScalabilityEnabling SRB federation for data interaction with other SRB installations and

for scaling future deployment

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

8

Deployment Structure

Online Replication

SRB Server

SRB server @ RAL

Resource Driver

MCAT

SRB server @ Oxford

Resource Driver

Database Servers @ RAL

User

DiskFarm

Disk Farm

MCAT Server @ RAL

DB1

DB n

Database Servers @ Manchester

DB1

DB n

SRB Server

MCAT

MCAT Server @ Manchester

Failover link

DiskFarm

SRB server @ HPCX

Resource Driver

DiskFarm

SRB server @ Leeds

Resource Driver

DiskFarm

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

9

MCAT Enabled ServerRAL

RAL

Oracle Replication

Replication Structure

Oracle Client

…..

Oracle 9i RAC (Nodes 1.. 8)

MCAT Enabled Server

Manchester

RAL

Oracle Client

…..

Oracle 9i RAC (Nodes 1.. 8)

StorageServer

User

Fail over link

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

10

Projected use of NGS SRB (1)• In an SRB network, the MCAT is the most time consuming and expensive to

maintain. • Projects may utilize the MCAT service provided by NGS in maintaining their

distributed file service and add their existing storage resources to the NGS SRB.• In doing so they can also collaborate with other participating projects easily.

SRB server @ Project

Resource Driver

DiskFarm

SRB server @ Project

Resource Driver

DiskFarm

NGS SRB

Disk space

Disk space

MCAT

ComputationalFarm

(project)

NGSComputational

Farm

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

11

Projected use of NGS SRB (2)

SRB Space(Used as a

Project file system)

Use of SRB as a project file system and with Condor/Globus – e-Minerals project

Condor Client Tools/ DAG Clients

Globus 2.4

Compute Cluster @Oxford

User @ Cambridge

User @ UCL

Condor G interface

CondorPool @

Cambridge

Code Repository Linux Solaris

Data Repository

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

12

Projected use of NGS SRB (3)

• Some projects such as the Integrative Biology project have the requirement to with other SRB networks. It is expected these projects would use the federation feature of SRB for inter-organizational collaboration.

• Also as projects progress they may want to maintain their own MCAT and create federation links with the NGS SRB for collaborating with other projects in there or use other services provided by NGS.

DB

SRB

DB

SRB

DB

SRB

Trust Relation

NGS Zone Zone BZone A

Trust Relation

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

13

Future additions

• More use of Federation concepts for • Scaling SRB installation (with other NGS sites or new projects)

• Performance improvements

• Look into different replication usage for performance and reliability improvements

• Improve SRB administration by possibly moving to a decentralized management from currently centralized SRB management.

e-Science AHM 200431st Aug – 3rd Sept 2004Nottingham

14

Questions ..