Embed Size (px)
Citation preview
http://www.ngs.ac.ukhttp://www.grid-support.ac.uk
http://www.eu-egee.org/http://www.pparc.ac.uk/http://www.nesc.ac.uk/
The Storage Resource Brokerand the NGS
Slides from Wayne Schroeder, SDSC andPeter Berrisford, RAL
Induction to Grid Computing and the NGS 3
Acknowledgements
• This tutorial selects slides from several sources, specifically from talks given by Wayne Schroeder (SDSC) and Peter Berrisford (RAL)
Induction to Grid Computing and the NGS 4
Goal
• Introduce use of the SRB for distributed file management on the NGS– This is the focus of the practical that follows
• NOTE:– Wayne Schroeder: “SRB does so much, people tend to
learn subsets and are often unaware of useful features”
– So explore further!• http://www.sdsc.edu/srb/
• For a full SRB tutorial, see: http://www.niees.ac.uk/events/srb
Induction to Grid Computing and the NGS 8
SRB Projects• Digital Libraries
– UCB, Umich, UCSB, Stanford,CDL– NSF NSDL - UCAR / DLESE
• NASA Information Power Grid• Astronomy
– National Virtual Observatory – 2MASS Project (2 Micron All Sky Survey)
• Particle Physics – Particle Physics Data Grid (DOE)– GriPhyN – SLAC Synchrotron Data Repository
• Medicine– Digital Embryo (NLM)
• Earth Systems Sciences– ESIPS– LTER
• Persistent Archives– NARA– LOC
• Neuro Science & Molecular Science– TeleScience/NCMIR, BIRN– SLAC, AfCS, …
Over 90 Tera Bytes in 16 million files
Induction to Grid Computing and the NGS 10
What is SRB?
• Storage Resource Broker (SRB) is a software product developed by the San Diego Supercomputing Centre (SDSC).
• Allows users to access files and database objects across a distributed environment.
• Actual physical location and way the data is stored is abstracted from the user
• Allows the user to add user defined metadata describing the scientific content of the information
Induction to Grid Computing and the NGS 11
How SRB Works
MCATDatabase
MCATServer
SRB AServer
SRB BServer
SRBClient
a
b
c d
e
f
g
• 4 major components:
– The Metadata Catalogue (MCAT)
– The MCAT-EnabledSRB Server
– The SRB Storage Server
– The SRB Client
Induction to Grid Computing and the NGS 12
SRB Client Tools
• Provide a user interface to send requests to the SRB server.
• 4 main interfaces: – Command line (S-Commands)
– MS Windows (InQ)
– Web based (MySRB).
– Java (JARGON)
• Web Services (MATRIX)
Induction to Grid Computing and the NGS 13
Planned Deployment on NGS
Online Replication
SRB Server
SRB server @ RAL
Resource Driver
MCAT
SRB server @ Oxford
Resource Driver
Database Servers @ RAL
User
DiskFarm
Disk Farm
MCAT Server @ RAL
DB1
DB n
Database Servers @ Manchester
DB1
DB n
SRB Server
MCAT
MCAT Server @ Manchester
Failover link
DiskFarm
SRB server @ HPCX
Resource Driver
DiskFarm
SRB server @ Leeds
Resource Driver
DiskFarm
Induction to Grid Computing and the NGS 14
Summary
• SRB provides NGS users with– a virtual filesystem
– Accessible from all core nodes and from the “UI” / desktop
– (will provide) redundancy – mirrored catalogue server
– Replica files
– Support for application metadata associated with files
Induction to Grid Computing and the NGS 15
Tutorial Overview
• Use of the Scommands– Commands for unix based access to srb
– Strong analogy to unix file commands
• Accessing files from multiple (two) sites
Induction to Grid Computing and the NGS 16
Getting Started
• Launch two “putty” connections to pub-234– One for commands run on pub-234
– One for connecting to grid-data.rl.ac.uk and running commands from there
• Open browser and follow link from http://homepages.nesc.ac.uk/~gcw/NGS/
• Follow the instructions from there.• Your SRB name is the same as your account on
pub-234
