Grid Interoperability Shootout

GridPP and NGSUK e-Science All Hands Meeting,

Nottingham 2007J Jensen, G Stewart, M Viljoen, D

Wallom, S Young(contact j.dot.jensen.at.rl.dot.ac.dot.uk)

Two National Grids

• GridPP – UK Grid for particle physics– And others

• NGS – National Grid Service– Everybody

• Different audience, middleware, aims…

• Interoperability is beneficial (isn’t it?)

Differences

PhysicsBuilt for physics UCs

SL4 (SLC4)gLiteSRMssh

Non-physicsBuilt for non-p UCsVarious OSGlobus, Condor,…SRBGSISSH

Differences

Command lineSupported by

specialised groups

V. big data volumesNo need for secrecy

Own or free sw.

Portals & cmd lineSupported by a

single (loose) group

Not so big data vol.Some confidentialLicensed software

Differences – dteam vs. ETF

“deployment team”

Deployment & ∫Ops and experts

Meeting “user” reps.(large expts or UB rep).

“Engineering Task Force”

EvaluationsOps and supportNo users?

The UI

• Site UIs• “User installable

UI”• OS dependent

• Portals– Specialised portals– Workflow

• Or use GSISSH• OS independent

Common stuff

• The CA• MyProxy• VOMS (sort of)• BDII & GLUE

schema

• The big VO, small VO problem

Authentication

• Both using e-Science CA– NGS has made further progress with login– Training CAs (low assurance certificates)– Celebrated GSISSH Terminal

• Portable, using certificates to log in

– SSO integration, via site MyProxy– Shibboleth (prototype portals)

RB and LB

• NGS “recently” started using RB– Submit to at least core sites– Depends on information system

• Also tested GridWay

RBgLite RB

You probably can’t read this (how to lie with statistics)

Computing Element

• GridPP: LCG CE gLite CE• gLite CE

– Submits jobs to Torque, LSF, SGE, …– BLAHP support (parameter passing to

batch)

MPI

• NGS providing MPI– Needed, jobs are less embarrassingly parallel– Took some effort– NGS benefited from (and worked with)

GridIreland– Select MPI jobs via globus-job-submit– Still need support in gLite RB

Monitoring

NGS

TG

GridPP

LCG

gstat, SAM

MonAMI

INCA, GITS

Storage Elements

• GridPP using DPM, dCache, CASTOR– DPM for “smaller” sites (up to 200 TB…?)– dCache for larger or more complex infra.– DPM has been evaluated by ETF– Recommended (but like many parts of

gLite not very portable (VOMS dep.))

Storage – SRM and SRB

SE SRB

globus-url-copy

gridftp (Argonne)

Storage – SRM and SRB

SE SRB

SRM

gLite FTS

gridftp

Storage – SRM and SRB

SE SRB

SRM

gLite FTS

gridftp

Pretend SRB is a “Classic SE”(GridFTP server with information interface)

Still supported via FTS

Transfer files instead of developinginterfaces!

Interoperability with no add’l effort !!

Storage – SRM and SRB?

SE SRB

SRM

lcg-*

info

Storage – SRM and SRB?

SRM SRB

SRM SRM2.2

This time an ASGC (Taiwan) projectAiming for “a demo” at SC 2007

Interfacing once again brought up

The big(ger) picture

NGS

OSG TG

GridPP

LCG

OMIIGrid

Ireland

AstroGrid

EGI?Other

NationalGrids

The big(ger) picture

NGS

OSG TG

GridPP

LCG

OMIIGrid

Ireland

AstroGrid

EGI?Other

NationalGrids

The big(ger) picture

NGS

OSG TG

GridPP

LCG

OMIIGrid

Ireland

AstroGrid

EGI?Other

NationalGrids

Another big(ger) picture

gin-data

gin-jobs

gin-auth

gin-info

OGF

Opportunities

• “Need for capability computing”– GridPP has high priority for ops

• Data catalogues, file transfers• Documentation

• Portal work• GSISSH• “Research of the month”• ETF (or ETF’s experience)• Documentation, Training• User group meeting

• How to make grids interoperate• Standards• Strong Expertise in UK(I)• Influence external projects

(priorities)

• Externalities– EGEE– OMII UK/Europe– EGI and suchlike

• Standards– OGF– W3C– OASIS– Specialised industry

or academic stds (e.g. archives)

National

Disciplinary Technology

Boundaries

National

Disciplinary Technology

Broken by GridPP (LCG)

and NGS (TeraGrid)

Boundaries

TRUST

National

Disciplinary Technology

Broken within Grids(to some extent)

Boundaries

National

Disciplinary Technology

Focus for interoperability

Boundaries

RecommendationsJob subm. Job subm.

Mgmt Mgmt

Storage Storage

Establish links when it makes sense – stronger insome areas than others

RB

monitoringL&B

Recommendations

CE

Job subm.

Catalogues

Infosys

Note differences in software stack – link when it makes sense

RB

datatransfer

Infosys

Storage Storage

Job subm.

monitoringL&B

Conclusions

• Opportunities for collaboration• Build on common areas

– Many individual areas– Strong expertise in UK in many fields

• Sometimes there are differences for a reason – permit site and grid differences