INFSO-RI-508833 Enabling Grids for E-sciencE Comparison of LCG-2 and gLite Author E.Slabospitskaya Location IHEP

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

Comparison of LCG-2 and gLite

Author E.SlabospitskayaLocation IHEP


INFSO-RI-508833

Outline

• What are LCG-2 and gLite? Roles.• gLite Architecture review

– What is gLite? Provenance of gLite from Alien, Grid3 et al.

– Schema of gLite Services. The main subsystems of gLite. Comparison to LCG-2.

o Workload management.

o Data management

o Information System

o Security

o Accounting

o Alien

• Current status of gLite

• Mapping to physical machines


INFSO-RI-508833

Future EGEE middleware

Globus 2 based Web services based

gLite-2gLite-1LCG-2LCG-1

http://glite.web.cern.ch/gliteEDGVDT . . .

LCG . . .AliE

n

Architecture: https://edms.cern.ch/document/476451

Design: https://edms.cern.ch/document/487871/


INFSO-RI-508833

Provenance of gLite

The gLite middleware is a Service Oriented Grid middleware providing services for managing distributed computing and storage resources and the required security, auditing and information services.

AliEn is a lightweight Grid framework which is built around Open Source components using the Web Services model. It has been initially developed by the ALICE collaboration as a production environment for the simulation, reconstruction, and analysis of Physics data in a distributed way.

The gLite Middleware is presently using a number of services taken from the AliEn system, namely the File Catalogue and the Task Queue (part of the AliEn Core Services), the Package Manager (part of the AliEn Site Services) and the GAS or Grid Access Service.

Target server platform is Red Hat Linux 3.0 or any binary compatible distribution, such as Scientific Linux, and Windows.


INFSO-RI-508833

gLite and LCG-2

LCG-2focus on production, large-scale data handling

• The service for the 2004 data challenges

• Provides experience on operating and managing a global grid service

• Development programme driven by data challenge experience

Data handling Strengthening the

infrastructure Operation, VO

management

• Evolves to LCG-3 as components progressively replaced with new middleware

-- target is to minimise the discontinuities of migration to the new generation

• Aim for migration plan by end of year

gLitefocus on analysis

• Developed by EGEE project in collaboration with VDT (US)

• LHC applications and users closely involved in prototyping & development (ARDA project)

• Short development cycles

• Co-existence with LCG-2• Profit as far as possible from LCG-

2 infrastructure, experience Ease deployment – avoid separate

hardware

• As far as possible - completed components integrated in LCG-2

improved testing, easier displacement of LCG-2

LCG-2 (=EGEE-0)

prototyping

prototyping

product

2004

2005

LCG-3 EGEE-1

product

les robertson - cern-it-5

LCG


INFSO-RI-508833

gLite Services

Service decomposition based on ARDA blueprint


INFSO-RI-508833

GAS

Client

File Catalog

Metadata

catalogue

WMS

API*- for appl

GAS

The Grid Access Service provides access to different Grid Services and actsas an adapter exposes a flat interface to the collection of components


INFSO-RI-508833

Job Management

.•A CE interfaces the local resource management system (e.g. LSF, PBS) to the Grid middleware. The currently released CE will be enhanced with a CE Monitor that pulls jobs from the WMS.

•The Worker Nodes behind the local resource management system host all the necessary clients to interact with the Grid middleware from within a job.

•The Package Manager allows the dynamic installation of application software.


INFSO-RI-508833

WMS


INFSO-RI-508833

AliEn Task Queue and EDG/LCG RB

REQUEST

MONITOR

SUBMIT

L & B

CE

TASK TASK QUEUEQUEUE

WMS

GAS

User

PUSH

PULLSubmitjobs


INFSO-RI-508833

CE

• Works in push and pull mode

• Site policy enforcement

• Exploit new globus GK and CondorC (close interaction with globus and condor team)

CEA … Computing Element Acceptance

JC … Job Controller

MON … Monitoring

LRMS … Local Resource Management System


INFSO-RI-508833

Data Management

.

•Data is stored in an SRM based storage system.

•The gLite-I/O server allows posix-like access to files stored in the SRM

•The local catalogue (LC) keeps track of the LFN:GUID:SURL mapping of local files

The file transfer/file placement service is used for moving files.

3 main service groups that relate to data and file access are:

Storage Element,

Catalogue Services

Data Transfer Scheduling .

Closely related to the data services are the security-related services and the Package Manager.


INFSO-RI-508833

Catalogs

File Catalog

Metadata Catalog

LFN

Metadata

• File Catalog– Filesystem-like view on logical file names

– Keeps track of sites where data is stored

– Conflict resolution

• Replica Catalog– Keeps information at a site

• (Meta Data Catalog)– Attributes of files on the logical

level– Boundary between generic

middleware and application layer

Replica Catalog Site A

GUID SURL

SURL

LFN

Replica Catalog Site B

GUID SURL

SURL

LFN

GUIDSite ID

Site ID


INFSO-RI-508833

Storage Element Interfaces

• SRM interface– Management and control– SRM (with possible evolution)

• Posix-like File I/O– File Access– Open, read, write– Not real posix (like rfio) SRM interface

rfio dcap chirp aio

Castor dCache NeST Disk

POSIXAPIFile I/O

Control

User


INFSO-RI-508833

Information System

.The R-GMA (Relational Grid Monitoring Architecture) servlet accepts connection from clients (producers), i.e. the services publishing and user jobs publishing information, and forwards the information to the appropriate consumers.

R-GMA is composed of the following services:

a. R-GMA server

b. R-GMA client is a set of client API in C,C++,Java and Python for the access the information and monitoring functionality of the R-GMA system

c. R-GMA site-publisher (each site) is responsible for publishing site information to the R-GMA server.

d. R-GMA service tool -regularly scans config files and updates a current Service Status R-GMA Table. CLI exists for modify these files and query the service table.


INFSO-RI-508833

R-GMA

R-GMA is based on the Grid Monitoring Architecture (GMA) from the Grid Global Forum (GGF), which is a simple Consumer-Producer model that models the information infrastructure of a Grid as a set of consumers (that request information), producers (that provide information) and a central registry which mediates the communication between producers and consumers. R-GMA offers a global view of the information as if each Virtual Organization had one large relational database.

Producers contact the registry to announce their intention to publish data, and consumers contact the registry to identify producers, which can provide the data they require. The data itself passes directly from the producer to the consumer: it does not pass through the registry.

R-GMA adds a standard query language (a subset of SQL) to the GMA model,


INFSO-RI-508833

Mapping gLite Services onto physical machines


INFSO-RI-508833

gLite Services and components

--Accounting Service (DGAS)

- AliEn File Catalogue (global catalogue – GC)

- File & Replica Catalogue (local catalogue – LC)

- Metadata Catalogue

- Computing Element (CE)

- File Transfer and Placement Service

- gLite-I/O Server and Client

- Logging and Bookkeeping Server (LB)

- Package Manager

-

•R-GMA Servers, Client and Service Tools

•VOMS and VOMS administration tools

•Authorization, Authentication and Delegation Services

•Standard Worker node (WN)

•User Interface

•Workload Manager System (WMS)

•Grid Access Service (GAS)

•Job Provenance

The following high level services are part of gLite middleware

Documents

INFSO-RI-508833 Enabling Grids for E-sciencE Comparison of LCG-2 and gLite Author E.Slabospitskaya Location IHEP