P.Kunszt DGC5 4.9.2002 1

EU DataGrid Data Management Workpackage : WP2

Status and Plans

Peter Z KunsztIT/DB

http://cern.ch/grid-data-management/EU Datagrid Conference Piliscsaba/Hungary

P.Kunszt DGC5 4.9.2002 2

Outline

• WP2: Tasks and Constituency• Task status

– Replication– Metadata– Optimization– Security

• Plans• Issues

P.Kunszt DGC5 4.9.2002 3

Data Management Tasks

• Data TransferEfficient, secure and reliable transfer of data between

sites

• Data ReplicationReplicate data consistently across sites

• Data Access OptimizationOptimize data access using replication and remote open

• Data Access ControlAuthentication, ownership, access rights on data

• Metadata StorageGrid-wide persistent metadata store for all kinds of Grid information

P.Kunszt DGC5 4.9.2002 4

Current Constituency

• CERN Peter Kunszt, Heinz Stockinger, Leanne Guy, Diana Bosio, Akos Frohner, Wolfgang Hoschek, Kurt Stockinger

• INFN Flavia Donno, Andrea Domenici, Livio Salconi, Giuseppe Andronico, Federico Di Carlo, Marco Serra

• PPARC Gavin McCance, William Bell, David Cameron, Paul Millar

• Trento Cultural Institute Ruben Carvajal Schiaffino, Luciano Serafini, Floriano Zini

• U.Helsinki/CSC Mika Silander, Joni Hahkala, Ville Nenonen, Niklas Karlsson

• KDC Stockholm Olle Mulmo, Gian Luca Volpato

P.Kunszt DGC5 4.9.2002 5

Additional Resources, collaborators

• LCG:Erwin Laure, Itzhak Ben-Akiva, James Casey

• PPDG: Andy Hanushevsky• Condor: Alain Roy• NorduGrid: Aleksandr Konstantinov• Globus/ISI: Ann Chervenak, Bob Schwarzkopf

P.Kunszt DGC5 4.9.2002 6

Current Status

• Replication: edg-replica-catalog – deployed– GSI version available.– Graphical User Interface available.

• Replication: edg-replica-manager – deployed– Atomic replication operations.– This component should be used for data management

tests.• Replication: GDMP – deployed

– More complex subscription-based functionality.– Has been tested and is proven to work.– Can be ‘abused’ to provide the same functionality as edg-

replica-manager but it is overkill and much harder to do.• Metadata Storage: SQL Database Service Spitfire –

deployed– Servlets on HTTP(S) with XML (XSQL)– GSI enabled access + extensions

• Optimization: Replica Selection OptorSim – working

P.Kunszt DGC5 4.9.2002 7

Current Status (cont.)

• Replication: Replica Location Service – deployable– Joint Globus – EDG development– Unit and performance tests successful– Distributed Replica Catalog service– Graphical User Interface available.

• Security: edg-security – in testing– Security for Web Services

• Metadata Storage: SQL Database Service Spitfire – in testing– Web Service interface over SOAP– Security through edg-security and proper local authorization– Secure data browser through a web interface

• Replication: Replica Manager Reptor – in testing and development

• Optimization: Replica Selection Optor – in testing and development

• Replication: Replica Metadata Catalog Repmec – in testing and development

• Security: edg-voms – in development, joint effort with WP6, WP7

P.Kunszt DGC5 4.9.2002 8

WP2 Strategy for 1.2

• Integrate a single-instance RLS into TB1.2 (replacement for edg-replica-catalog)

• Integrate RLS into edg-replica-manager and GDMP• Focus users away from GDMP and towards edg-replica-

manager for simple data management use cases.• Increment stability and usability of edg-replica-manager• Configuration

– Simplify configuration and build system of deployed components

– Unify build for all components, adopt edg-standards• Testing

– Large effort on unit testing and functionality tests inside WP2 for all components.

– Close interaction with ITeam and the new testing group

P.Kunszt DGC5 4.9.2002 9

Towards Testbed 2

• Focus away from GDMP• Introduction of Reptor (‘refactorization of GDMP’)

– Initially with identical functionality– Incremental and controlled addition of requested

functionality– Use R-GMA for information gathering and publishing

• edg-replica-manager will stay as a command-line interface to Reptor, easy migration for users

• Security: work towards new security model with VOMS and local authorization.

• Documentation:– Guides– Examples, How-to– Tutorials

P.Kunszt DGC5 4.9.2002 10

Progress in WP2 Workshop

• Application of the conclusions of the Retreat towards the WP2 Strategy

• Reallocation of resources according to the strategy.

• Re-focus Development• Detailed workplan for the next incremental

releases.• Better understanding of the technical issues,

interoperation of components.

P.Kunszt DGC5 4.9.2002 11

Progress in this Conference

• Joint session with WP5– Agreement to continue with current solution in

GDMP, will be used in edg-replica-manager as well– WP5 will produce a prototype service by the end

of September. These services will be used by Reptor.

– Close collaboration to make MSS access for Atlas successful.

• Joint session with WP10– Control flow, Metadata requirements– Security requirements– Close collaboration on deploying a WP10

metadata catalog.

P.Kunszt DGC5 4.9.2002 12

Progress in this Conference

• Interaction with WP3– Agreement on usage of edg-security module

for R-GMA

• Joint session with WP1-4-5– Agreement on the Zamboni daemon for

worker node storage administration. Joint development of WP1-2-4

• Joint session with applications WP8-9-10– Detailed feedback on current status was very

useful. – We have taken the conclusions to heart in our

current strategy.

P.Kunszt DGC5 4.9.2002 13

Issues

• Testing on the ‘real’ testbed.• Dependencies upon components by other WPs

need to be understood so that we will have a good release plan

• Optimizing Communication

P.Kunszt DGC5 4.9.2002 14

Thank you for your Attention