Tony Doyle GridPP Oversight Committee 15 May 2002

Tony Doyle GridPP Oversight Committee15 May 2002

Tony Doyle - University of Glasgow

Document MappingDocument Mapping

Exec SummaryExec Summary GoalsGoals Metrics for successMetrics for success Project ElementsProject Elements

Risks/DependenciesRisks/Dependencies(and (and

mechanisms)mechanisms) SummarySummary

PMB-02-EXECPMB-02-EXEC PMB-01-VISIONPMB-01-VISION PMB-02-EXECPMB-02-EXEC Gantt Charts,Gantt Charts,

PMB-05-LCG,PMB-05-LCG, TB-01-Q5-Report, TB-01-Q5-Report, TB- TB-02-UKRollout, PMB-06-02-UKRollout, PMB-06-TierAstatus, PMB-04-TierAstatus, PMB-04-ResourcesResources

PMB-03-STATUS, PMB-03-STATUS, PMB-07-INSTRUMENTSPMB-07-INSTRUMENTS

PMB-02-EXECPMB-02-EXEC


OutlineOutline

The Vision The Vision Thing…Thing…

GridGrid1.1. ScaleScale2.2. IntegrationIntegration3.3. DisseminationDissemination4.4. LHC AnalysesLHC Analyses5.5. Other AnalysesOther Analyses

6.6. DataGridDataGrid

7.7. LCGLCG

8.8. InteroperabilityInteroperability

9.9. InfrastructureInfrastructure

10.10.FinanacesFinanaces SummarySummary


GridPP DocumentsGridPP Documents

GridPP Project Management Board

Vision Statement

From Web to Grid -Building the next IT Revolution

DOCUMENT IDENTIFIER: GridPP-PMB-01-Vision

Date: 07/05/2002

Version: 1.0

Document status: FINAL

Author Tony Doyle


GridPP VisionGridPP Vision

From Web to Grid - From Web to Grid - Building the next IT Building the next IT RevolutionRevolution

PremisePremise

The next IT revolution will The next IT revolution will be the Grid. The Grid is a be the Grid. The Grid is a practical solution to the practical solution to the data-intensive problems that data-intensive problems that must be overcome if the must be overcome if the computing needs of many computing needs of many scientific communities and scientific communities and industry are to be fulfilled industry are to be fulfilled over the next decade.over the next decade.

AimAim

The GridPP Collaboration The GridPP Collaboration aims to develop and deploy aims to develop and deploy the largest-scale science the largest-scale science Grid in the UK for use by the Grid in the UK for use by the worldwide particle physics worldwide particle physics community.community.

Many Challenges..Shared distributed

infrastructure For all experiments


GridPP ObjectivesGridPP Objectives

1. SCALE: GridPP will deliver the Grid software 1. SCALE: GridPP will deliver the Grid software (middleware) and hardware infrastructure to (middleware) and hardware infrastructure to enable the testing of a prototype of the Grid for enable the testing of a prototype of the Grid for the LHC of significant scale. the LHC of significant scale.

2. INTEGRATION: The GridPP project is designed to 2. INTEGRATION: The GridPP project is designed to integrate with the existing Particle Physics integrate with the existing Particle Physics programme within the UK, thus enabling early programme within the UK, thus enabling early deployment and full testing of Grid technology deployment and full testing of Grid technology and efficient use of limited resources. and efficient use of limited resources.

3. DISSEMINATION: The project will disseminate the 3. DISSEMINATION: The project will disseminate the GridPP deliverables in the multi-disciplinary e-GridPP deliverables in the multi-disciplinary e-science environment and will seek to build science environment and will seek to build collaborations with emerging non-PPARC Grid collaborations with emerging non-PPARC Grid activities both nationally and internationally.activities both nationally and internationally.

4. UK PHYSICS ANALYSES (LHC): The main aim is to 4. UK PHYSICS ANALYSES (LHC): The main aim is to provide a computing environment for the UK provide a computing environment for the UK Particle Physics Community capable of meeting Particle Physics Community capable of meeting the challenges posed by the unprecedented data the challenges posed by the unprecedented data requirements of the LHC experiments.requirements of the LHC experiments.

5. UK PHYSICS ANALYSES (OTHER): The process of 5. UK PHYSICS ANALYSES (OTHER): The process of creating and testing the computing environment creating and testing the computing environment for the LHC will naturally provide for the needs for the LHC will naturally provide for the needs of the current generation of highly data of the current generation of highly data intensive Particle Physics experiments: these intensive Particle Physics experiments: these will provide a live test environment for GridPP will provide a live test environment for GridPP research and development.research and development.

6. DATAGRID: Grid technology is the framework used to 6. DATAGRID: Grid technology is the framework used to develop this capability: key components will be develop this capability: key components will be developed as part of the EU DataGrid project and developed as part of the EU DataGrid project and elsewhere.elsewhere.

7. LCG: The collaboration builds on the strong 7. LCG: The collaboration builds on the strong computing traditions of the UK at CERN. The computing traditions of the UK at CERN. The CERN working groups will make a major CERN working groups will make a major contribution to the LCG research and contribution to the LCG research and development programme.development programme.

8. INTEROPERABILITY: The proposal is also integrated 8. INTEROPERABILITY: The proposal is also integrated with developments from elsewhere in order to with developments from elsewhere in order to ensure the development of a common set of ensure the development of a common set of principles, protocols and standards that can principles, protocols and standards that can support a wide range of applications. support a wide range of applications.

9. INFRASTRUCTURE: Provision is made for facilities at 9. INFRASTRUCTURE: Provision is made for facilities at CERN (Tier-0), RAL (Tier-1) and use of up to four CERN (Tier-0), RAL (Tier-1) and use of up to four Regional Centres (Tier-2).Regional Centres (Tier-2).

10. OTHER FUNDING: These centres will provide a focus 10. OTHER FUNDING: These centres will provide a focus for dissemination to the academic and for dissemination to the academic and commercial sector and are expected to attract commercial sector and are expected to attract funds from elsewhere such that the full funds from elsewhere such that the full programme can be realised.programme can be realised.

(…. WHAT WE SAID WE COULD DO (…. WHAT WE SAID WE COULD DO IN THE PROPOSAL)IN THE PROPOSAL)


Grid – A Single ResourceGrid – A Single Resource

Peta Bytes of data storage

Many millions

of events

Many samples

Distributed resources

Many 1000s of computers required

GRIDA unified approach

Worldwide collaboration

Various conditions

Heterogeneous operating systems



Grid - What’s been happening?Grid - What’s been happening?

A lot…

GGF4, OGSA and support of IBM (and others) GGF4, OGSA and support of IBM (and others) [as opposed to .NET development framework and passports

to access services] Timescale? September 2002

W3C architecture for web servicesW3C architecture for web services Chose (gzipped) XML as opposed to other solutions for

metadata descriptions… and web-based interfaces

linux linux [as opposed to other platforms… lindows??]

C++ (experiments) and C, Java (middleware) APIsC++ (experiments) and C, Java (middleware) APIs [mono - Open Source implementation of the .NET

Development Framework??]

OGSA



GridPP ContextGridPP Context

Provide architecture and middleware

Use the Grid with simulated data

Use the Grid with real data

Future LHC Experiments

Running US Experiments

Build Tier-A/prototype Tier-1 and Tier-2 centres in the UK and join worldwide

effort to develop middleware for the

experiments


EDG TestBed 1 StatusEDG TestBed 1 Status

Web interface Web interface showing status of showing status of (~400) servers at (~400) servers at testbed 1 sitestestbed 1 sites


GRIDextend to all expts


LHC computing at a glanceLHC computing at a glance

The investment in LHC computing will be massiveThe investment in LHC computing will be massive LHC Review estimated 240MCHF (before LHC delay) 80MCHF/y afterwards

These facilities will be distributedThese facilities will be distributed Political as well as sociological and practical reasons

Europe:267 institutes, 4603 users

Elsewhere: 208 institutes, 1632 users

Eng. & accel. services

Infrastructure (non-physics)

non-LHC share of base physics svcs & infrastr.

LHC share of base physics services &

infrastructure

Physics WANComputer centre

refurbishment

PrototypeOutsourced

administration & operation

Tier 0 investment

Tier 1 investment

0

10

20

30

40

50

60

2001 2002 2003 2004 2005 2006 2007 2008

year

MC

HF

Funding available (MTP)

1. s1. sccaallee


RTAG StatusRTAG Status

6 RTAGs created to date: 6 RTAGs created to date: RTAG1 (Persistency Framework; status: completed) RTAG2 (Managing LCG Software; status: running) RTAG3 (Math Library Review; status: running) RTAG4 (GRID Use Cases; status: starting) RTAG5 (Mass Storage; status: running) RTAG6 (Regional Centres; status: starting)

Two more in advanced state of preparation:Two more in advanced state of preparation: Simulation components Data Definition Tools

7. LCG7. LCG


Fabrics & Grid DeploymentFabrics & Grid Deployment

LCG Level 1 Milestone: deploy a LCG Level 1 Milestone: deploy a Global Grid Service Global Grid Service within within 1 year1 year sustained 24 X 7 service including sites from three continents

identical or compatible Grid middleware and infrastructure

several times the capacity of the CERN facility and as easy to use

Ongoing work at CERN to increase automation and Ongoing work at CERN to increase automation and streamline configuration, especially for migration to streamline configuration, especially for migration to RedHat 7.2.RedHat 7.2.

Aim to phase out old CERN solutions by mid-2003.

7. LCG7. LCG


LCG TimelineLCG Timeline

2002 200520042003

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4Q1 Q2 Q3 Q4

Prototype of Hybrid Event Store (Persistency Framework)

Hybrid Event Store available for general users

Distributed production using grid services

First Global Grid Service (LCG-1) available

Distributed end-user interactive analysis

Full Persistency Framework

LCG-1 reliability and performance targets

“50% prototype” (LCG-3) available

LHC Global Grid TDR

applicationsapplications

gridgrid

1. 1. titimmeessccaallee


Be a part of this?Be a part of this?

LCG DevelopmentLCG Development– – Long Term Attachment at CERNLong Term Attachment at CERN This will enable Grid developments This will enable Grid developments

in the UK to be (more) fully in the UK to be (more) fully integrated with long-term Grid integrated with long-term Grid development plans at CERN.development plans at CERN.

The proposed mechanism is:The proposed mechanism is: 1. submit a short one-page outline 1. submit a short one-page outline

of current and proposed work, of current and proposed work, noting how this work can best be noting how this work can best be developed within a named team at developed within a named team at CERN, by e-mail to the GridPP CERN, by e-mail to the GridPP Project Leader (Tony Doyle) and Project Leader (Tony Doyle) and GridPP CERN Liaison (Tony Cass).GridPP CERN Liaison (Tony Cass).

2. This case will be discussed at 2. This case will be discussed at the following weekly GridPP PMB the following weekly GridPP PMB meeting and outcomes will be meeting and outcomes will be communicated as soon as communicated as soon as possible by e-mail following that possible by e-mail following that meeting. meeting.

NotesNotes1. The minimum period for LTA is 3 1. The minimum period for LTA is 3

months. It is expected that a work months. It is expected that a work programme will be typically for 6 programme will be typically for 6 months (or more).months (or more).

2. Prior DataGrid and LHC (or other) 2. Prior DataGrid and LHC (or other) experiments' Grid work are experiments' Grid work are normally expected.normally expected.

3. It is worthwhile reading3. It is worthwhile readinghttp://http://cerncern..chch//lcglcg//pebpeb/applications/applications in order to get an idea of the areas in order to get an idea of the areas

covered, and the emphasis placed, covered, and the emphasis placed, by the LCG project on specific by the LCG project on specific areas (building upon DataGrid and areas (building upon DataGrid and LHC experiments' developments).LHC experiments' developments).

4. Please send all enquiries and 4. Please send all enquiries and proposals to:proposals to:

Tony Doyle <[email protected]> Tony Doyle <[email protected]> andand

Tony CASS <[email protected]>Tony CASS <[email protected]>


Summary of LCGSummary of LCG

Project got under way early this yearProject got under way early this year Launch workshop and early RTAGs give good input for high-level Launch workshop and early RTAGs give good input for high-level

planning …planning … … … to be presented to LHCC in Julyto be presented to LHCC in July New plan takes account of first beam in 2007New plan takes account of first beam in 2007 No serious problems foreseen in synchronising LCG plans with No serious problems foreseen in synchronising LCG plans with

those of the experimentsthose of the experiments Collaboration with the many Grid projects needs more workCollaboration with the many Grid projects needs more work Technical collaboration with the Regional Centres has to be Technical collaboration with the Regional Centres has to be

establishedestablished Recruitment of special staff going well (but need to keep the Recruitment of special staff going well (but need to keep the

recruitment momentum going)recruitment momentum going) Serious problem with materials fundingSerious problem with materials funding

7. LCG7. LCG


Building upon SuccessBuilding upon Success

The most important criterion for establishing the status of The most important criterion for establishing the status of this project was the European Commission review on this project was the European Commission review on March 1st 2002. March 1st 2002.

The review report of project IST-2000-25182 DATAGRID is The review report of project IST-2000-25182 DATAGRID is available from PPARC. available from PPARC.

The covering letter states “As a general conclusion, the The covering letter states “As a general conclusion, the reviewers found that the overall performance of the project reviewers found that the overall performance of the project is good and in some areas beyond expectations.” is good and in some areas beyond expectations.”

The reviewers state “The deliverables due for the first The reviewers state “The deliverables due for the first review were in general of excellent quality, and all of them review were in general of excellent quality, and all of them were available on time… All deliverables are approved. The were available on time… All deliverables are approved. The project is doing well, exceeding expectations in some project is doing well, exceeding expectations in some areas, and coping successfully with the challenges due to areas, and coping successfully with the challenges due to its size.” its size.”

6. DataGrid6. DataGrid




WP1 – Workload Management WP1 – Workload Management (Job Submission)(Job Submission)

1. Authenticationgrid-proxy-init

2. Job submission to DataGriddg-job-submit

3. Monitoring and controldg-job-statusdg-job-canceldg-job-get-output

4. Data publication and replication (WP2)globus-url-copy, GDMP

5. Resource scheduling – use of CERN MSS

JDL, sandboxes, storage elements

Important to implement this

for all experiments…



WP2 - SpitfireWP2 - Spitfire 6. DataGrid6. DataGrid


WP3 - R-GMA WP3 - R-GMA

Consumer Servlet

RegistryAPI

Consumer Servlet

RegistryAPI

Consumer Servlet

RegistryAPI

Consumer Servlet

RegistryAPI

Sensor Code

ProducerAPI

Application Code

ConsumerAPI

ProducerServlet

RegistryAPI

Registry Servlet

SchemaAPI

Schema Servlet

“Event Dictionary”

Application Code

ArchiverAPI

DBProducer

DBProducerServlet

Archiver Servlet

ConsumerAPIConsumer

APIConsumerAPIConsumer

API

User code here.Builds on R-GMA

Database Structures.

User code monitors output here.



The LCFG Architecture

LCFG Source Files

mkxprof

Web Server

XML Profi le

ldxprofldxprof

GenericComponent

GenericComponent

rdxprofrdxprof

LCFG Components

DBM File

Server

Client

WP4 - LCFG WP4 - LCFG



Interface

Queue M

anager

Request Manager

Pipe Manager

Tape

Disk

Named Pipe

1

2

3

4

5

6

78

Interface Layer The Core and the Bottom Layer

MS

MHandler

Named Pipe

Named Pipe

Named Pipe

Pipe Store

Network

Data Flow Diagram for SEData Flow Diagram for SEWP5 – Storage ElementWP5 – Storage Element

A consistent interface to MSS.A consistent interface to MSS. MSSMSS

CastorCastorHPSSHPSSRAID arraysRAID arraysSRMSRMDMFDMFEnstoreEnstore

Interfaces Interfaces GridFTPGridFTPGridRFIOGridRFIO/grid/gridOGSAOGSA



WP6 - TestBed 1 StatusWP6 - TestBed 1 Status

Web interface Web interface showing status of showing status of (~400) servers at (~400) servers at testbed 1 sitestestbed 1 sites

GRIDextend to all expts



WP7 – Network MonitoringWP7 – Network Monitoring 6. DataGrid6. DataGrid


WP7 - EDG AuthorisationWP7 - EDG Authorisationgrid-mapfilegrid-mapfile generation generation

o=testbed,dc=eu-datagrid, dc=org

CN=Franz Elmer

ou=People

CN=John Smith

mkgridmap grid-mapfile

VOVODirectoryDirectory

““AuthorizationAuthorizationDirectory”Directory”

CN=Mario Rossi

o=xyz,dc=eu-datagrid, dc=org

CN=Franz ElmerCN=John Smith

Authentication Certificate



ou=People ou=Testbed1 ou=???

local users ban list



1. Realistic Large-Scale Tests1. Realistic Large-Scale Tests Reliability! Need reliable dg-job-*

command suite 2. Data management2. Data management

Reliability! Need reliable gdmp-* command suite, file-transfer commands

3. Mass Storage Support3. Mass Storage Support Working access to MSS (CASTOR

and HPSS at CERN, Lyon) 4. Lightweight User Interface4. Lightweight User Interface

Put on a laptop or std. Desktop machine

5. Portability Demonstrable portability of

middleware: a) use other resources, b) debugging

6. Scratch Space Job requests X amount of

scratch space to be available during execution, system tells job where it is

7. Output File Support JDL support for output files:

specify where output should go in JDL, not in job script

WP8 - ApplicationsWP8 - Applications 6. DataGrid6. DataGrid


Expt. FeedbackExpt. Feedback 4. and 5. Expts4. and 5. Expts


= Minimal = Minimal e-Bureaucracye-Bureaucracy

8. Interoperability8. Interoperability

5. Other Expts5. Other Expts


GRID JOB SUBMISSION GRID JOB SUBMISSION – External User Experience– External User Experience 5. Other Expts5. Other Expts


Things Missing, apparently…Things Missing, apparently… 5. Other Expts5. Other Expts


Expt. FeedbackExpt. Feedback 4. and 5. Expts4. and 5. Expts


GridPP PosterGridPP Poster 3. Dissemination3. Dissemination


Tier 1/A EDG PosterTier 1/A EDG Poster 3. Dissemination3. Dissemination


BaBar PosterBaBar Poster 3. Dissemination3. Dissemination


LHCb PosterLHCb Poster 3. Dissemination3. Dissemination


ScotGRID PosterScotGRID Poster 3. Dissemination3. Dissemination


Identifiable Progress...Identifiable Progress...

t0

t1

3. Dissemination3. Dissemination


WebLogWebLog

Allows every area/sub group to have its own 'news' pagesAllows every area/sub group to have its own 'news' pages


GridPP & Core e-Science CentresGridPP & Core e-Science Centres

NeSCNeSC Close ties, hosted 2nd GridPP Collaboration Meeting,

Collaboration on EDIKT Project? Training...

BelfastBelfast Replied but not yet up and running.

CambridgeCambridge Close ties, hosted 3rd GridPP Collaboration Meeting. Share one

post with GridPP. Will collaborate on ATLAS Data Challenges.

CardiffCardiff Replied - contacts through QM (Vista) and Brunel GridPP Group.

Written formally to all e-Science centres inviting contact and Written formally to all e-Science centres inviting contact and collaboration with GridPP.collaboration with GridPP.



GridPP & Core e-Science CentresGridPP & Core e-Science Centres

LondonLondon No formal reply but close contacts through IC HEP Group.

IC will host 5th GridPP Collaboration Meeting.

ManchesterManchester No collab. projects so far. Manchester HEP Group will host

4th GridPP Collaboration Meeting.

NewcastleNewcastle In contact - Database projects?

OxfordOxford Close ties, collaboration between Oxford HEP Group and

GridPP on establishment of central Tier-2 centre? CS/Core-GridPP-EDG links? Probably host 6th GridPP Collaboration Meeting.

SouthamptonSouthampton Replied but no collaboration as yet.



GLUEGLUE

How do we integrate with developments from elsewhere in How do we integrate with developments from elsewhere in order to ensure the development of a common set of order to ensure the development of a common set of principles, protocols and standards that can support a principles, protocols and standards that can support a wide range of applications?wide range of applications?

GGF… GGF… Within the Particle Physics community, these ideas are Within the Particle Physics community, these ideas are

currently encapsulated in the Grid Laboratory Uniform currently encapsulated in the Grid Laboratory Uniform Environment (Environment (GLUEGLUE). ).

Recommend this as a starting point for the wider Recommend this as a starting point for the wider deployment of Grids across the Atlantic. See deployment of Grids across the Atlantic. See http://www.http://www.hicbhicb.org/glue/GLUE-v0.1.doc.org/glue/GLUE-v0.1.doc (Ruth Pordes (Ruth Pordes et al.)et al.)





UK Tier-A/prototype Tier-1 CentreUK Tier-A/prototype Tier-1 Centre

RolesRolesTier-A Centre for BaBar Tier-A Centre for BaBar EDG testbed(s)EDG testbed(s)LCG prototype Tier-1 Centre LCG prototype Tier-1 Centre prototype Tier-1 for LHC experiments (Data Challenges prototype Tier-1 for LHC experiments (Data Challenges independent independent of LCG development…)of LCG development…)Interworking with other UK resources (JIF, JREI, eSC) Interworking with other UK resources (JIF, JREI, eSC) = = UK portalUK portalexisting LEP, DESY and non-accelerator experimentsexisting LEP, DESY and non-accelerator experiments

PurchasesPurchases First year = Hardware Advisory Group (HAG1)First year = Hardware Advisory Group (HAG1) Determine balance between cpu, disk, and tape Determine balance between cpu, disk, and tape Experts on specific technologiesExperts on specific technologies Propose more HAGs (2 and 3).. Propose more HAGs (2 and 3).. Needs to be successful in all roles...Needs to be successful in all roles...

9. Infrastructure9. Infrastructure


Rollout of the UK Grid for PPRollout of the UK Grid for PP

Operational stability of GridPP middleware = Testbed teamOperational stability of GridPP middleware = Testbed team The “gang of four” … Andrew McNab, Steve Traylen, Dave The “gang of four” … Andrew McNab, Steve Traylen, Dave

Colling (other half) and Owen MoroneyColling (other half) and Owen Moroney Ensures the release of “Testbed” quality EDG softwareEnsures the release of “Testbed” quality EDG software

documentation lead for other system managers in terms of implementation pre-defined software cycle releases (2 months..)

Subject of the Rollout Plan… “Planning for EDG Testbed Subject of the Rollout Plan… “Planning for EDG Testbed software deployment and support at participating UK software deployment and support at participating UK sites” (Pete Clarke, John Gordon)sites” (Pete Clarke, John Gordon)

LCG is the proposed mechanism by which the EDG LCG is the proposed mechanism by which the EDG testbed at CERN becomes an LCG Grid Service. The testbed at CERN becomes an LCG Grid Service. The evolution of the EDG testbed to the LCG Grid Service will evolution of the EDG testbed to the LCG Grid Service will take account of both EDG and US grid technology. Need to take account of both EDG and US grid technology. Need to take account of this..take account of this..



Longer Term..Longer Term..

LCG Grid ServiceLCG Grid Service Takes account of EDG and US grid technology Takes account of EDG and US grid technology A large-scale Grid resource, consistent with the LCG A large-scale Grid resource, consistent with the LCG

timeline, within the UK.timeline, within the UK. Scale in UK? 0.5 Pbytes and 2,000 distrib. CPUsScale in UK? 0.5 Pbytes and 2,000 distrib. CPUs

= GridPP in Sept 2004 = GridPP in Sept 2004 ““50% prototype”50% prototype”

2002 200520042003










LHC Global Grid TDR


gridgrid

2002 200520042003


2002 200520042003










LHC Global Grid TDR


gridgrid



£17m 3-Year Project£17m 3-Year Project

£3.78m

£5.67m

£3.66m

£1.99m

£1.88m

CERN

DataGrid

Tier - 1/A

Applications

Operations

Five componentsFive components Tier-1/A = Hardware + ITD Support Staff DataGrid = DataGrid Posts + PPD Staff Applications = Experiments Posts Operations = Travel + Management + Early Investment CERN = LCG posts + Tier-0 + LTA

10. Finances10. Finances

Dave Dave BrittonBritton


1. Recruitment1. Recruitment

EDG Funded PostsEDG Funded Posts (Middleware/Testbed) (Middleware/Testbed) All 5 in post + 1 additional

EDG Unfunded Posts EDG Unfunded Posts (Middleware/Testbed)(Middleware/Testbed) 15 out of 15 in post

GridPP PostsGridPP Posts (Applications + Tier1/A) (Applications + Tier1/A) Allocated Dec 2001 13 out of 15 in post

CERN PostsCERN Posts First Round = 105 Applicants, 12 Offers, 9 Accepted 4 in Applications, 2 Data Management, 3 Systems Second Round = 140 applicants, 9 Offers Third Round ~ 70 Applicants Aim ~ 28 posts

Status Feb'02

Owen Moroney starts Jan 2002Frederic Brochu (starts 1st Apr 02, part funded by CeSC)Regina Tam started 7th Jan 02Steve Traylen started 14th Jan 02Alexander Holt from 1st Aug 01Gavin McCance started 1st Sep 01 (1FTE for 3 yrs)Will Bell started 1st Nov 01 (1FTE for 2 yrs)Dave Colling from 1st Oct 01 (WP1&6)Phillip Lewis from 1st Oct 01 (WP8)Michael George started 1st Oct 01Andrew McNab started 1st Nov 01Started ~15th Apr 02Started 1st Jan 02Arijeet Datta started 21st Jan 02Mike Gardner started 28th Jan 02Paul Mealor started 1st Jul 01

WP3 : Antony Wilson started 4th Jun 01 Laurence Field started 28th Aug 01 Xiaomei Zhu started 28th Jan 02 Manish Soni started 2nd Apr 02WP5 : Owen Synge started 27th Nov 01 Timothy Eves started 18th Dec 01WP8 : Stephen Burke started 1st Nov 01


2. Monitoring Staff Effort [SM]2. Monitoring Staff Effort [SM] Robin Robin MiddletonMiddleton


3. Progress towards deliverables..3. Progress towards deliverables..

GridPP Experimental Support :Milestone/Deliverable specification and schedule

Milestone/Deliverable Specification

Experiment: LHCb Due: 2002-Q2

Title Technology review document

D/M [Type] [D] DocumentDescription ???? what are you actually reviewing and for what purpose ????Dependencies:(if any)

None



Title Detailed requirements, architecture and design document

D/M [Type] [D] DocumentDescription ???This document will establish the LHCb requirements for its Grid

based analysis and MC production, the architecture it wishes to use inassembling components, and the detailed design of the first prototype.???

Dependencies:(if any)

None



Title First prototype of Grid interface

D/M [Type] [M] Prototype & demonstrationDescription Interface originally based on command lines modelled on current Atlas

and LHCb production tools. It will allow simple Gaudi “analysis” jobs tobe submitted to EDG testbed sites in addition to MC production jobs. Ageneric script syntax will need to be developed and job submission scriptsshould be automatically generated. The scripts have to have an errorcatching mechanism. Functionality will include:

DataGrid job submission (WP1 tools - already exist)

GridPP Experimental Support :Milestone/Deliverable specification and schedule


Experiment: SAM Core Development Due: 2002-Q2

Title Technology Review

D/M [Type] [D] Technology Review reportDescriptionDependencies:(if any)

None



Title Architecture Design

D/M [Type] [D] Document specifying architecture designDescription In parallel with the technology review, the architecture to be used for the

upgrade of SAM will be specified.Dependencies:(if any)

None



Title Demonstration of phase-1 SAM development

D/M [Type] [M] Report on demonstrationDescription The first modifications to SAM will be

- the inclusion of GridFTP as a transport option.- the inclusion of Condor-G to allow submission of a job to a remote

site (site chosen by hand)- Decentralisation of existing SAM information services and development of a

prototype of information services that provides information on availableresources and tracks them.

This milestone will be a demonstration of these upgrades operatingsuccessfully by submitting jobs to key sites in the US and UK

GridPP Experimental Support:Milestone/Deliverable specification and schedule

QCDGrid


Experiment: QCDGrid Due: 2002-Q2

Title Develop an XML Schema for lattice QCD Calculations

D/M [Type] [M] Prototype XML Schema for meta-data catalogue (see nextdeliverable)

Description We aim to develop an XML schema, to define the format of the meta-datadocuments of the lattice QCD data files in an extensible and scientificallymeaningful manner.


None



Title Develop a meta-data data catalogue for QCD simulation data

D/M [Type] [M] Meta-data catalogue and browser demonstratorDescription We aim to develop a XML Database Server (a meta-data catalogue) for

storing and querying the lattice QCD meta-data. Access to this meta-datawill be via command-line tools and through a Browser (see nextdeliverable).


None



Title Develop a Browser for interrogating the meta-data data catalogue

D/M [Type] [M] Meta-data catalogue and browser demonstratorDescription We aim to develop a Browser to allow users to query the meta-data

catalogue. The browser will supply a single interface to the meta-datacatalogue, and ultimately to the data catalogue (see next deliverable).


None

Pete Pete ClarkeClarke


-1. Next steps..-1. Next steps..

O(100k)O(100k) CLRC support through to Sept 04 Other experiments – unfunded in peer review process Tier-2 centres – unfunded initially

£2.3m £2.3m eDIKT (e-Data, Information and Knowledge Transformation) [SHEFC] Particle

Physics = application area - assignment of two (of twelve) FTEs in initial planning. Discussions ongoing with EPCC.

O(€O(€100m)100m) The first call for Framework VI will be early next year. Call out now for expressions of interest for new networks and integrated

projects. Draft document led by David Williams (CERN) “Enabling Grids and e-Science

in Europe” plans to extend the current paradigms with CERN at its focus as the European e-Science Centre.

We believe this is the right approach. Incorporates the UK’s e-Science agenda, adding a European dimension. It also

recognises the central role of CERN and builds upon the recent successes of EDG.

PPARC Contact: Neil Geddes

10. Finances10. Finances


Andrew McNab - Manchester HEP - 10 May 2002

Green Dot G1.1.3 G2.0(b) EDG-CEBabar-CEBirmingham y y yBristol y y y y yBrunel y yCambridge yEdinburgh y yGlasgow y yImperial y y yLancaster y yLiverpool y yManchester y y y y yOxford y yQMUL y y yRAL y y y yRHUL y yUCL y

Testbed Status OverviewTestbed Status Overview MetricsMetrics


What is in place in the UK testbed?(an RB centric view of the world)Only GridPP and Babar VOs

Imperial

R. B.

JSS

II

LB

Bristol

Replica Catalogue

Imperial

CE, SE, UIRAL

CE, SE, UI

Birmingham

CE, UI

Liverpool

CE, UI

Bristol

CE, UI

QMUL

CE, UI

RHUL

CE, UI

IN2P3-Babar

UI

MetricsMetrics


Grid Support CentreGrid Support Centre

UKHEP CA uses primitive technologyUKHEP CA uses primitive technology It works but takes effort 201 personal certs issued (98 still valid) 119 other certs issued (93 still valid)

GSC will run a CA for UK escience CAGSC will run a CA for UK escience CA Uses openCA; Registration Authority uses web

We plan to use itWe plan to use it Namespace identifies RA, not Project

Through GSC we have access to skills of Through GSC we have access to skills of CLRC eSCCLRC eSC

Use helpdesk to formalise support later Use helpdesk to formalise support later in the rolloutin the rollout


UKUK

e-Sciencee-Science

CertificationCertification

AuthorityAuthority

MetricsMetrics


SummarySummary

A vision is only useful if its A vision is only useful if its sharedshared

Grid success is fundamental Grid success is fundamental for PPfor PP

1.1. Scale in UK? 0.5 Pbytes and Scale in UK? 0.5 Pbytes and 2,000 distrib. CPUs 2,000 distrib. CPUs

GridPP in Sept 2004 GridPP in Sept 2004

2.2. Integration – ongoing.. Integration – ongoing..

3.3. Dissemination – external Dissemination – external and internaland internal

4.4. LHC Analyses – ongoing LHC Analyses – ongoing feedback mechanism..feedback mechanism..

5.5. Other Analyses – closely Other Analyses – closely integrated using EDG toolsintegrated using EDG tools

6.6. DataGrid - major investment = DataGrid - major investment = must be (and is so far) successful must be (and is so far) successful

7.7. LCG – Grid as a Service LCG – Grid as a Service

8.8. Interoperability – sticky subjectInteroperability – sticky subject

9.9. Infrastructure – Tier-A/1 in place, Infrastructure – Tier-A/1 in place, Tier-2’s to follow… Tier-2’s to follow…

10.10. Finances – (very well) under control Finances – (very well) under control Next steps on framework VI..Next steps on framework VI.. CERN = EU’s e-science centre?CERN = EU’s e-science centre? Co-operation required with other Co-operation required with other

disciplines/industrydisciplines/industry

11.11. Monitoring mechanisms in placeMonitoring mechanisms in place

12.12. Emphasis on deliverablesEmphasis on deliverables


ExecutiveExecutive22 Summary Summary

Significant progress... Significant progress... Project is now well defined Project is now well defined

in a broad sense and is in a broad sense and is progressing on a series of progressing on a series of fronts.fronts.

We have responded and We have responded and outlined our plans to outlined our plans to address the concerns of the address the concerns of the last OC concerning:last OC concerning:

1. WP5;

2. Rollout plan;

3. Monitoring instruments;

4. Metrics for success.

The project has demonstrated The project has demonstrated progress in:progress in:

1. Widespread deployment of EDG testbeds in the UK;

2. Integration with specific experimental areas (BaBar, UKDMC and LISA); and

3. Demonstrating Grid deployment in the UK at the NeSC opening.

We see various challenges ahead:We see various challenges ahead:1. Development of more detailed

metrics and monitoring of outputs;2. Management of changes due to

external developments (e.g. OGSA);3. Development of Tier-2 deployment;4. Engagement of the UK HEP

community; and5. Future funding initiatives such as

Framework VI.