Slide Slide David Britton, University of Glasgow IET, Oct 09 1

Prof. David BrittonGridPP Project leaderUniversity of Glasgow

GridPP26 Collaboration Meeting29th March 2011

Efficient & Effective

Slide Slide

Outline

• Where are we?• A brief look back• The theme of this

meeting• A brief look forward

David Britton, University of Glasgow GridPP26 2

Slide Slide

201320122011

Where are we?

David Britton, University of Glasgow GridPP26 3

2001 2002 2003 2004 2005 2006 2007 2008 2009

--------- GridPP1 ------------------ GridPP2 ------------------ GridPP3 ---------

GridPP2+

From Web to Grid From Prototype to Production

From Production to Exploitation

2010 2014

------------- GridPP4 -----------

Computing in the LHC era

During GridPP3, the CPU delivered by the UK grew a factor of 10.

Seneca: "Every new beginning comes from some other beginning's end."

Slide Slide

10/12/2002 28/06/2003 14/01/2004 01/08/2004 17/02/2005 05/09/2005 24/03/2006 10/10/2006 28/04/2007 14/11/20070

20

40

60

80

100

120

UK reliability10/12/2002 28/06/2003 14/01/2004 01/08/2004 17/02/2005 05/09/2005 24/03/2006 10/10/2006 28/04/2007 14/11/2007

0

20

40

60

80

100

120

"Tier1 Reliability"

Observations on GridPP3

David Britton, University of Glasgow GridPP26 4

Data Taking

Data Taking

GridPP3

Slide Slide

Observations on GridPP3

David Britton, University of Glasgow GridPP26 5

During GridPP3, the UK was the largest EGEE contributor of CPU to the LHC VOs

But, we have struggled to get CPU efficiency (CPU/Wall time) consistently above 85%. Many factors here.

85%

Slide Slide David Britton, University of Glasgow GridPP26 6

Observations on GridPP3

… but I doubt the fates are out of ammo yet!

We dodged a lot of bullets…

SSC

CSRR89

CASTOROPN

LHC

Disk

Disk

HW-£

Slide Slide

Efficient & Effective

David Britton, University of Glasgow GridPP26 7

Getting some exercise whilst putting the baby to sleep and walking the dog: Efficient and effective!

…. But clearly one needs to do choose effective things to do efficiently!

Slide Slide

Efficient & Effective

David Britton, University of Glasgow GridPP26 8

Efficiency: Number of tickets assigned/day.

Effectiveness: Fraction of tickets assigned to correct expert

100%

Efficiency is more about speed or quantity.Effectiveness is more about quality.

Help-desk operation

Slide Slide

Meetings?

David Britton, University of Glasgow GridPP26 9

Since the beginning:

16 OC meetings16 CB meetings30 UB/EB meetings420 PMB meetings776 UK-ROC (inc dTeam)

Slide Slide

GridPP4 Structure

David Britton, University of Glasgow GridPP26 10

Oversight Committee changes from bi-annual to a mid-term and full-term review.

The CB stays the same: Group leaders from each institute.

User Board becomes entirely virtual with a User Coordinator responsible for resource scheduling and community contact.

The PMB stays largely the same.

The dTeam becomes the ops-Team and is re-aligned with new Tier-2 structure and the development of a UK NGI.

Slide Slide

Tier-2 Structure

David Britton, University of Glasgow GridPP26 11

Evolve from 4 x Tier-2 coordinators plus 4 x deputies (the latter did not broaden the site representation) to 8 people from the key sites (as identified by the experiments) plus EGI representatives. Goal is to better align ops-team with the experiments and with EGI/NGI. It also ensures engagement in the national operations of all sites in which significant manpower has been invested. Tier-2s may decide to keep much of their internal coordination as is.

Slide Slide

GridPP4 Funding

David Britton, University of Glasgow GridPP26 12

GridPP4 award is now approved for the full four years (April 2011 to March 2015) with the exception of the Tier-2 hardware funding.

We are clarifying with STFC that we buy a service from the Tier-2s, thus this funding should probably not (all) be classified as “capital”. This may help to secure approval.

The first-tranche of GridPP4 grants came in about 2.5% higher than planned. We need to save this money on the second-tranche (reduction of 1-month? per post). To facilitate this, we will arrange for no-cost extensions of the first-tranche grants to provide a 6-month overlap.

We probably don’t want to rush to issue the new grants just yet due to the new grants system.

Slide Slide

Elephant #1

David Britton, University of Glasgow GridPP26 13

19-Apr-01 14-Jan-04 10-Oct-06 06-Jul-09 01-Apr-12 27-Dec-14-3.5

-2.5

-1.5

-0.5

0.5

1.5

2.5Disk Costs

Series1Best fit

Date

Ln

(K

£/T

B)

19-Apr-01 14-Jan-04 10-Oct-06 06-Jul-09 01-Apr-12 27-Dec-14-6.0

-5.0

-4.0

-3.0

-2.0

-1.0

0.0 CPU Costs

Series1

Best fit to past purchases

Series5

Series7

Date

Ln

(K£

/KS

I2K

)

GridPP4 Proposal: Disk+CPU:£2.8m at Tier-1; £2.7m at Tier-2

19-Apr-01 14-Jan-04 10-Oct-06 06-Jul-09 01-Apr-12 27-Dec-14-6.0

-5.0

-4.0

-3.0

-2.0

-1.0

0.0 CPU Costs

Series1

Best fit to past purchases

Series5

Series7

Date

Ln

(K£

/KS

I2K

)

19-Apr-01 14-Jan-04 10-Oct-06 06-Jul-09 01-Apr-12 27-Dec-14-3.5-2.5-1.5-0.50.51.52.53.5

Disk CostsSeries1Best fit

Date

Ln

(K

£/T

B)

Current Estimates: Disk+CPU£3.5m at Tier-1 £3.3m at Tier-2

Slide Slide

Elephant #2

David Britton, University of Glasgow GridPP26 14

Manage with 2011Shift from 2013 to 2012Big increase for LHCbShift towards Tier-2s

Slide Slide

Whither to?

David Britton, University of Glasgow GridPP26 15

Towards simplicity.

Less demands on custodial disk storage: storage-free Tier-2s? commercial solutions (W1R0)? virtual data?

More commonality: ARC, gLite, Unicore: EMI? Experiment proprietary solutions. Virtualisation.

Taming the Ogres: CASTOR (and beyond) Data Bases Tape Systems

Exploiting the new:

More network-centric architectures?

New computational architectures?

Future storage concepts?

Economically viable commercial offerings?

Slide Slide

Summary

David Britton, University of Glasgow GridPP26 16

In GridPP4, driven by the increasing demands from LHC data, we need to strive towards greater efficiency and effectiveness. This is the underlying theme of the meeting.

It is important because in the next few years we are likely to experience for the first time, real and sustained contention for resources. This will be against a backdrop of financial austerity, higher-than-planned hardware prices, and increased experiment resource requests.

We need to remain focused on the, sometimes changing, needs of the experiments. It will take agility, good-humour, cooperation, innovation, anticipation, hard-work, some deep-breaths, (and so on)...

This meeting marks the end (of GridPP3) and the beginning (of GridPP4). It is an opportunity to share in our past successes and refine our vision of the future.