GridPP Cloud Update & EGI Federated Cloud

Adam HuffmanImperial College

204/06/14

GridPP Cloud Update

304/06/14

GridPP Cloud Update

Upgrade to OpenStack IcehouseComponent by component upgrade with minimal downtime now possible (cf. CERN)

Added new compute nodeNow 216 cores, 692GB RAM

Gradually improving monitoring

404/06/14

GridPP Cloud Update

504/06/14

GridPP Cloud Usage

Stable usage by ATLASStill some problems with the tracking of quotas and instances in the error state

604/06/14

ATLAS Usage

Row 1 Row 2 Row 3 Row 40

2

4

6

8

10

12

Column 1

Column 2

Column 3

704/06/14

ATLAS Usage

804/06/14

ATLAS Usage

904/06/14

GridPP Cloud Usage

Stable usage by LHCbWork by Andrew McNab using Vcycle scheduler, Micro-CernVM image

1004/06/14

LHCb Usage

772 jobs since May 19th

1104/06/14

LHCb Usage

1204/06/14

GridPP Cloud Usage

Ongoing CMS usage by Stealth Cloud at ImperialAndrew Lahiff working on resurrecting glideinWMS submission from RALPlan to add GridPP Cloud as a production CMS resource

Overlap with HLT/CERN AI/Tier 0 work

1304/06/14

GridPP Cloud Observations

Wariness of campus networking re. Public IPsSeparate EGI site record needed to facilitate accounting etc.Quotas probably need to be handled dynamically

Can VO job schedulers cope with this?Schedulers still don't handle instances in error state perfectly

Ideally want mix of flavours/job types to maximise hardware utilisation

ATLAS using 8 vCPUs, LHCb using 1

1404/06/14

GridPP Cloud Plans

Add Ceph storage for Glance (images), Cinder (volumes) and S3-style object storageMove to High Availability setupEGI Federated Cloud

1504/06/14

EGI Federated Cloud

1604/06/14

EGI Federated Cloud

Production launch at EGI Community Forumhttps://indico.egi.eu/indico/materialDisplay.py?contribId=127&sessionId=65&materialId=slides&confId=1994 Appears to aim at replication of grid infrastructure over cloud APIsRequires:

VOMS-enabled KeystoneOCCIAccountingImage marketplaceInformation system

1704/06/14

EGI Federated Cloud

Puppet support for components is unclear (outdated Github repo)Appears to have different motivations than GridPP'Different pricing models under consideration' – Models for Sustainability (under development)Unclear yet how to marry VO and FedCloud usage patterns

Fudge this issue by separating clouds for now

1804/06/14

Thank you …

Frank BerghausAndrew McNab

For pointers to usage plots