A Federated opproach

e-Infrastructure convergence in UK 24/11/2015

David Salmon

Background

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 2

From e-Science to e-Infrastructure

• UK e-Science programme – 2001-2009

• Research Council International Review • Recommendations… • HPC papers – lobbying UK governmnet 2010-2011

• Department for Business and Skills

• £154M announced late 2011 • £100M + on HPC in various forms • £26M Janet (Jisc) • Delivering from 2012 onwards

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 3

UK e-Infrastructure components

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 4

• Foundation components

• Data communications substrate

• National R&E network Janet plus high-capacity peerings with: • National commercial Internet Service Providers • International: NRENs via GEANT & commercial

Internet • Compute & Data Storage services – internal and

external plus national frameworks • Research Council National Laboratories and Facilites • Community resources – University/Organisation level

compute & storage resources & local facilities 24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 5

Janet Fibre network & e-Infrastructure extensions

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 6

E-Infrastructure extensions

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 7

Janet: not just big bandwidth

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 8

Netflix

Voicenet

Akamai

Virgin Radio

Bogons

Logicalis UK Pipex

BBC

Datahop

InTechnology

INUK

RM Education

LINX multicast

NHS

Redstone

Google

NetrinoUK Gamma

Updata

aql

Voicenet

NetrinoUK

Google

Limelight

Limelight

Akamai BTnet

Init7

Amazon

Microsoft EU

TMnet

Globelynx

10Gbit/s

1Gbit/s

100Gbit/s

Global Transit Tata

Global Transit Level 3

GÉANT

GÉANT+

Tele-house

TN +TW

LINX

Telecity HX

Microsoft EU (via TW)

Total external connectivity ≈ 900 Gbit/s

Leeds J6 Core

PoP

Global Transit Level 3

Akamai

Google

Manc’r

VM for LGfL

IXManchester

InTechnology

NHS

Exa Networks

Synetrix

BBC (CWGames) One Connect

Glasgow J6 Core

PoP

HEAnet

IXLeeds

BBC (CWGames)

Gamma

BBC (CWGames)

Total by the end of this year ≈ 1 Tbit/s

e-Infrastructure

• Scales • National • Regional • Local – Organisation

• Facility Types • Research Infrastructures • Research domain facilities

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 9

National / International

• STFC (Science and Technologies Facilities Council) • Rutherford Appleton Laboratory • Daresbury Laboratory & Hartree Centre • GridPP – Particle Physics • DIRAC HPC – Particle astrophysics & cosmology

• Wellcome Trust Sanger Institute • EMBL/EBI Hinxton • Norwich BioScience Institutes • Francis Crick Institute • Met Office (MONSooN HPC partnership with NERC – Natural

Environment Research Council) • European Centre for Medium range Weather Forecasting

(ECMWF) 24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 10

National & Regional

• National • EPSRC Archer HPC • EPSRC Research Data Facility

• Regional • ARCHIE-WeSt – Scotland • N8 – Northern England • HPC Midlands • MidPlus HPC • Great Western 4 (GW4) • Science and Engineering South • HPC Wales

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 11

• Other Research Intensive Universities • Local Compute & storage

• Other Publicly funded entities • Catapult Centres – 8 and growing

• Area focus on academic/industry bridge & SME support • Eg Digital Catapult • Satellite Applications Catapult

• NPL – National Physical Laboratory • Others…

• Industry & Commercial entities • Private compute & storage • Commercial service providers

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 12

Co-ordination

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 13

Co-ordination activities

• E-Infrastructure Leadership Council (UK) • Ministerial chair • Research Council, academic & industry members

• UK cross-Research Council e-Infrastructure Working Group • Project Directors Group – broad stakeholder

representation • Security and Access Management Working Group • Cloud Working Group

• HPC Special Interest Group – self organised

• (Big Data Special Interest Group) – self organised

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 14

Stakeholders & contributors

• Research Councils • Jisc • Mature research communities – have private

infrastructure/s • Particle physics • Particle astrophysics and cosmology • Natural Environment Science • Bio-molecular sciences • Medical and Life Sciences

• All looking to a more general solution with workload mobility between private and public/commercial services

• Research community expertise 24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 15

Active areas

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 16

Jisc activities with Research communities

• Networking • Peerings – ISPs. Commercial service providers… • End-to-end performance – encourage more effective use of

networks, higher-throughput • Information assurance and networking - Ann Harding tomorrow

• Research Data management work • Security & access management Working Group

• Contribute to concensus on AAI approaches & mechanisms – more from David Fergusson tomorrow

• Framework contracts for access to commercial providers • Amazon, Microsoft

• HPC access portal - Arcus • Datacentre framework contracts • Collaboration – EGI, EUDAT, e-IRG 24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 17

Facilitating Industry engagement

A connectivity perspective

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 18

Janet Reach Scheme

• £4M e-Infrastructure funds “reserved” for industry support

• Connections to Janet ?

• State-aid compliant scheme designed • Project applications • Must be “innovative” • External assessment • Must contribute 40-50% of total project value • Connection can be 50-60% of total project value

• These are R&D projects (low TRL) – Not close-to-market

• Values are “Market-rate”

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 19

Janet Reach

Large company – Rolls-Royce -10Gbit/s SMEs 1) Assisted Living – 1Gbit/s 2) Radio Astronomy – 2.5 Gbit/s +2/3 more

State-aid

• From UK Government:

• “State-Aid: The Basics Guide”

• If you think your activity might might be state-aid.. • Can you re-design it so it isn’t ? • If you can’t avoid it, take a well-trodden path • Think “State-aid” early • Don’t ignore it • Seek advice…

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 21

Janet Reach

• Heavyweight, high-overhead & beurocratic ! • Inflexible for all concerned – project proposal, asessment,

monitoring – much harder to adapt to changing requirements.

• Vision – industry & academic engagement is fine • Future options ?

• Market-value offerings – by definition no state-aid issue • Flexibility – contract structure • Lowest feasible recurrent cost combined with metered use

and billing • Still have connection overhead in time terms – little

appreciated how long & complex this can be ! • My view needs a minimum 3 year forward-look to be

sensible • 1 year to plan, agree and implement, then 2 years service.

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 22

Commercial entities connecting to Janet

• Not a quick or short-term fix

• Need to have a medium to longer term persistent requirement (and a rational & objective one !)

• If not, many things still possible !

• NRENs have excellent peerings with commercial ISPs – Janet certainly has

• Good starting place – use existing service arrangements via IP peerings

• Establish a programme of work and review as this evolves

• When scale becomes appropriate, consider a direct peering with the NREN/academic partner/facility.

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 23

Convergence ? – evolution…

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 24

Summary points

• Plenty of physical infrastructure • Private / domain specific • Commercial

• Data communications • High quality & capability network

• Critical and end-to-end (multi-domain, holistic) • Organisations, NRENs, GEANT & global partners, ISPs…

• AAI

• Drive evolution through experience of major research communities • Moving workload to external service providers

• Elastic expansion 24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 25

Convergence ?

• “One-stop-shop” – premature • Too simple, real world is complex

• Evolving for “long tail”

• Federated approach where appropriate

• Rational & objective approach to analysing requirements

• Genuine progress in some areas !

• Stakeholder communications

• Strong substrate

• Much development needed in the soft environment above the physical infrastructure

24/11/2015 Title of presentation (Go to ‘View’ menu > ‘Header and Footer…’ to edit the footers on this slide) 26