Page 1

How networks can fulfil today‘s and tomorrow‘s Grid demands

- Organisational and Technical Challenges -

Marcus Pattloch, Karin Schauerhammer, Klaus Ullmann(DFN-Verein, Germany, pattloch@dfn.de)

29. March 2007, ISGC 2007, Taipei

Page 2

Contents

• User Collaboration Structure• Economical and Technical Challenges

– Some Technical Definitions– Examples: X-WiN, Geant2 and the LHCOPN– OPN Building Blocks– Forecasts

• VPNs/OPNs, Grids’ Role for Networking, Transmission Technology, User Demands

– Conclusions • Bandwidth Provision, Future Developments and Needs

• Organisational Challenges

Page 3

How do networked users collaborate?

• Research collaboration has in almost all cases an international dimension

• Example: For LHC collaborations the processes for the four experiment‘s data evaluation has an international dimension of networking which is vital for success

• NRENs (National Research Networks) and Geant2 (Pan European Network) have to adapt to that situation - not only for the LHC experiment evaluation process

Page 4

Economical and Technical Challenges

Page 5

Some Technical Definitions

• VPN: Virtual Private Network– a „user-owned network“ which is built from a

basic technical platform (IP / SDH / DWDM...)

• OPN: Optical (Virtual) Private Network– a VPN based on optical technology

• Hybrid Network– Router based network using a VPN / OPN as a

platform

Page 6

Forecast (F1)

• Communication Market– F1: Liberalisation of communication market in all

European countries will continue, will (amongst others) give better opportunities for research networks and will decrease the „digital divide“ (or the economic conditions for that divide)

Page 7

Example 1: X-WiN (German NREN)

Page 813.10.2006

X-WiN (1): Topology inc. cross-border fibre

FFO

Fibre A

Fibre B

Fibre C

Fibre D

GAR

ERL

BAY

FZJ

AAC BIR

POTTUB

FZK

GSI

DUI

BRE

HAN

BRA MAGBIE

FRA

HEI

STU

REG

DRE

CHE

ZIB

ILM

LEI

JEN

ESF

HUBADH

AWI

Richtung Basel

KEH

Switch/GARR

MUE

Surfnet

Renater

KAISAA

ZEU

HAMDES

DKRZ

KIE

ROS

PSNC

Geant2

Page 9

X-WiN (2): (Hybrid network) Features

• Platform available for national VPNs/OPNs and for the national part of international VPNs/OPNs

• Possible due to lively fibre market in Germany

• Optical technology delivers ample bandwidth, i.e. 160*10 Gbit/s per link

• Costs per 10 Gbit/s link are relatively low (as in Geant2) - in the order of 90 K€/a for 10 Gbit/s

Page 10

• More performance– performance increase by factor 4 since 01/06 (same costs) – more performance available as of 01/07 (for the same price)

• More flexibility– no volume charging (and no usage limitation)– Ethernet as additional access technology– hybrid PoPs enable VPNs

• Higher availability– during design for backbone implicitly taken into account

X-WiN (3): Targets (network design)

Page 11

X-WiN (4): Optical platform

• Toolbox for the provision of– DFNInternet (10 Gbps to 10 Mbps)– VPNs/OPNs based on optical links– services like DFNVideoConference, DFN-PKI,

DFN-CERT, DFNRoaming, DFNNews unchanged

• New cost structures for optical networks enable very economic network solutions for specialised services like Grids (for example OPNs)

Page 12

FZK

X-WiN (5): Router Platform for IP

BIE

MUE

DUI

BIR

GOE

KAS

MAR

GIE FRA

GSI HEI STU GAR

BAY

REG

AUG

ERL

POT

ILM

JEN

LEI

CHE

DRE

HAN

BRE DES TUB

ZIB ADH

HUB

KIE ROS GREEWE

BRA

SAA

AAC

HAM

FFO

2x10GE

10GE

1GEKAI

WUE

FRACRS-

POTCRS-

HANCRS-

ERLCRS-

MAG

XR 43 CISCO7609

KR

FZJ

AWI

Page 13

X-WiN (6) - Status

• Technical concept backbone– backbone consists of (dark) fibre and leased

circuits– operational responsibility: DFN– more than before bought in partial services like

24/7 hotline– much more than before DFN PoPs

• Economic concept– backbone put together from different service

offerings from the market and integrated under DFN responsibility

– most effective usage of competition on the market

Page 14

X-WiN (7): Scalability of architecture

• Options for transfer– 10 Gbit/s ~ 1014 Byte/d or 100 TByte/d– eleven 10 Gbit/s links -> more than 1 PByte/d or

roughly 0.5 ExaByte/a

• If one 10 Gbit/s link per T1 is not sufficient– installation of just another 10 Gbit/s link T0 - T1 if

possible on physically separated fibre path– architecture covers this completely– upper limit of the technology is 160 links per

optical path

Page 15

Conclusion (C1)

• Bandwidth Provision– C1: Bandwidth provision, which has been a major

economic problem for any research network over the past decade, will not be a big problem in the future (for 10 Gbit/s and below)

Page 16

Forecasts (F2 & F3)

• VPNs / OPNs– F2: VPNs/OPNs will in a few years carry the bulk

of scientific data in European networks (NRENs/Geant) and perhaps also world-wide

– F3: Migration to that scenario is an evolutionary rather than a revolutionary process defined by user group’s needs and available new network technology

Page 17

Forecasts (F4 & F5)

• Role of Grids– F4: Grids and Grid like systems will for the time

being be a major driver for the VPN migration scenario process (examples LHCOPN, DEISA network etc.)

– F5: Networking technology developments like network management (example: monitoring of VPNs/OPNs in multi domain environments) or network security are still needed for serving „Grid infrastructures“

Page 18

OPN Building Blocks

Page 19

OPN Building Blocks (1): E2E Links

GEANT2

NREN1 NREN2 NREN3E1 E3

E2 E2E Link 2E2E Link 3

• E2E Links are dedicated optical multi-gigabit connections• Essentially P2P links, usually using SDH/SONET or Ethernet

• E2E Links are planned as a regular service of Géant2:• Cooperation of several NRENs needed to operate E2E Links• Users need Single Point of Contact (SPOC)

•E2E Link Coordination Unit (E2ECU) brings together Users and NRENs during operations

Page 20

OPN Building Blocks (2): Workflow E2ECU

• Workflows define the interaction between Actors

• Actors: Authorized Users (no End Users), e.g. – LHC GGUS (Global Grid User Support)

– E2ECU (End-to-End Link Coordination Unit )– NREN TNOCs (Thin Network Operation Centres of NRENs)

• Mostly human/organisational communication

• Full life cycle of E2E links is covered

• For now, only Workflows for technical aspects are defined

• Refinements still under discussion in GN2-JRA4/WI3

Page 21

OPN Building Blocks (3): E2E Link Monitoring

• Status information corresponds to network layers 1/2

• Multiple technologies / vendors used to provide an E2E service– Status information is logical abstraction from vendor

solution– No information about physical devices necessary

• Status of partial links (within domains and connecting domains) is provided by NRENs

• E2E link status is an aggregation of partial links

Page 22

• http://cnmdev.lrz-muenchen.de/e2e

Demo Monitoring

Page 23

Example 2: Geant2

Page 24

Geant2 topology as of 09/06

Page 25

Example 3: LHCOPN

Page 26

CERN T0

IN2P3PIC

CNAF

RAL

GRIDKa

NorduGrid TRIUMF

ASCC

SARA

BNL FNAL

NorduNet

S-Janet

Surfnet

DFN

GARR

Rediris

Renater

GEANT2

LHC TIER0 – TIER1 OPN, scenario based on work by R. Sabatino (DANTE)

SWITCH

Page 27

LHCOPN in Europe

• T1-T0 primary connection– for „Geant2 fibre cloud NRENs“ through Geant2

• T1-T1 secondary connection– on separate fibre paths through other fibre.

Secondary connections provide resilience

• OPN approach– high data volume expected, Grid middleware

driving this approach; „low“ prices for optical links due to liberalised situation per country enables it

Page 28

T2 communication to T1 in DE (1)

• Open Issues– Which access pattern is requested by the T2s to

T1? Only GridKa or other T1s as well? „Other T1s“ would be T1s in other NRENs.

– Which access pattern is requested by the T2s to other T2s or T3s?

– What are quantitative access patterns of T2s?

Page 29

T2 communication to T1 in DE (2)

• No specification available yet

• In Germany T2- and T3-sites are known, networking them is now on the agenda

• Principles for this part of networking could be:– T2 sites need 1 Gbit/s access to T1 (which one?)– Build resilient ring of core-T2 sites in Germany– T3 sites access data through extended

DFNInternet service

• Problem has to be solved in 2007

Page 30

Forecasts (F6 - F8)

• Transmission Technology– F6: On a (per user-) stream basis 10 Gbit/s will be

the main bandwidth to be used for the next 2-3 years, perhaps even longer - the difficulty is to get data from sources to sinks at higher speeds than 10 Gbit/s

– F7: 40 Gbit/s or 100 Gbit/s per (user-) stream will follow

– F8: 10 Gbit/s equipment will be very „cheap“

Page 31

Forecasts (F9 & F10)

• Qualitative User Demands– F9: The user (group) demand in the research area

is in almost all cases „multi-domain / multi vendor“ in networking terms (--> see LCG example)

– F10: Users will require „intelligent networks“, i.e. network technology which adapts (at best dynamically) to their requirements

Page 32

Conclusions (C2 & C3)

• Developments– C2: One of the main future challenges for the

developments of research networking is to further work out solutions for multi-domain environments for operational purposes

– C3: Work started in Geant2 („E2E“) but solutions have to be driven further according to developing demands for example from Grid communities like the particle physics community

Page 33

Conclusion (C4)

• Future Needs– C4: Intelligent networks (i.e. „intelligent“ VPNs /

OPNs adaptable and more flexible to user needs) have to be further developed in the future, i.e. VPNs „on demand“ or dynamic VPNs

Page 34

Organisational Background

Page 35

Governing Structure

• Presently the NREN Policy Committee (NRENPC) is successfully governing the networking policy definition and always devised a flexible substructure (for example Exec) to adapt to management needs

Page 36

The NRENPC as of 01/07

Country NRENAustria (ACOnet) Belgium (BELNET)Bulgaria (BREB/ISTF)Croatia (CARNet)Czech Republic (CESNET) Cyprus (CYNET)Germany (DFN)Estonia (EENet) France (RENATER) Greece (GRNET)Hungary (HUNGARNET) Ireland (HEANet)Israel (IUCC)Italy (GARR) Latvia (LATNET)Lithuania (LITNET) Luxembourg (RESTENA)Malta (UoM)Netherlands (SURFNET)

Country NRENNordic Countries (NorduNet)

Denmark, Finland, Iceland, Norway, Sweden

Poland (PSNC) Portugal (FCCN) Romania (RoEduNet) Russia (JSCC)Slovakia (SANET)Slovenia (ARNES) Spain (RedIRIS)Switzerland (SWITCH) Turkey (ULAKBIM)United Kingdom (UKERNA)

Plus Non-Voting Members:DANTE, TERENAPerm. Observers: CERN, AMREJ, MARNET

Page 37

Summary

• Economic situation for data networks improved drastically within the last 10 years

• 10 Gbit/s VPNs economically achievable today

• High performance network technology is/has been introduced in a couple of NRENs and Geant2; they will be upgraded according to available new network technology

• Main development topic: multi-domain issues

• Close coordination between demanding user groups and networkers is absolutely necessary