1

Grid networking in EU DataGRID

TERENA conferenceLimerick - 5th of June 2002

Pascale PRIMETManager of the workpackage “Network” of the DataGRID project

INRIA/ RESO - ENS LyonPascale.Primet@ens-lyon.fr

2

Outline

• The European DataGRID “network”• High performance Grid Networking• Grid Network Monitoring in EDG• Network Services for the GRID• Perspectives

3

Grid technology

• The purpose of a Grid is to – aggregate a large number of resources– to build a high performance – computing and storage environment.

• The distributed resources may be– interconnected via a VPN– or the Internet.

4

European DataGRID project• The EDG project http://www.eu-datagrid.org/ aims to provide

production quality testbeds, using real-world applications with real data:

• High Energy Physics• process the huge amount of data from LHC experimentations

• Biology and Medical Imaging– sharing of genomic databases for the benefit of international cooperation– processing of medical images for medical collaborations

• Earth Observations– access and analysis of atmospheric ozone data collected by satellites as

Envisat-1

• Calendar : january 2001 to december 2003• Funded by the European Union

5

EDG - Partners• CERN – France• CNRS – France

– Testbed (WP6)– Network (WP7)– Bio application (WP10)

• ESA/ESRIN – Italy • INFN – Italy • NIKHEF – The Netherlands• PPARC - UK

6

European DataGRID project

• 7 applications distributed among 6 virtual organisations

• 11 organisations over 15 countries• 40 sites in Europe• Based on the European GEANT backbone

and National NREN’s http://ccwp7.in2p3.fr

7

EDG - Infrastructure

8

High Performance Grid Networking

• Technical collaboration with Network providers– Requirement studies (Application and middleware)– Available infrastructure and services review– Enhanced Network services tests

• Technical collaboration with Grid users– End to end monitoring – Transport protocols studies and optimisation– E2E performances problems identification– Network cost functions realization for scheduling

9

« Physical » view of a Grid NetworkPublic NetworkNo securityNo predictable performancesNo control on the traffic

The flat INTERNET

Resource = CE (computing element) or Resource = SE(storage element)

10

Logical view of the Grid Network

11

EDG WP7 «Network » activities

Provisionning Monitoring

E2E QoS and Transport Services

Security

Manager: Pascale Primet - INRIA/RESO – 25 persons- 2,5 funded

12

EDG WP7 activitiesT7.1 : Technical Collaboration with Dante/NRENs

– Pilot services test (QoS, multicast)– Dedicated machines in GEANT PoPs

T7.2 : QoS and advanced services - QoS services test with biological/medical applications - Reliable Multicast Protocol test and deployement - High performance transport protocol (TCP/nonTCP)

T7.3 : Network Monitoring Architecture– Design and deploy a Network Monitoring Infrastructure– Visualize and analyze monitoring data

T7.4 : Security => EDG Security team Applications

Middleware

Infrastructure

Managem

ent

Testbe

d

13

Collaboration with GEANT

• E2E : Close participation to pilot services– Test of IP Premium service/WP10

• In Backbone : (our proposal)– Use of dedicated machines in GEANT POPs

• Amsterdam, Geneva, London

– Tests of high throughput transfers– Test of IP multicast for Reliable Multicast– Sharing WP7 monitoring and DANTE monitoring data

14

Network provisioning

• Network Requirements studies• Application Requirements (WP8, WP9, WP10)• Middleware Requirements

• Physical Networks 1. GEANT : 2.5 Gbps to 10 Gbps2. NRENs : from 155Mbps (or less) to 2.5Gbps3. Regional networks: from 2Mbps to 155Mbps4. Local Area Networks : from 10Mbps to 1Gbps)

• Is a « Virtual Private Network » required for the DataGRID ?

• concept definition / VPN technologies review

• See our D7.1 document on WP7 EDG site

15

Methodology

1 Flows

2 Logical links

3 Physical links

4 Monitoring

16

Output file

Application requirement studies Top down stream identification

Data ServerData Server

Data Server

CPU Client

CPU Client

CPU Client

CPU ClientCPU Clientdesktop

desktop

desktop

CERN

CPU Client

Tier 1Tier 2

Tier 3-4

Application scheduler

Comp. resource

Comp. resource

GOME data

GOME processor

GOME archives

Visualisation

Product Archives

GOME product

User control / monitoring

dedzdscdcdscscsdcdscdcds

Input File Database Process Binary

dxcs

GRID

N° Name Application WP TypeTransfert volume(Mbytes)

Frequency(in days)

Average bitrate (Kbit/s)

constraints Observations

1 Monté Carlo Data Réplication LHCb 8 gridftp 30 000 24 66 6672 ENVISAT Data from ground station to storage centreMETEO 9 tcp 5 000 000 0 66 1383 10 86 400 80 000

Flows list

WP8 WP9 WP10

17

Some numbers

• HEP applications:– Bulk Data transfer : from 100Mb/s (TB1) to 1Gb/s cont.

(TB3)

• Medical applications:– Interactive Traffic with burst of more than 1Gbyte– Real Time High Performance Vizualisation/Simulations

18

Network performances measurement (1)

For Provisioning:– To be available, via visualization to human

observer (user, network/system administrators)– To provide tools for network performances

measurement, problems identification and resolution (bottlenecks, point of unreliability, quality of service needs, topology…)

– To achieve network performance forecast and optimization – Capacity planning

19

Network performances measurement (2)

For Resource Brokers:– Network performance parameters are used for

optimizing resource allocation (replication, MPI, Remote file access…)

– Network performance metrics must• be published to the Grid Information System• Be accessible through aggregated functions called

by Grid resource broker services (computing and data storage).

20

Architectural design

• four functional units :– monitoring tools or sensors – a repository for collected data; – the means for data analysis to generate network

metrics; – the means to access and to use the derived

metrics. • See our D7.2 document on WP7 EDG site

21

Network Monitoring Architecture

P_RTPL P_NWS Middleware

Data Collector Raw

IPerf GridFTP SNMP …PingErSensors

Repository

Publication

MapCenter

RTPL

Resource BrokerNetwork managers

Data processor

LDAP

ForecasterAnalysis

22

Measurement methods

• Active methods– Injection of traffic inside the network for testing

performances between two points– problem: may be intrusive (TCP/UDP throughput)

• Passive methods– Collect traffic informations in one point of the network :

router, switch, dedicated passive host, computing element or storage element (GRIDftp logs)…

– Problem : give network usage, not capacity

23

Identify bottlenecks and real throughput availability

Active measurement

24

Passive measurement

Passive measures at one point

25

Metrics and tools

• Round Trip Delay => PinGER (Lyon->nikhef)• Packet Loss => PinGER (Lyon->nikhef)• TCP throughput => IPerfER (nikhef -> Ral)• UDP throughput => UDPMon (CZ->Cern)• site connectivity => MapCenter • service availability => MapCenter• OneWay metrics => RIPEncc test boxes

26

Some results

• from testbed sites to CERN– Pinger RTT: Average: 25ms– OWD: average: 9ms– OWL: average 0 to 0,3%– TCP Throughput : from 0 to 350Mb/S

27

PingER results

28

IPerfER Results

29

Schema and LDAP backend

• Grid applications/mw are able to access network monitoring metrics via LDAP services according to a defined LDAP schema.

• LDAP back end to make measurements visible through the Globus GIIS/GRIS system has been developed. – that fetch, or have pushed, the current metric information

from the local network monitoring data store. • R-GMA is tested as an alternative solution to Globus

MDShttp://ccwp7.in2p3.fr/mapcenter

30

Network Cost functions

Network metrics published in LDAP repositories are used by resource brokers and replica managers through network cost functions :

Time = networkCost (SE1, SE2, filesize)Computed from

1. GridFTP logs2. TCP throughput measurements (aggregated)3. RTT Measurements (aggregated)

31

EDG Network Cost Function

Network Element => Network COST function

32

EDG MapCenter Tool

– Connectivity of sites– Availability of services running over all sites involved – Efficient and flexible model to logically and graphically

represent all communities, organization, applications running over grids.

– MapCenter enables representation of any level of abstraction (national and international organizations, virtual organizations, application etc) needed by grid environments.

– http://ccwp7.in2p3.fr/mapcenter

33

Network and Transport Services

• QoS: – Demonstrate and build experience in use of

E2E diffserv services in Grid context– Feedback experiences to GEANT/DANTE,

NRENs and LANs• Transport

– High performance transport protocols – Reliable multicast protocols tests

34

QoS and Grid Applications

• 4 types of flows => Required Services– Bulk data transfer => Scavenger, AF– Interactive flows => AF, EF, ECN, others?– Real-time flows => EF, others?– Test traffic => Scavenger

35

QoS and Experimental work

• Routers configuration : WRR, DRR…• QBSS : in LAN and LFN (CERN-Caltech)• ECN and TCP over ECN• Alternative models:

– ABE, EDS, proportional DS• E2E Premium service for Medical

Applications

36

High Performance Transport

TCP mechanisms optimization– Tests of applicability of new mechanisms

• Use of QoS solutions– diminution of Packet Loss– Active queue management (WRED, ECN)– TCP over DiffServ (AF, EF, PDS, EDS…)

• Reliable Multicast Protocol– Test and deployement of JRMS and TRAM

37

Issues and perspectives

• Refine NetworkCost functions algorithms• Scheduling of active measurements• Sensor deployment scalability• Automatic metrics analysis• Network performances forecasting• QoS services E2E availability and effectiveness• Transport services deployment

38

Conclusion• In testbed0 and testbed1 the networking functionality was

here – IP technology: Best effort– GEANT has been deployed– A Performance Measurement Architecture developed

• In testbed 2 and testbed 3– Grid application performance optimization– End to end performance analysis– Test and provide enhanced network and transport services :

Premium, Scavenger, Multicast

39

WP7 and other collaborations

• WP7 and EU DataTAG collaboration– QoS service study and experiment– High Throughput study and experiment– Network monitoring and measurement

• GGF– GHPN research group

• Other European Grid projects (FR e-toile, UK e-science, INFN grid…)

40

For more information

• Consult our sites:– http://ccwp7.in2p3.fr– http://eu-datagrid.web.cern.ch