Service Differentiation and Grids Pascale Vicat-Blanc Primet Benjamin Gaidioz, Pierre Billiau, François Echantillac, Mathieu Goutelle, Fabien Chanussot

Service Differentiation and Grids

Pascale Vicat-Blanc Primet

Benjamin Gaidioz, Pierre Billiau, François Echantillac, Mathieu Goutelle, Fabien Chanussot

INRIA - ResoLIP Laboratory

Ecole Normale Supérieure de Lyon

France

[email protected]

P. Vicat-Blanc Primet 2

Outline

Requirements for E2E Service Differentiation in Grids

The EDS approach (DataTAG project)

The QoSinus approach (e-Toile/VTHD project)

Conclusion


Typology of Grid flows

Applications flows: Input & Output data Inter process communication messages (MPI, DSM, synchro…) Codes coupling Interactions Vizualizations Voice/Video in collaborative environments

Control flows: Grid environment deployment Applications deployment Control and Management of the Grid (middleware)

Monitoring, scheduling, loading, reporting, alarms…

All these flows share the same « network resource » and the same bottlenecks


Example: e-toile : Infrastructure

VTHD 2.5 à 10 Gb/s

VTHD 2.5 à 10 Gb/s

16 power PClinked by Myrinet

8 x 2 PC linked by SCI

2 Gb/s

1 Gb/s

1 Gb/s

1 Gb/s

1 Gb/s

1 Gb/s

1 G

b/s

250 PC en cluster

1 cluster Myrinet de 10 PCs1 cluster de 8 PCs

IRISARennes

CEASaclay

PRiSMVersailles

EDFClamart

ENSLyon

ID-IMAGGrenoble

SUNGrenoble

ID-IMAGGrenoble

SUNGrenoble

ENSLyon

ID-IMAGGrenoble

EDFClamart

ENSLyon

ID-IMAGGrenoble

PRiSMVersailles

EDFClamart

ENSLyon

ID-IMAGGrenoble

CEASaclay

PRiSMVersailles

EDFClamart

ENSLyon

ID-IMAGGrenoble

IRISARennes

CEASaclay

PRiSMVersailles

EDFClamart

ENSLyon

ID-IMAGGrenoble

Routeur actif

Serveur 8 processeurs

Machine SMP

Service de dépôt dedonnées IBP

Routeur actif

IBCP

Serveur 3* bipro

Serveur biproMP760

12 PC bipro

16 Sun Cobalt

1 cluster de PC

Experimental Testbed

6 PC bipro

16 PC bipro

Clusters de PC 1,9 Ghz

Serveur bipro1,2 Ghz

Production Testbed

CERN

EuropeUS


Grid Flows characteristics

Mice, Elephant, Lièvres et Tortues, … Throughput:

Rates: more than 9 orders of magnitude Few bytes for interactive traffic or control traffic To petabytes for bulk data transfer.

Delay: Very heterogeneous needs Some applications are very sensitive to latency (MPI; visu) Bulk Data Transfer delays have to be controlled

Reliability : Generally reliable (=> TCP) but some apps are loss tolerant (Astro)

Communication models: Point to point, point to multipoint, multipoint to point, multipoint to point Collectives operations, synchronisation barriers...


1. Tags and myocardiumautomatic extraction

2. Motion estimation

3. Quantification

time

Medical Images processing : Pipeline

tagged MRI sequences

From 20MB to 2GB/frame


How to control the performances?

Packet level (Network QoS) ~1 à 100s Mechanisms: classifiers, marquers et conditionners (routers) Models: IntServ, DiffServ, Corestateless,Proportional, EDS…

Round trip time level (E2E QoS) ~1 à 100 ms Congestion control and flow control (TCP, TFRC)

Session level s, mn, or hr Admission control, Resource reservation (RSVP), routing Load sharing, MPLS-TE, BoD

Long term Days, months... Provisionning, planification, oD


Explored Approaches (INRIA RESO)

Grid really need End to end QoS (bulk to MPI & vizual.)Packet differentiation is already there in IP equipments PQ, WFQ, CBQ, WRR, RED, WRED… Lot of issues with IS & Diffserv

Service differentiation at transport levelTwo approaches have been explored at INRIA: E&E : DataTAG (assumption: bottleneck is in access&LAN)

Relative IP packet differentiated forwarding Each connection manages its individual QoS End protocol has to be adapted (SlowStart or AIMD)

Edge to Edge : e-Toile (assumption : bottleneck is in WAN) An Independant API defined and integrated in mw to specify session QoS

goals QoSINUS as a Grid network Service Interact with the Grid Measurement Infrastructure


EDS approach


Equivalent Differentiated Service Model

Goal: Sharing the network resources (bottleneck) and control the E2E performances according to the application specific requirements

=> delay sensitive/ loss sensitive/rate sensitive…

Constraints: new PHB at IP level Differentiated forwarding services without pricing No admission control required. PHB definition restricted to local parameters (no layer violation) The transport layer has to integrate some adaptation

mechanisms to contribute to end to end performance control.


Equivalent Differentiated Services

Proportionality

Asymmetry (cf ABE)


Equivalent Differentiated Services

The EDS model defines an arbitrary number N of classes.Differentiation on delay and loss rate for each class.A class i gets a delay coef di and a loss rate coef li. These coef are constants. let i and j be two classes, the router schedules and drops their

packets so that there is a ratio di/dj between local queuing delays and li/lj between local loss rates.

In order to avoid having privileged classes, coefficients are set:

if di<dj then li>lj or

if di>dj then li<lj for all I in [1,N] and j in [1,N]


Adaptive Packet Marking: simple algorithm

Delay constraint Loss constraintSelected class

t t

lossdelay


AIMD EDS packet marking principle


Validation

EDS layer3 has been implemented in NS and in the Linux QoS kernel

EDS layer 4 has been implemented in SCTP via an adaptation of the AIMD algorithm in NS and Linux kernel and tested on a local emulated platform (NistNet) and on DataTAG link


Results for a mix of traffic (NS simulations)

Interactive trafficTransfer delay <60%

Real-Time trafficLatency constraint respect 2x >

Bulk transfer#timeout

EDS3/4


QoSINUS approach


Develop a Grid testbed: On the Very High Bandwidth experimental network (VTHD) “Active Grid Technology” (dynamicity of the grid)

Develop a middleware prototype: Programmable Network and communication Libraries NFSp & GXFER, MPI madeleine, MOME (DSM), Active network services (QoS, Mcast)

Perform tests with high end applications computing intensive, data intensive, network intensive validation of a “high performance grid” model targeting large scale numerical simulations.

e-Toile GRID project goals

Management Monitoring Security IHM

Globus 2.2 Duroc, GRAM MDS, GRIS/GIIS GSI RSL

e-Toile: Allocator, Loader SIC - SPAM GSI - authoriz. LDT - GUIDE


Programmable network INRIA RESO/LIP)

Active nodes TAMANOIR and IBP depot (Loci/UTK) deployed at the edge of VTHD

Gigabit supported with a TAN cluster (~1.3Gbits/s): TAN cluster = a front-end with back-ends for load balancing

ENS Lyon

fro

nta

le

Back-end 1

Back-end 2

Back-end N

…

VTHD

TAN CEA

Paris

TAN CERN

GenèveActive Flow

Actif flow

Receiver

Receiver


QoSINUS: E2E Performance controllability

QoSINUS: Quality of Service Negociate, Invoke, Use

Goals: E2E QoS : an interface « application » <-> « network » Application QoS objective: eg. E2E transfer delay Use Network QoS: DiffServ (packet prioritization) A programmable service (adapt API & algorithm)

QoSinus principles Specification and negociation of a SLS for a microflow by Grid

scheduler or application Programmable mapping of the QoS objective in a packet DSCP

in the first active node (use EF, AF, BE, LBE…). Dynamic Adaptation of packet marking based on measurement

results (network and flow).


QoS objectives programming


IPv6inIPv4

IPv6 over MPLS

Nancy

Lyon

Grenoble

Nice

Rouen

ParisAUB

ParisMSO

ParisSTL

Rennes

INRIANancy

INRIA Lyon

FTRDGrenoble

FTRDSophia

FTRDCaen

ENSTFTRD Issy

OpentransitConnectivité IPv6

FTRD Rennes

INRIA

FTRD Lannion

ENST BrBrest

ENST BrRennes

INRIASophia

INRIARennes

IPv6/IPv42.5Gbps

IPv6/IPv41 Gbps

IPv6/IPv4STM1/4

IPv6 sur tunnel

IPv4 seulement Routeurs de sites VTHD++/eToile

Juniper : M20/M40/T640

Cisco GSR 12000

TSR Avici

INT

HEGP

CEA

EDF

PRISM

IMAGSun

Eurecom

VTHD++ plate-forme

CHU

CERN

INRIA Grenoble

CERN


The VTHD backbone

“Really Very High Bandwidth”: provides 1Gb/s to 2Gb/sdirect access links Up to 4 x 2.5Gb/s in the core;

experimental network great availability Advanced services (Multicast, DiffServ, IPv6, MPLS,

GMPLS/UNI…)

connected to other research networks in EU through the DataTAG link (CERN in Geneva).

The VTHD network is deployed by France Telecom RNRT project VTHD and VTHD++


DiffServ in VTHD

Application

VTHD

EdgeEF (10%)

Streaming 30% (AF TCP)

AF-UDP (30%)

Best-effort 30%


Experimental results in e-Toile/ VTHD


Conclusion

Diffserv philosophie provides the mean to extend the IP forwarding model with scalable and easy to deploy service differentiation mechanisms.Difficult to avoid it if we want to control performances in GRIDS !Standard PHB are deployed (Premium, LBE) in EU NRNs

EDS or propDS provide simple and autonomous solutions to add differentiated services in an IP network. An incremental solution (for access links and LANs) Adaptive end to end transport protocols (packet marking in AIMD...)

QoSINUS exploit and control DiffServ ingress point transparently. Provides a simple and extensible API to application (XML) Provides a multi-domain and transparent solution


Future Work: Grid5000

Measure the gain obtained with challenging grid applications and grid infrastructures. Interaction with novel transport protocols for bulk transfersExplore deeply the multi-domain multi-service problemExplore the scalability of the EDS and QoSINUS approaches.

GRID5000 project : a large scale cluster interconnection in France With about 5000 processors aggregated With high performance DiffServ network links (RENATER) + high performance latency emulation tools. http://www.grid5000.org

Interconnected with GN2


More info

RESO project at INRIA:

http://www.ens-lyon.fr/LIP/RESO

e-Toile: http://www.urec.cnrs.fr/etoile

VTHD: http://www.vthd.org

GRID5000: http://www.grid5000.org

[email protected]

Documents

Service Differentiation and Grids Pascale Vicat-Blanc Primet Benjamin Gaidioz, Pierre Billiau, François Echantillac, Mathieu Goutelle, Fabien Chanussot