Tiziana Ferrari Diffserv deployment in the wide area: network design and testing1 Diffserv deployment in the wide area: network design and testing Tiziana

Tiziana Ferrari Diffserv deployment in the wide area: network design and testing 1

Diffserv deployment in the wide area: network design and testing

Tiziana FerrariItalian National Institute for Nuclear Physics

[email protected]

TF-TANT task forcehttp://www.dante.net/tf-tant/


Overview

• Why diffserv?

• Test programme

• testbed

• interim results:

– classification and marking

– policing

– scheduling

• QoS measurement

• services:

– VLL


Motivation

• Why diffserv?– independent of layer 2 technologies– interoperability:

> diffserv domains National Research Networks (NRN)

> diffserv region: NRN + 1 diffserv core domain> independent diffserv implementation in each

diffserv domain – traffic aggregation– scalability– transparency to the application


Motivation (cont)

– (some) benefits:> network layer:

• VLL service, a potential replacement of ATM permanent connections

• resource allocation on congested links • QoS support in/to NRN which are not ATM

based> application layer:

• QoS support for mission critical applications (RSVP - diffserv)

e.g.: good QoS support to client-server distributed applications thanks to traffic aggregation


Diffserv and the European network model

DS domain

DS domain

DS domainDS domain

Non DS capabledomain

MPLS

NRN

NRN

TEN-155

(marking)

Markingpolicing scheduling

shaping

DS domain

NRN

NRN

NRN


Overview of test programme

• Diffserv – DSCP: class selectors, EF, AF– test of basic QoS features in diffserv platforms– parameter tuning– interoperability– services: definition, implementation, validation through test

applications (video streaming, IP telephony, distributed applications)

• diffserv and intserv (end-to-end QoS)• diffserv and MPLS • QoS measurement • Policy management


Work accomplished (June 99-Oct 99)

• network configuration

• baseline performance testing

• marking

• policing:

– functionality, token bucket parameter tuning (normal and exceed burst size) with TCP traffic

– CB-WFQ (Class Based WFQ): traffic isolation

• scheduling:

– CB-WFQ: traffic isolation, VLL service in the wide area

– SCFQ (Self Clocked Fair Queuing): EF, AF and best-effort


Work accomplished (cont)

• QoS measurement (ongoing):

– parameters: one-way delay and one-way delay variation

> LAN and WAN (VLL service implementation and validation)

– traffic generators with GPS antennas

– NTP

• definition of test services:

– point to point virtual leased line

– one-to-many virtual leased line

– resource allocation on congested links

– better than best-effort

– services for remote instrumentation control (AF)


Diffserv test network

EPFL

CSELT

Uni. Bologna


LAN layout (example)


Equipment

• Test workstations (ATM, Ethernet, Fast Ethernet, Giga Ethernet interfaces)

• traffic generators (3 SmartBits, Netcom Systems) connected to GPS rx

• ATM switch(es) in each test sites

• 2 Cabletron Ethernet switches

• DS capable routers:

– CISCO C7200 and/or C7500

– IBM 2212 and IBM 2216 (in 5 sites, 2 routers per site)


Interim Results


Classification, marking and policing

• Multi-field classification: OK

• packet marking: OK

• testing of different “exceed actions”: OK

• policing (at a configurable rate):

– token bucket: parameter tuning for TCP traffic performance optimisation

– normal burst

– exceed burst


Policing: test of exceed actions

Throughput:SWITCH: 1.20 MbpsDANTE: 0.38 Mbps (drop effect)

drop

precedence 0


Policing: TCP burst tolerance

• Normal and exceed burst tuning with TCP traffic

• single and multiple TCP streams

optimum values are functions of the rate R at which

traffic is policed, in particular:

– normal burst = 0.5 sec * R

independently of the platform

– exceed burst = 2 * normal burst


Policing: TCP burst tolerance (cont)

Throughput of 1 TCP connection (Mbps)Exceed (bytes)Normal

(bytes) 32000 48000 64000 96000 12800032000 0.98 1.23 1.23 1.25 1.2548000 1.09 1.21 1.25 12564000 1.18 1.24 1.2596000 1.24 1.25128000 1.25

Table 4: throughput of 1 TCP connection for increasing values of the normal and exceed burst size

Aggregate throughput of 5 concurrent TCP connection (Mbps)Exceed (bytes)Normal

(bytes) 32000 48000 64000 96000 12800032000 1.26 1.26 1.25 1.26 1.2548000 1.25 1.26 1.25 12664000 1.25 1.27 1.2596000 1.26 1.26128000 1.25

Table 5: throughput of 5 TCP connections for increasing values of the normal and exceed burst size

• CISCO platforms• Target rate: 1.25 Mbps



• IBM platforms (IBM 2212)• Small TCP burst tolerance in a policer can completely starve a TCP stream. Token bucket depth is function of the rate at which traffic is policed tuning needed



Bucket size(bytes)

Test length(sec)

TCP Throughput(Kbps)

2200 Connection stalled ~ 04400 60 0.97

60 35.2120 74.7180 89.8240 88.5300 95.6360 98.2420 99.3

6600

480 100.68800 300 118.911000 300 124.413200 300 124.815400 300 126.017600 300 125.364000 300 125.0

EF behaviour aggregate of TCP traffic, target rate = 163 Kbps

(IBM 2212)


Policing, two-parameter TB:optimum burst size for different traffic rates

•Small TCP burst tolerance: it can completely starve a TCP stream•optimum token bucket size: function of the policing rate

IBM2212


Scheduling and policing:two-parameter TB and premium buffer size

Recommendation: token bucket size < premium buffer size(for low packet loss probability)


Scheduling: CB-WFQ

Scheduling implementation to provide minimum bandwidth

guarantees to classes

• tested on CISCO platforms - C7200, IOS 12.0(5)T -• class definition: based on class selectors for aggregation OK

• bandwidth allocation: no starvation, dynamic allocation in case of classes which are not fully deploying the capacity guaranteed to them OK

• traffic isolation between EF BA and best-effort: – several combinations of TCP and UDP traffic

– variable number of TCP streams (different aggregation degrees)


CB-WFQ: set-up


CB-WFQ: traffic isolation, 1 TCP stream

Throughput of a single TCP high priority connection:CB-WFQ rate = 1300 Kbps

Test site (tx) Neighbour (rx) Throughput (Kbps)GRNET 1250CERN

INFN 1210CERN 700GRNET

Uni. of Utrecht 710CERN 100

Uni. of Stuttgart 100INFN

Uni. of Utrecht 100SWITCH NA

Uni. of Stuttgart 880Uni of Twente

Uni. of Utrecht 880GRNET 1110

INFN 1160Uni of Utrecht

Uni. of Twente 1180Table 6: results of CAR and WFQ tests with 1 TCP high priority stream and UDP background traffic• Inconsistent results, for some sites TCP throughput < 1.25 Mbps

• different ATM burst tolerances on the WAN connections seem seem to be the cause traffic isolation on WAN connections can be an issue


CB-WFQ: traffic isolation, multiple TCP streams

Throughput of multiple high priority TCP connections:CB-WFQ rate = 1300 Kbps, UDP background traffic at 2 Mbps

Source site Destination site Number ofTCP streams

Aggregate applicationTCP throughput (Kbps)

1 1003 130

10 221

INFN CERN

20 6801 7003 1000

10 1180

GRNET CERN

20 1270

• Overall TCP performance of the behaviour aggregate improves when the aggregation degree increases• TCP performance of a single connection still low problem under investigation


Scheduling: Self Clocked Fair Queuing

• IBM 2212

• Policy = (traffic profile, validity period, diffserv action)

• Diffserv action = (type of marking, queue type, bandwidth)

• optimum traffic isolation (tests only with UDP)

Premium TCP traffic throughput, target rate: 163 KbpsTest

numberStreams BE

throughput(Kbps)

AssuredThroughput

(Kbps)

Premiumthroughput

(Kbps)

Total throughput(Kbps)

1 BE 1967.7 / / 1967.72 A / 1968.0 / 1968.03 P / / 159.8 159.84 BE + A 649.8 1367.0 / 2016.85 BE + P 1852.5 / 159.8 2012.36 A + P / 1852.0 159.8 2011.87 BE + A + P 617.8 1236.9 159.8 2014.6

Premium: 163.8 Kbps guaranteed (8% PPP link bw)Assured: 819.2 Kbps (40% PPP bw)


QoS measurement

• Resource allocation monitoring

– for resource allocation and network dimensioning

• performance measurement

– passive

– active (invasive traffic)

– for service validation

– to understand the effect on end-to-end performance of buffering in one router or in a chain


Parameters

• parameters of interest:

– one-way delay, requirements: clock synchronisation> GPS based synchronisation

• SmartBits (Netcom Systems)> NTP (Network Time Protocol)

• NTP client / server hierarchy– one-way delay variation

> deployment of inter packet gap at rx and tx

– packet loss probability

– throughput


QoS measurement: 1-way delay and instantaneous delay variation (LAN)

SmartBitstx rx

Goal : VLL service implementation and validationMetric: Type-P-One-way-Delay (RFC 2679), type-P-one-way-ipdv Variables:

• WFQ/SCFQ EF queue depth• packet size• EF queue “weight” • traffic volume


One-way delay: average vs packet size


Latency in time


ipdv (*without* best-effort)


ipdv (*with* best-effort)


Services

• Virtual leased line:– point to point– one to many

• capacity allocation on congested links (e.g. US links)

• better-than-best-effort (qualitative definition)


VLL


VLL: implementation


Future work

• Weighted Random Early Discard (WRED) testing

• completion of IBM testing

• study of TCP related issues: ACK colouring, TCP and UDP aggregations with AF

• interoperability testing (other future platforms: NORTEL, TELEBIT, TORRENT, Linux)

• service implementation and validation

• diffserv and MPLS

• diffserv and intserv

• policy management and bandwidth brokerage


To know more

• Activities of the TF-TANT task force:

– http://www.dante.net/tf-tant

• Diffserv test programme and interim test results:

– http://www.cnaf.infn.it/~ferrari/tfng/ds

• QoS measurement test programme:

– http://www.cnaf.infn.it/~ferrari/tfng/qosmon

Documents

Tiziana Ferrari Diffserv deployment in the wide area: network design and testing1 Diffserv deployment in the wide area: network design and testing Tiziana