CHAMELEON - A System for Adaptive QoS Provisioning

Rajesh Krishna Balan

• Internet has proliferated rapidly

Date Hosts 12/69 4 12/79 188 01/89 80,000 07/95 6,642,00007/95 8,200,00007/96 16,729,00007/97 26,053,00007/98 36,739,00007/99 56,218,00007/00 80,000,000

(projection)

• Strong Interest in carrying Voice traffic over the Internet - 250 billion conversation minutes by 2005

• Internet is designed for non-real time data communications - Assuring QoS is a problem.

Tele-X Tele-X

Trunk lines

Why there is no such problem in POTS?

• Statistical guarantees are provided

VoIPGateway

VoIPGateway

Internet

VoiceFlow

Technical challenges

• Lack of guarantees in terms of BW, packet loss, delay and jitter - Quality of Voice over Internet suffers. IP is only best effort!!

• QoS guarantees are primarily provided by two mechanisms:

• RSVP (Resource Reservation Protocol)• Priority Queue Mechanism

• DiffServ framework initiative under IETF and QBone initiative under Internet2 or Next Generation Internet hold promise.

Objectives

To search for and acquire various hardware and / or software tools which can generate application specific traffic which follow TCP dynamics.

To verify and evaluate the properties and capabilities of TCP tunnels.

To develop a complete system which uses TCP tunnels to provide QoS for various classes of traffic. This system should also be adaptive and thus be able to adjust itself to the traffic conditions on the network.

To develop a mechanism by which the system can be used as an efficient yet reliable link layer protocol for transmitting TCP data over lossy links.

Earth-110.10.1.1

Compaq Intel

Neptune10.10.5.2

LinuxIntel OEM

Mars10.10.0.4

LinuxIntel OEM

Venus10.10.0.3

LinuxIntel EtherExpress Pro 100B (Org)

Mercury

LinuxIntel EtherExpress Pro 100B (Org)

EarthTrunking Gateway

Linux

Linux

Jupiter

Saturn

TCP-Trunking Testbed Setup(Centre For Internet Research)

Rajesh & Boon Peng3rd September 1999

InternetEarth-2

10.10.2.1Compaq Intel

10.10.5.3

Compaq Intel

Uranus-110.10.5.3

Compaq Intel

ananda-r9137.132.89.91Compaq Intel

Saturn-210.10.4.1

Compaq Intel

Saturn-110.10.3.2

Compaq Intel

Jupiter-210.10.3.1

Compaq Intel

Jupiter-110.10.2.2

Compaq Intel

Pluto

Linux

Pluto-110.10.0.254Intel OEM

Pluto-210.10.1.254Intel OEM

Trunking GatewayLinux

Traffic Aggregator

Delay / Drop BoxLinux

Revised on 19th July 2000

KryptonTraffic Aggregator

Linux

Uranus-210.10.5.4

Compaq Intel

Krypton-110.10.4.254Intel OEM

Krypton-210.10.5.254Intel OEM

Uranus

10.10.0.2

Traffic Generators

Need for Realistic Traffic Generators

Mathematical Models (Poisson, Erlang, Exponential etc.)

Trace-based Models (collect usage statistics / data and replay etc.)

Ad-Hoc Models (models with no realistic component. E.g. models which just generate packets and send them out as fast as possible etc.)

Hardware Traffic Generators

Very expensive ( > $70,000 for a full system)

Used for stress testing production networks

Generally unable to generate application level traffic (telnet, HTTP etc.) which follow TCP dynamics (re-transmissions, window size negotiations etc.)

Ttcp Iperf Surge IPB AB Tcplib TG Netperf Netspec Rtpplay

Telnet NO NO NO NO NO YES YES NO YES NO

ftp (1)

(TCP bulk)

YES YES NO NO NO NO NO YES YES NO

ftp (2) (realistic) NO NO NO NO NO YES YES NO NO NO

HTTP NO NO YES YES YES NO NO NO YES NO

UDP voice (3)

(UDP bulk)

NO NO NO NO NO NO YES NO NO NO

UDP video (4)

(UDP bulk)

NO NO NO NO NO NO YES NO NO NO

RTP voice (5)

(realistic)

NO NO NO NO NO NO NO NO NO YES

RTP video (6)

(realistic)

NO NO NO NO NO NO NO NO NO YES

Works

properly?

YES YES YES YES YES Only a

library

(7)

NO

(8)

YES NO (9) YES

Software Traffic Generators

admission control

traffic management TCP tunnel

encapsulation mechanisms

incoming packet

outgoing encapsulated packet

Subsystem 1 Subsystem 2

TCP trunk

decapsulation mechanisms

incoming encapsulated

packet

original data packets sent to destination via normal routing

TCP tunnel

Subsystem 3

TCP trunkTCP tunnel

Principle of TCP Tunnels• TCP tunnels are virtual circuits through which all

traffic belonging to a class of application are aggregated.

• A single aggregated flow in a tunnel can be managed better than individual flows across a congested link.

Principle of TCP Tunnels (cont)• Provides statistical guarantees to classes of traffic (eg., voice

traffic should get at least 5% of the BW; Video traffic should get at least 10% of the BW, but no more than 20%).

• Guarantees can be assured because of the traffic distribution pattern (eg., all flows do not peak at the same time)

admission control

traffic management TCP trunk

encapsulation mechanisms

incoming packet

outgoing encapsulated packet

Subsystem 1 Subsystem 2

TCP trunk

decapsulation mechanisms

incoming encapsulated

packet

original data packets sent to destination via normal routing

TCP trunk

Subsystem 3

• Aggregation of traffic• Reliable link level protocol• Protection of flows• Congestion handling is moved from core

to edge of the network

Back Pressure Effect of TCP Tunnels

0

200

400

600

800

1000

1200

0

14.6

28.7 43

57.2

71.3

85.4

99.6

114

128

142

157

171

185

199

213

228

242

256

270

285

299

313

Time (sec)

Insta

nta

neou

s Q

ueu

e O

ccu

pan

cy

(packets

)

Core RouterMax=96

Limit=1000

Tunnel Router

Instantaneous Queue occupancies for core and tunnel routers using TCP tunnels

The ChameleonA complete solution that can help in the design of Voice over IP Gateways with QoS guarantees.

Traffic -Modeling of VoIP.

Research and development at the network protocol layer (TCP/IP etc.).

Started in co-operation with Harvard University.

ChameleonTele-X Tele-X

Designed to support multiple classes of traffic over the Internet with explicit minimum statistical guarantees.

INTERNET

End Machines

Border Router

Encapsulated aggregated data in the TCP tunnels

Border Router

Chameleon

Aggregation of Incoming Data into TCP Tunnels by Chameleon

End Machines

Chameleon Implementation

QoS routines are compiled into the Linux kernel to enable the various QoS providing buffer and scheduling mechanisms (like RED, CBQ etc.) in the Linux kernel.

User level program, TC used to activate the various buffer and scheduling mechanisms. This requires different parameters to be passed into the program TC.

Chameleon Implementation (2)Place packet into linked list

Remove packets from list and place into TCP tunnels after adding length of packet in front of packet

incoming packet

outgoing encapsulated packet

Firewall

1) Read length of pack2) Read packet

incoming encapsulated

packet

original data packets sent to destination via normal routing

TCP tunnels

Data Flow Diagram of TCP Tunnel Software

User spaceKernel space

libpcap

User space

Kernel spacePacket sent to TCP tunnel receiver as that is the destination address for TCP tunnel packets

1) Read IP destination of packet2) Write packet into raw socket.

Deployment

INTERNET

Border RouterRunning Chameleon

LANsLANs

Border RouterRunning Chameleon

INTERNET

Border Router

LANsLANs

Border Router Commodity Linux Machine running Chameleon

Commodity Linux Machine running Chameleon

Experimental Setup

Border Router Border RouterDelay/Error Box

End Machines End Machines

Virtual TCP tunnels between the 2 border routers

Logical setup of Experimental Testbed

Results - TCP Bulk Traffic

0

10

20

30

40

50

60

70

80

1 15

29

43

57

71

85

99

113

127

141

155

169

183

197

211

225

Time (s)

Th

rou

gh

pu

t (K

Byte

s/s

)

s

Without Chameleon

With Chameleon2

2

1

1

One 2Mbps UDP CBR streamOne 2Mbps UDP exponential on/off streamTen TTCP bulk transfer

Shows effectiveness in protecting TCP bulk traffic

Results - Web TrafficTraffic Mix• 100 web sources• 2 video streams

• Jitter is less• Substantial improvement in

End-to-end response time

% Improvement in RTT values with Chameleon

123.31%31.86%

923.82%

434.27%

0100200300400500600700800900

1000

Average RTT MaximumRTT

MinimumRTT

RTT Std

% Im

pro

ve

me

nt % Improvement with

Chameleon

Results - UDP Packet LossTraffic Mix• 10 TCP bulk sources• 2 video stream

• Packet loss is less

Data Bytes Received for UDP Traffic

96.70%

79.65%

0.00%10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%

100.00%

With Chameleon Without Chameleon

Data Bytes Received

Results - Adaptive NatureTraffic Mix• 10 TCP bulk sources• 2 UDP streams

200

400

600

800

1000

1200

1400

1 91 181 271 361 451 541 631 721 811 901 991

Time (s)

Th

rou

gh

pu

t (K

By

tes

/s)

UDP CBRUDP on/offTCP BulkTotal

1

12

2

3

3

4

4

Reliable Data Link Layer

Lossy / Wireless links are becoming increasingly common in todays networks

However, TCP performs poorly in cases where packets are lost due to corruption

Chameleon can be used to “isolate” the lossy links from the rest of the network

Versions of TCP optimised for lossy environments, like TCP HACK, can be used by the Chameleon

Reliable Data Link Layer (2)

Border Router Border RouterDelay/Error Box

End Machines End Machines

Virtual TCP tunnels between the 2 border routers

Logical Setup of Experimental Testbed for lossy link experiments

Lossy Link being simulated by Delay/Error Box

Reliable Data Link Layer - Results

0

10000

20000

30000

40000

50000

60000

70000

80000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Percentage Packet Loss (%)

Th

rou

gh

pu

t (K

Byte

s/s

)

sack

hack+sack

hack

new reno

1

2

34

1234

Throughput of Chameleon versus percentage packet loss for short latency (10 ms) link with random single packet

errors

Reliable Data Link Layer - Results (2)

0

5000

10000

15000

20000

25000

30000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Percentage Packet Loss (%)

Th

rou

gh

pu

t (K

Byte

s/s

)

sack

hack+sack

hack

new reno

1

2

3

4

1234

Throughput of Chameleon versus percentage packet loss for long latency (300 ms) link with random single packet errors

Reliable Data Link Layer - Results Burst Errors

12988

20083.8 19759.5

15595.8

3540.6

16436

19948.6

540.2

4642.8

0

5000

10000

15000

20000

25000

2 5 10Length of Burst Error (packets)

Thro

ughp

ut (K

B/s)

hack+sackhacksack

2% burst error

11016.211693.4

8531

5958.27696.4

1402.2

211.2745.4

8560.8

0

2000

4000

6000

8000

10000

12000

2 5 10Length of Burst Error (packets)

Thro

ughp

ut (K

B/s)

hack+sackhacksack

5% burst error

Reliable Data Link Layer - Results Burst Errors (2)

4854.4

0

1653.22011.2

788.6

3151.8

2109.41071

350

1000

2000

3000

4000

5000

6000

2 5 10Length of Burst Error (packets)

Thro

ughp

ut (K

B/s)

hack+sack

hack

sack

15% burst error

7193.85992.2

2772.44440.2

2931.61402.2

2949.4

142 360

2000

4000

6000

8000

10000

2 5 10Length of Burst Error (packets)

Thro

ughp

ut (K

B/s)

hack+sack

hack

sack

10% burst error

Future Work

Port Chameleon fully into kernel space

Test Chameleon in different network conditions

Optimise Chameleon for UDP traffic

Optimise Chameleon as a VoIP gateway

More work needs to be done on the use of the Chameleon as a reliable data link layer

Fully automate the Chameleon

Summary

Chameleon enables the provisioning of QoS for various classes of traffic in an easy yet effective mannerNo modifications to existing protocols and applications and is totally transparent to the end usersThe Chameleon is also adaptive and reacts seamlessly to changes in network bandwidth. It will strive to satisfy all statistical QoS contracts Applicable not only to voice, but other multimedia traffic as well.

Expertise in traffic generation and analysis: Poisson - telnet; Heavy-tail (Pareto) - web traffic; exponential on-off

- voice

Acknowledgements

A/P A. L. Ananda

Mr. Lee Boon Peng

TCP Trunk Project Group Dr. Lillykutty Jacob Dr Winston Seah Khoon Guan Renjish Kumar Liu Yong Xiang Chan Lee Lee Yeo Ann Kian

Questions?