An Application of Active Congestion Control in

Abdollah Aghassi

A thesis submitted in conformity with the requirernents for the degree of Master of Applied Science

Graduate Department of Electrical and Computer Engineering University of Toronto

O Copyright by Abdollah Aghassi 2000

National Library I*I of Canada Bibliothéque nationale du Canada

Acquisitions and Acquisitions et Bibliographie Services services bibliographiques

395 Wellington Street 395. rue Wellington Ottawa ON K1A ON4 Ottawa ON K i A ON4 Canada Canada

Vwr Ne Vmre Rfcimce

Our 6k Notre reterence

The author has granted a non- exclusive licence dowing the National Library of Canada to reproduce, loan, distribute or sel1 copies of this thesis in rnicrofonn, paper or electronic formats.

L'auteur a accordé une licence non exclusive permettant à la Bibliothèque nationale du Canada de reproduire, prêter, distribuer ou vendre des copies de cette thèse sous la fome de microfichelfilm, de reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation.

An Application of Active Congestion Control in TCP-Vegas

Abdoilah Aghassi

Master of Applied Science, 2000

Department of Electrical and Cornputer Engineering

University of Toronto

Abstract

TCP-Vegas, with its innovative congestion avoidance technique. achieves a better

link utilization than other TCP vaiants. However, it fails to fairly allocate the available

bandwidth arnong the usen. In particular, it penalizes the old connections in favor of the

new connections. Furthemore, TCP-Vegas has the potential to induce persistent

congestion in the network. These drawbacks al1 stem from the problem of over-

estimation of propagation delay.

To improve the faimess performance of TCP-Vegas, this thesis proposes a

probing mechmism based on the active networking approach. The mechanism requires

support from the active network nodes to immediately forward the probe packets. The

acknowledgments of probe packets at the sender provide a better estimate of the

propagation delays. Our simulation resuits show that our proposed active TCP-Vegas

fairly allocates the bandwidth arnong the connections. In addition, the better estimation of

propagation delays leads to reduced packet backiog in the buffer, and hence results in a

lower level of network congestion than the original TCP-Vegas.

Acknowledgement

1 would like to express my sincere gratitude and appreciation to my supervisor,

Professor A. Leon-Garcia, for his invaluable guidance, patience, advice and

encouragement throughout the course of my M.A.Sc. program and preparation of this

thesis.

1 would dso like to thank Professor 1. F. Blake, Professor J. Choe, Professor E.

Law and Professor S. W. Davies for their advice and comments dunng my final oral

presentation.

I wish to acknowledge in particular my fellows and friends in the

Communications group for their fruitful discussion and proof-reading of the thesis.

The financial support from ITRC is gratefully acknowledged.

Table of Contents

................................................................ 1 . INTRODUCTION 1 ................................................................................. 1.1 Motivation -3

.................................................................................... 1.2 Objective 4

...................................................................... 1.3 Thesis Organization S

........................... 2 . TCP CONGESTION CONTROL OVERVIE W 7

................................................................................ 2.1 Introduction .8

.......................................... 2.2 Problems with Current TCP Implementation 9

.......................................................... 2.3 Proactive Congestion Control 10

................................ 2.3.1 Vegas Congestion Avoidance Mechanism 12

............... 2.4 Major Drawback of Delay-Based Congestion Control Algorithrns 14

................................. 2.4.1 Errors in Round Trip Delay Mesurement 14

.................................................................... 2.5 TCP-Vegas Pro blems 15

...................................................................... 2.5.1 Unfairness 16

......................................................... 2.5.2 Persistent Congestion 31

...................... 3. ACTIVE NETWORKlNG and CONGESTION . 24

.............................................................................. 3.1 Introduction 2 5

............................................. 3.2 Explonng the Network by Smart Packets 28

................................. 3.2.1 In-Band and Out-Band Monitor Packets.. 30

............................ 3.3 Major Drawbacks of the Active Networking Appronch 32

..................................... 4 . ACTIVE CONGESTION CONTROL 34 ............................................................................... 4.1 Introduction 35

................................................................... 4.2 Enhanced TCKVegas 36

................................... 4.2.1 Faimess Evduation of Enhanced Vegas 36

........................................................ 4.3 Vegas and DRR at the Gatew ay 40

.............................................................................. 4.4 Active Vegas 41

...................................................... 4.4.1 Preliminary Simulations 43

................................... 4.4.2 Single Bottleneck Network Simulations 45

4.4.2.1 Round Trip Time Measurement ...................................... 51

4.4.2.2 Fairness Evaiuation .................................................... 52

...................................................... 4.4.2.3 Buffer Occupancy 54

4.4.2.4 S u m q of Simulation Results .................................... -55

.............................................. 4.4.3 Multi-hop Network Simulation 56

................................................................................. 5.1 Discussion 61

................................................................................ 5 -3 Draw backs 62

............................................................. 5.3 Future Research Directions 63

.................................................................. 5.4 Research Contributions 64

List of Figures

. ................................ Figure 2.1. General pattern of round trip time vs window size 8

Figure 2.2. Window algorithm of TCP.Vegas .................................................. 13

Figure 2.3. Network mode1 ........................................................................ 16

............................................. Figure 2.4. Simulation results when C = 0.5Mbps 19

................ Figure 2.5. Network topology after the new connection joins the network ..20

.................. Figure 2.6. Simulation results after the new connection joins the network 20

33 ......................................... Figure 2.7. Congestion window for each connection .-- Figure 3.1. Traditional passive network pmdigm ............................................ 2 5

Figure 3.2. Active network pattern ............................................................... 26

Figure 4.1: Connection goodputs and instantaneous queue lengths for enhanced Vegas.38

Figure 4.7: Connection podputs and congestion windows for old and new connections39

Figure 4.3. Packets in transit for active Vegas ................................................. .42

Figure 4.4. Priority queue at the gateway ........................................................ 42

Figure 4.5. Network mode1 ....................................................................... -43

................................. Figure 4.6. Connection goodputs for 3 active Vegas sources 44

................................. Figure 4.7. Connection goodputs for 4 active Vegas sources 45

Figure 4.8. Network topology .................................................................... -46 . .......................... Figure 4.9. Goodput vs connection index for 4 different scenarios 47

.................................. Figure 4.10. Buffer occupancies for the different scenarîos .48

........ Figure 4.1 1: Mean goodput of each connection under different Vegas algorithm A9

Figure 4.12. Buffer occupancy for original Vegas (infinite buffer size) ..................... 50

Figure 4.13. Connection BaseRï73 for ail scenarios ........................................... 51

. ................ Figure 4.14. Faimess vs bottieneck link bandwidth (link delay = 20msec) 53

Figure 4.15: Faimess vs . bottleneck Iink propagation delay (link capaci ty = 4Mbps) . . - 3 4

Figure 4.16. MuIti-hop network mode1 ........................................................... 56

........................ Figure 4.17. Average congestion windows for different connections 57

. Figure 4.18. Goodput vs connection index for active and original Vegas .................. 58

List of Tables

........................ Table 4.1 : Connection throughput for 3 different versions of Vegas 44

................... Table 4.2. Buffer occupancies for 4 different implementations of Vegas 55

.................................................... Table 4.3. Summary of simulation results - 3 5

................................. Table 4.4. BaseRTT estimation for active and original Vegas 59

vii

List of Acronyms

AACC

ACC

ATM

CARD

DAWA

DiffSew

DRR

ECN

FIFO

FrP

HTTP

IntServ

IF'

M'Tu

RED

RSVP

R n

SACK

TCP

Tri-S

VC

WFQ

WRR

Adaptive Admission Congestion Control

Active networking Congestion Control

Asynchronous Transfer Mode

Congestion Avoidance using Round-trip Delay

Defense Advanced Research Projects Agency

Differentiated Service

Deficit Round Robin

Explicit Congestion Notification

First In First Out

Fi le Transfer Protoco 1

HyperText Transfer Protocol

Integated Service

Internet Protocol

Maximum Transmission Unit

Random Early Detection

Resource ReserVation Protocol

Round Trip Time

Selective ACKnowledgement

Transmission Control Protocol

Slow Start and Search

Virtual Circuit

Weighted Fair Queuing

Weighted Round Robin

Chapter 1

Introduction

The existence of both new and old technologies in computer networks with

different link capacities leads to congestion. When the network load is light, the network

is able to provide the requested throughput. As the load increases beyond the network

capacity, the buffers start building up, and the throughput stops increasing. This is the

point at which congestion starts to occur. Any attempt to increase the load beyond this

point will result in further loss and less throughput. To rernedy these problems, a

congestion detectiordcontrol algorithm is required. Moreover an increaseldecrease

algorithm in the sender provides a rnechanism for preventing congestion after it is

detected in its early stages.

Congestion detectiodcontrol, which regulates the amount of traffïc, is a

distributed algorithm. It can be divided into two sub-algorithms: the link algorithm,

which is executed inside the network to detect congestion and retum this information to

the sender; and the source algorithm, which is executed at the edge of the network to

adjust the sending rate based on the congestion condition inside the network. In a simple

word, the role of congestion detectiodcontrol is to balance supply and demanà

As a supplier, when demand increases, the link algorithm increases the link price,

and as a consumer, when pnce increases, the source algorithm decreases demand.

Therefore in a limited time interval, the system will converge to a senling region. in the

congestion detectiodcontrol algorithm, the link pnce represents a congestion measure

inside the network,

How can the congestion measure be retumed to the sender? In other words, how

cm the source algorithm estimate the available supply? Ideally, one cm design the source

and link algorithms jointly so that they c m communicate and uansfer information back

and forth to move the level of congestion inside the network to a desirable operating

point. However in the current Intemet, there is no explicit feedback from the network.

and there is no direct communication between the source and link algorithms. There are

some implicit signs in the network that imply the congestion measure, the status of the

congestion in the network. In other words, source algorithm should estimate the availabie

bandwidth, the supply, with some implicit indicators. For example, packet drop is one of

these indicators, when the network gets congested packets start dropping. Another sign is

round trip delay. when the network gets congested, packets expenence longer round trip

delay. It is worth mentioning that packet drop and round trip delay correlate direc tly to

the type of link algorithms used.

Apart from providing other services, the transport layer is responsible for

regulating the traffic flow. This prevents a fast host from over-running the network

capacity, and plays the role of the source algorithm in the congestion control mechanism.

Transmission control protocol (TCP) is one of the main protocols in the transport layer of

the current Intemet. Many of the popular Intemet services, including HTTP, FïP,

TELNET, and so on rely on the services provided by TCP. There are several varieties of

TCP, including TCP Tahoe, Reno, New-Reno, SACK, and Vegas. Al1 TCP variants,

except Vegas, estimate the link price with packet dropl. Packet drop is an essential part of

their congestion control mechanisms, as stated previously, packet drop is a measure of

congestion and link pnce. Packet drop is equivaient to an increase in link pice that

should result in a decrease in demand, which is represented by the congestion window.

With a drop tail buffer however, Reno continually cycles between sinking into congestion

- - -

' We inchde the explicit congestion notification (ECN) rnarking in the category of packet &op.

2

and recovenng from it. To remove this condition from Reno and its variants. random

early detection (RED). as a link algorithm. tries to create an early packet &op to signal

the presence of the congestion to Reno. Packet &op results in an inefficient use of the

available bandwidth due to many retransmissions of the sarne packet after packet drops

occur. In addition. since Reno will eventually force al1 connections to have the same

congestion window size. it sipificantly discriminates against connections with longer

propagation delays [Il .

In contrast to Reno and its variants, TCP-Vegas estimates the level of congestion

by monitoring the difference between the rate it is expecting to see and the rate it is

actually sending. To cdculate the expected rate, actual rate, and their difference. Vegas

needs to measure the round trip delay, including the propagation delay. In other words,

round trip delay is a measure of congestion and link price. Increase in round trip delay

cm be assumed as an increase in link price. Again, based on supply and demand rule, the

congestion window, which indicates the demand, should be decreased.

1.1 Motivation It is clear that TCP-Vegas does not introduce any additional losses in the network

when it estimates the available bandwidth; therefore it utilizes bandwidth more efficiently

han Reno, due to fewer retransmissions. Experirnental results presented in [2] and [3]

show that Vegas achieves between 37 to 71 percent better throughput than Reno. With

this improvement in throughput, TCP-Vegas is a possible solution for the future Intemet.

But at lest two concems remained unsolved: does TCP-Vegas achieve a fair allocation

of resources; and does Vegas lead to persistent congestion. In particular, Vegas

discriminates against old connections when a new connection joins the network. The

main reason for this discrimination is in the calculation of expected rate.

The expected rate cm be achieved when there is no congestion in the network (no

queuing delay component in the round trip delay). When a new connection joins to an

aiready congested network, it includes the conesponding queuing delay in the round trip

delay measurement, which in turn results an error in its expected rate measurement.

Briefly stated, new connection over-estimates the expected rate. This results in an unfair

distribution of resources among old connections.

Furthemore, Vegas' strategy is to adjust the congestion window in an anempt to

keep the ciifference between expected and actual rates in a limited interval. This is

equivalent to keeping a small number of packets buffered in the routen along the path,

e.g., between 1 to 3 packets for each connection. The over-estimation of propagation

delay or minimum round trip delay causes an under-estimation of the number of packets

backlogged in the buffers; therefore it pushes more packets in the buffen, and creates

persistent congestion.

1.2 Objective We believe, for TCP-Vegas to be deployed in the future Intemet, it requires to be

modified to improve the unfaimess among different usen. To improve the overall

performance of Vegas, either the source algorithm should be changed or a compatible

link algonthm should be investigated or both.

The objective of this research is to improve the fairness performance of TCP-

Vejas. As mentioned previously, the unfair behavior of Vegas has its root in delay

measurement erron. The measurement of round trip delay that does not include my

queuing delay is our main concern.

The most straightfonvard kind of mechanism is to send a packet along the route,

being forwarded without experiencing any queuing delay as it goes; we cd1 these active

mechanisms. This monitor/controI packet experiences the minimum round trip delay.

Active mechanisms require support from the gateways; each gateway along the route

must forward the active packet with a high pnority order.

The aim of this research is to investigate the idea of active probing to improve the

performance of a transport protocol, in particular its faimess. To achieve this goal: first

the necessary changes to the source aigorithm, TCP-Vegas, should be added to enable the

protocol to send active packets; second the necessary active mechanisrn in the gateways,

link aigorithm, should be implemented.

To demonstrate the validity of our proposai, we want to implement our active

Vegas protocol in the ns-2 simulator 1151. To support fast fonvarding, a prionty drop tail

buffer will dso be irnplemented in the buffers. The specific types of measurements we

would like to perfom are:

r Faimess Perfionnance. Evaluate the fairness performance of active Vegas, and show

the level of improvement.

a Persistent Congestion. Demonstrate the effectiveness of active Vegas to decrease the

number of buffered packets compared to original Vegas.

In particular, the single bottleneck and multi-hop networks will be considered.

1.3 Thesis Organization The remainder of this thesis is arranged in the following order. In Chapter 2, an

overview of several methods of TCP congestion control will be presented, then important

problems of current TCP implementation, TCP-Reno, will be explained. nien we will

focus on congestion avoidance methods based on round trip delay, in particular. TCP-

Vegas. We will also show that over-estimation of round trip delay causes major problem

in TCP-Vegas that leads to an unfair treatrnent of old connections, and persistent

congestion in the network.

Since Our proposai is based on the probing mechanism with active packets, in

Chapter 3 we describe how a monitor/control packet c m improve the performance of a

large complex network by obtaining the information from the core of the network and

using this information to update the end point entity parameters, namely congestion

control parameten. In this chapter, we describe and enurnerate the advantages of using

monitor packets for exploring the network and surnmarize the applications of in-band and

out-band approaches.

In Chapter 4, we introduce Our own methodology, Active Vegas, to resolve the

unfaimess of TCP-Vegas. It begins by examining performance issues of the current

enhancements to the TCP-Vegas proposal to overcome its failure. Then, based on

simulation results, it discusses the limitations of these enhancements. To remedy these

limitations, this chapter presents a probing mechanism to estimate the minimum round

trip time accurately. We investigaie the effectiveness of active Vegas in providing a Fair

allocation of resources to al1 connections. Furthemore, we will show that active Vegas

backiogs fewer packets in the buffers compared to the original Vegas.

Finally, in Chapter 5, we discuss the advantages and disadvantages of our

proposal. There are also many other open issues conceming the congestion conuol

mechanism in TCP-Vegas including rerouting, and the format of the control packet that

wili be explained in the future research direction of this chapter. We finally enurnerate

the research contributions of this thesis.

Chapter 2

TCP Congestion Control Overview

The transmission control protocol (TCP) is an ad-hoc mechanism to provide a

highly reliable end-to-end protocol in a packet-switch network. To have a diable

transmission, a handshaking mechanism between the sender and the receiver is required.

WC793 is one of the early documents that specifies this handshaking mechanism. One of

the main duties of TCP protocol is flow control. That means, TCP controls the arnount of

data sent out by the source. In other words, the role of TCP apart from ensuring a reliable

delivery of packets, is to adapt the sending rate of the source to the capacity of the

network and the destination. Usually the part of the flow control mechanism that deals

with capacity inside the network is called congestion control.

In this chapter, an overview of several methods of TCP congestion control will be

given and then important problems of curent TCP implementation (TCP-Reno) will be

explained. We will focus mainly on congestion control avoidance rnethods based on

round trip delay, in paaicular, TCP-Vegas. We will also show that over-estimation of

round trip delay causes a major problem in TCP-Vegas that leads to an unfair treatrnent

of old connections, and persistent congestion in the network.

2.1 Introduction

In order to use network bandwidth efficiently, TCP controls its flow rate. To

control its flow rate, TCP needs to estimate the available bandwidth in the network using

a bandwidth estimation scheme. Bandwidth estimation is dificult since there is no

explicit feedback from the network and in most of the current implementations, they

assume the network is a black box with no explicit feedback. For this reason. they are

called implicit feedback schemes.

Flow control of TCP is based on a window mechanism, which consists of lirniting

the number of outstanding packets. The value of window size represents the source

throughput. Without flow control, the source can satunte one or several routers inside the

network, which c m introduce many losses and retransmission, resulting in a low goodput

(The goodput is the rate of useful data delivered by the network). Figure 2.1 shows

general patterns of round trip time (Rn) and goodput as a function of window size.

I I I

Delay I I I I

I l 1

W* Wzdow Size

Figure 2.1 General pattern of round trip time and goodput as a function of window size

Unfortunately, due to the heterogeneity of the network, the optimum value for W*

of the window size is not known beforehand. TCP must choose the optimum vaiue of the

window size based on its estimation of the current available bandwidth, How can the

available bandwidth inside the network be estimated? Usually, there are some implicit

signs in the network that implies the congestion measure, the status of the congestion in

the network. For example, packet drop is one of these signs, when the network gets

congested packets start dropping. Another sign is round trip delay, when the network gets

congested, packets experience longer round trip delay. Furthemore, packet drop and

round trip delay depend on the queue management and service scheduling in the network.

In other words, how packets are treated in the network highly affects the packet drop and

round trip delay. Therefore, bandwidth estimation by the TCP mechanism is tightiy

correlated to the link rnechanism that provides these implicit signs. For example, random

early detection (RED) as a link mechanism provides necesSay &op signds for the TCP-

Reno to estimate the available bandwidth. It's worth noting that bandwidth estimation in

Reno is based on packet &op. Now imagine a TCP mechanism that does not use packet

drop as a congestion measure, therefore it will not be compatible with a link algorithm

like RED. An important point is the compatibility between the source aigorithm. TCP

mechanism in this study, and the link algorithm, scheduling and queue management.

Regarding the available bandwidth estimation methods, TCP congestion control

schemes c m be classified into two categories; proactive and reactive methods [4].

Reactive methods come into play after congestion is developed. Based on the feedback

from the congested network, they will take some action that drives the network out of this

state. The Jacobson congestion avoidance mechanism [SI is one of the reactive methods

that is activated mainly by packet drop which happens &ter the congestion occurs.

Proactive methods come into play before congestion is developed to avoid, and reduce

the possibility of network congestion. Proactive methods try to prevent congestion from

occumng. Admission control and traffic shaping can be categorized in proactive

methods. However, in this research, we are not to compare the performance of proactive

and reactive methods. It is certainly clear that proactive methods perform better than

reactive methods; when a network is congested, packets are Iost, bandwidth is wasted,

and more.

2.2 Problems with Current TCP Implementation

The cument implementation of TCP, TCP-Reno, has no mechanism to detect the

congestion just at the beginning because it needs to create losses to estimate the available

bandwidth of the connection [6]. In other words, Reno's mechanism to estimate the

available bandwidth is continually increasing the window size, using up buffers dong the

connection's path, until it makes the network congested and packets are lost. Therefore,

Reno is continually congesting the netwotk and creating its own losses to estimate the

current available bandwidth. In this regard, TCPReno is reactive rather than proactive.

The Jacobson method uses the additive increase and multiplicative decrease

algorithm as a measure to probe the network and estirnate the available bandwidth. Due

to the nature of adàitive increase and multiplicative decrease of window size, there is an

inherent osciIIation problem in the Jacobson method and consequently in TCP-Reno. In

other words, in Reno, the window size oscillates between the maximum window size

(maximum window size which does not introduce any &op) and half the maximum

window size. Consequently, oscillation in window size leads to oscillation in queue

length and this oscillation in queue length leads to oscilIation in round trip delay of

packets. (This is clear in Figure 2.1). Furthemore, variation in the queue length also

results in larger delay jitter (we are not interested in the jitter variation, since TCP is not

carrying real time traffic that are sensitive to high jitter variation). In addition, such a

probing mechanism is very inefficient to utilize the available bandwidth because of many

retransmissions of the same packet after the packet drop occurs.

Another major problem with the Jacobson method is its unfairness to the users

with longer round trip delays. In other words, TCP-Reno significantly discriminates

against connections with longer propagation delays [l]. This is again created by inherent

characteristics of the additive increase and multiplicative decrease algorithm. Because it

will eventually force al1 of the users to have the sarne window size, this will result in

lower throughputs for connections with larger propagation delays.

2.3 Proactive Congestion Control

The are several proposed approaches to proactive congestion control. Since

congestion control consists of several phases (e.g. slow start, congestion avoidance, . . .), by being proactive we are focusing mainly on the congestion avoidance phase. One of

the methods which is reported in the literature is based on round trip delay variation. In

other words, instead of approaching to the optimal vaiue of the window size by

introducing packet drops in the network, with the additive increase and multiplicative

decrease operation, it attempts to use round trip delay variation to estimate the available

bandwidth.

Jain et al [7], [8] propose a mechanisrn called "Congestion Avoidance using

Round-trip Delay" or C . W . The whok idea in this proposd is Y e r y simple. Back to the

Figure 7.1, when the source window size increases. the queues in each node will build up,

and consequently, the round trip time will increase. Since increasing the window size is

equivalent to an increase in load, it means that increase in load will eventually lead to an

increase in round trip delay. In other words, optimum value of window size Leads to

maximum throughput and minimum round trip delay. They define power as the ratio of

throughput and round trip delay. By maximizing power they try to achieve the optimum

value for window size. Moreover, for window controlled networks, the connection

throughput is approximately equal to the ratio of the number of outstanding packets and

the round trip delay, which is the ratio of window size and round trip delay. If the

network is not congested and load is low, changes in window size does not result in

changes in round trip delay. On the other hand, when the network is congested, any small

increase in window size will lead to a large increase in round trip delay. In this way, the

gradient of the round trip delay venus the window size c m be used as a good measure to

estirnate available bandwidth and to adjust window size, very close to the optimal vaiue

of window size.

Wang and Crowcroft [9] propose another method called "Slow Start and Search"

or Tri-S. The main idea in Tri-S is simila. to CARD, which is the flattening of the

throughput when the network is congested. M - S uses the current throughput gradient to

search for the optimum value of window size instead of using delay gradient as in CARD.

In addition, sirnilar to CARD, Tri-S is still dependent on round trip delay rneasurement to

calculate the changes in the throughput dopes.

Brakmo [2],[10] proposes another congestion avoidance mechanisrn based on

round trip delay measurement called TCP-Vegas. In the next part we explain the

congestion avoidance mechanisrn of Vegas in detail.

23.1 Vegas Congestion Avoidance Mechanism

The main idea behind congestion avoidance mechanism in Vegas is to measure

and control the arnount of data in transit by looking at changes in throughput. When the

network is not congested, the actual thmughput will be very close to the expected

throughput. In contrat, when the network is congested, the actual throughput will be

srnalier than the expected throughpur. Vegas uses rhis Merence in throughput as a

measure for window adjustment instead of using changes in throughput slope as in Tri-S.

In the congestion avoidance phase, the connection is not being overfiowed.

therefore, the expected throughput is given by:

Erpected = WindowSize / BaseRTï

where WindowSize is the size of the current congestion window, which in the congestion

avoidance phase with no packet &op, is equd to the number of packets in transit and

BaseRTT is the minimum of al1 measured round trip delays. The maximum value for

expec ted

throughput is achieved when there is no congestion in the network. That means, a good

estimate of BaseRTT should not include any queuing delay and theoretically it will be

equal to twice the propagation delay plus transmission delay.

The calculation of actual throughput cm be done by recording two values; the

sending time for a specific segment and the number of packets sent until the acknowledge

of that specific segment arrives and dividing the number of packets transrnitted by the

actud round trip delay of that specific segment. The maximum number of segments in

transit c m not be greater than window size, and it cannot be far less than window size if

no packet drops. Therefore, we c m approximate the actual throughput as:

Actual = WindowSize / rn

where rn is the actuai round trip delay of a segment.

Based on the value of diff = (Erpected - ActuaZ)BaseRîT, the congestion window will

increase by one segment, decrease by one segment, or remain unchanged. diff will be the

estimated backlogged queue size, if BaseRTT is measured accurately and does not

include any queuing delay, (no &op in the congestion avoidance phase). Vegas tries to

keep this backlogged queue size in a lirnited interval. The forrnal algorithm for window

update is as follows:

diff < a

diff >

o th envise

The window algorithm can also be illustrated pphically as shown in Figure 2.2.

Size

Figure 2.2 Window aigorithm of TCP-Vegas

As mentioned before, without congesting the network, Vegas tries to be proactive

by estimation of available bandwidth. This mechanism is totally different from Reno's

congestion avoidance mechanism. That means, it does not need to introduce losses in the

network to estimate the available bandwidth; therefore, it utilizes the bandwidth more

efficiently than TCP-Reno. The main result reported in [2] is that Vegas cm achieve

between 37 and 71 percent better throughput than Reno. Here we are not trying to

evaluate those results or compare the performance of Vegas and Reno. For more detailed

compatison and evaluation see [3], [Il], [U].

2.4 Major Drawback of Delay-Based Congestion Control Algorithms

A congestion conwl scheme is to maintain the balance of demand and supply in

the network to achieve two basic objectives: link capacity is fully utilized and resources

are shared arnong users in a fair manner.

Although TCP-Vegas is vying to propose a better utilization of the link capacity

and less variation of delay jitter, it suffers from a main disadvantage. This drawback is

not only limited to TCP-Vegas but aiso to almost al1 of the delay-based approaches to

congestion control. Since delay-based congestion control algorithms are heavily

dependent on measurement and estimation of round trip delays, any over or under-

estimation of round trip delays will affect their performances and degnde the proposed

improvements.

2.4.1 Errors in Round Trip Delay Measurement

In this part we are trying to investigate why and when there will be an error in

round trip delay estimation. End-to-end delay can be divided into severai components:

transmission delay, propagation delay, packet processing delay, and queuing delay.

Transmission delay is the time required for a packet to be sent physically. In other words,

it is the tirne ffom the first bit in a packet is sent until the last bit in the packet is sent.

Packet processing delay is the time required for a router to forward a packet. Packet

processing delay is the time ciifference between picking up the packet and sending it on

the attached link. Packet processing delay does not include the queuing delay.

In our theoretical analysis. we assume that the processing time is negligible and

packet processing delay can be ignored. Furthermore, we assume that the capacities of

the access links are large enough that they do not incur any transmission delay. Hence,

the transmission delay for each packet will be 1IC second, where C is equal to the

bottieneck link bandwidth (packet per second). Therefore, round trip delay can be divided

into two parts: propagation delay and queuing delay. Each of these two parts c m create

errors in the round trip delay measurement.

Since the topology of the network changes from time to time due to the failures of

the links and changing the route of the segments (rerouting), propagation delay for an

already established connection could vary because of the changes in the path of the

connection. Consequently, propagation delay at time t is not necessarily equal to

propagation delay at time t + A ( A > O). Therefore rerouting causes the propagation

delay to be changed and therefore may create error in round trip delay. In Our resemh.

we did not take the possibility of route changes into consideration.

Moreover, since the state of the congestion in the network changes a11 the time,

the queue length in every router will Vary. Depending on the trafic characteristic. the

queuing delay will also Vary, which may create error in round trip delay.

2.5 TCP Vegas Problems

As we saw in the previous section. there could be an error in round trip delay

estimation due to the rerouting and the congestion. Since B a s e R n has an important role

in the congestion detection and the congestion avoidance mechanisms of TCP-Vegas, i t is

crucial to have a good estimate of BaseRZT.

In the following sections, we are trying to show how any inaccurate estimation of

round trip delay c m lead to an unfair share of the total bandwidth (131. Furthermore, the

overestimation of the propagation delay pushes up buffer backlogs that leads to

persistent congestion 1141.

2.5.1 Unfairness In order to show that Vegas fails to achieve the second goal of congestion control,

i.e. fairness, we define a scenario and show both anaiytically and experimentally that

TCP-Vegas treats old connections unfairly.

Figure 2.3 Network model

The network model that we will use in this scenario is described in Figure 2.3.

The mode1 consists of three sources (Sl. S2, S3), three destination (Dl , D2. D3), two

intermediate routers (RI, FU) and links connecting those elements. Suppose the

bottleneck link is between R1 and R2 (al1 of other queues are empty so there is no

queuing delay associated with thrm). Therefore, the only place that queue may built up

is on RI . Let the bottleneck link capacity be equal to C (segmentkecond) and

propagation delay between S , and D, be equal to r, (sec). Suppose source S 1 and source

S2 establish a connection to destination Dl and destination D2 respectively at the same

time and source S3 establishes its connection to destination D3 afienvards. Queue

management in R1 is Drop Tai1 with enough bufTer space, which introduces no &op.

Before connection 3 joins. we can deduce fiom Equation (2.1) that S 1 and S2

congestion windows reach to a settling point such that a < diff' c P (i = 1,Z). Therefore

we have:

Windo wSi'e, Windo wSize, a < ( BaseRTI;

) . BaseRTT, < p , rtt,

Since there is no queuing delay involved before connection 3 joins, BaseRlT, would be 1

measured accurately as 2 ri + - (assuming high access link bandwidth). Consequently C

we have:

From Equation (7.3), we cm observe one of the advantages of Vegas over Reno

that is not to force al1 of the connections to have the same value of window size. As

mentioned before, Reno eventuaily forces al1 of the users to have the same window size

which will result in lower throughputs for connections with Iarger propagation delays.

Here, in Vegas, as we see, window size is approximately proportional to propagation

delay and longer propagation connections will have larger window sizes which results in

a fair capacity allocation to ail of the connections. But there is an important requirement

that the measurement of BaseRTT. which the window mechanism of Vegas is based on,

should be perfect. Unfortunately, Vegas fails to achieve this requirement most of the

time, particularly when a connection is established in an aiready congested network.

To see how Vegas treats old connections when a new connection is established,

suppose connection 3 joins the network. Let and ,u4Pef be the mean queue size in

router R1 before and after connection 3 joins the network respectively. For connection I

and 2, Equation (2.3) still holds and mi (i = 1,2) cm be calculated as:

and

Since connection 3 docs not have

will set i ts BaseRTT to:

and,

a ieasonabb approximation of

(2.5b)

propagation delay, it

Therefore after reaching to the stable condition, the size of congestion window for

connection 3 will be:

If we consider the same propagation delay for ail of the connections ( 5 = r , for i = 1.2,3)

and compare Equations (2.8) and (2.5b), we will observe that the congestion window for

connection 3 converges to a larger value than the congestion windows for connections 1

and 2. Thus the new connection cm achieve higher bandwidth than the old connections.

To confimi the above analysis, we run two simulations with Network Simulator

(ns-2) 1151. Network model, which used in the first simulation, was shown in Figure 2.3.

The following parameters were used: dl link bandwidths and propagation delays except

bottleneck link are lMbps and Smsec respectively, the bottleneck link propagation delay

is 20msec, and Vegas congestion control parameters are set to: a = I and f l = 3. Packet

sizes for d l connections are the sarne and are equal to IKbyte. Each simulation lasted for

60 seconds. Sources SI and S2 establish their connections after 1 second and connection

3 joins to the network after 10 seconds.

Figure 2.4 Simulation results when C = 0.5 Mbps. (a) Connections goodputs

(b) P kfi- and Pu1,er

As i t can be seen From Figure U a , after comection 3 joins the network, the

goodputs of the previously established connections will decrease very much, compared to

the higher goodput of the last co~ection. Figure 2.Jb shows 4 packets backlogged in the

bottleneck queue before comection 3 joins. The reason behind this is that Vegas atternpts

to keep at least a packets but no more than ,û packets in the queues. Due to queuing

delay corresponding to this backlog, last comection over-estimates BaseRTT which

leads to opening its window size more than its share. This scenario will happen every

time a new comection joins the network and it may lead to a persistent congestion. In

order to validate, another simulation was run based on Figure 2.5. in this scenario we let

another connection, connection 4, join the network after 20 seconds. Al1 other parameters

are the sarne as the previous simulation. As it was expected, later comection

establishment of S4 leads to a very unfair allocation of bandwidth particularly for

connections 1 and 2.

L

DI

S2 i

Bottfeneck link

u - Figure 2.5 Network topology after new connection joins the network

Figure 2.6 Simulation result when C=O.j Mbps. (a) Connections goodputs (b) Instantaneous queue lenght

As it was expected, the settling point of the congestion window for new connection is

larger than their shares and the reason is essentially due to over-estimation of baseRïT

2.5.2 Persistent Congestion

It can be obsewed from Figure 2.6b that every time a new TCP-Veys connection

joins the network, the queue size jumps to a much higher value which is not desirable and

is the major drawback of TCP-Vegas. Corresponding to the parameter s e thg in our

simulation, we expect that each Vegas connection keeps between one to three packets in

the gateway. For example, in Figure 2.4b, four packets build up in the buffer, i.e. two

packets for each connection; before the third connection joins the networks. Therefore,

we expect that after connection 3 joins the network, the queue size should not be more

thm seven packets in the buffer. However, Figure 2.4b shows that the number of packets

in the buffer is 11 packets. The sarne result can also be seen in Figure 1.6b. Before

connection 3 joins the network, the queue size is four packets, d e r connection 3 joins,

the queue size is 11 packets and after connection 4 joins the network, the queue size is 15

packets. In other words, the queue length in the buffer is more than it should be and will

lead to a persistent congestion in the network.

Again, the reason behind this phenornenon is error in minimum round trip delay

estimation, which causes the new connections to keep more packets in the buffer. We

leamed that Vegas estimates the backlog in the buffer by d a (diff = (Expected - Actiial)BaseRTT) and tries to keep this backlog between a and 8. In the following, we

will show that any over-estimation of BaseRTT results in an under-estimation of the

backlog. In other words, the over-estimation of BaseRïT leads to greater backlog. Let us

assume that Vegas measures its BaseRïT as:

BaseRTT = real-minRïT + E (2.9)

where, real-min-MT is the minimum possible value for round trip delay when there is

no congestion, and E ~ S the error corresponding the queuing delay. Consequently. the

backlog will be:

WindowSize * real - min- R ï T WindowSize * E diff = ( WindowSize - - (2.10)

?tt rn

WindorvSize * E Equation (2.10) shows an under-estimation of in the backlog. It means,

rtt

WindowSire * E Vegas will keep the actud baclog between a + and ,û +

rtt

WindowSize * E

rtt

To confirm this with simulation result, let us go back to Figure 2.6b. Because of

error in Basertt for comection 3, the number of backlogged packets for c o ~ e c t i o n 3 is 6

packets. To calculate the backlog estimation error. the congestion window size is sho~vn

in Figure 2.7. The congestion window, 6. real - m i n R T , and rtt for comection 3 are

equal to 6 packet. 64msec. 76msec, and 140msec respectively. Therefore,

F.VindotvSix * E will be approximately equal to 3 packets. which exactly matches with

rtt

our simulation result shown in Figure 2.6b.

+ Seriesl i Series2 A Series3 x Series4

Figure 2.7 Congestion window for each comection

From the above analysis and observation, it can be concluded that over-estimation

of BaseRn aiso creates a large number of packets backlogged in the buffer that will lead

to a persistent congestion. Our expectation from Vegas is to maintain at most ,8 packets

per each source in the buffer. Therefore, with n active connections. queue len,gh should

be no more than n p , but our simulation results show that the current implementation of

Vegas accumulates more than n /3 packets in the buffer. Increasing the backlog queue

length with an increase in the number of active connections is still one of the weak points

of Vegas performance.

Chapter 3

Active Networking and Congestion

Active networking is a revolutionary networking technology that offers a change

in the traditional network paradigm by introducing programmability into the basic

infrastructure of the network. Explonng and justifying this change in the network

architecture requires understanding the advantages, disadvantages and trade-offs of the

new approach. There are two active network architectures: The programmable router

approach, and the capsule approach. In the first part of this chapter, we bnefl y overview

the active network concept and enurnerate the tmdeoff between flexibility and efficiency.

We note that congestion control is a prime candidate for active networking, since it is

especiall y an intra-network event and is potentiail y far removed from the applications. In

the second part of this chapter, we describe how smart packets improve the performance

of large complex network by obtaining the information from the core of the network and

using this information to update the end point entity parameters, namely congestion

control panmeters. In the 1 s t part of this chapter, the major drawbacks of the active

network approach will be explained.

3.1 Introduction

Current networks face many problems, e-g., difficulty of integrating new

technologies, poor performance. a ~ d difficulty of accomrnodating new services in the

existing architecturai model. There are two main bottlenecks to the solutions of these

problems: the Intemet itself is hard to change, and current network evolution is limited by

the speed of standardization. The standardization process, which needs to be 3greed on by

a large number of vendors, manufacturen, and in general with different point of views, is

lengthy; thus it delays implementation of new services on a large scale. As a

consequence, the introduction of new network services at this level is very slow. For

example, there was a span of five or more years from the time that RSVP was proposed

to the time that it was deployed, even in a very limited manner. Active networking is to

be a remedy to these problems.

Although the idea of using active or smart packets is not new and has been

previously proposed in the literature [4] [16] 1171 [Ml, but the concept of active

networking resulted from the discussion within the DARPA in 1994 and 1995 on the

future direction of networking systems [19].

Traditional networks transparently transfer data from one end to the other, i.e., the

network is not sensitive to the packets it carries and they are transported between the end

systems without any modification. In other words. the packets are static and the network

nodes simply use routing information to forward the packets', and al1 packets are ueated

sirnilarly.

router/svPitches Pay load Only forwarding basai Payload

Figure 3.1 Traditional passive network paradigm

' With the implementation of DiffServ and InrServ and other complicated services. the network does not oniy forward the packets anymore.

Active networks introduce programmability into the basic infrastructure of the

network. Each user cm inject programs contained in messages into the network that is

capable of perfonning computation and manipulation on behalf of the user.

If we think of the iP header in the traditionai networks as input data to a virtual

machine, we c m imagine a packet in the active network containing rnethods as well as

data payload. In other words, the user specifies how the packet should be processed and

fowarded. Figure 3.2 descn hes the active network paradiem.

FI Pay load

Active Execution Environment

Pay load

Infrastructure

Figure 3.2 Active network pattern

The network c m be "active" in two ways:

Discrete or programmable switch/router; Switches perform computation on the user

data flowing through them. The injection of the methods into the network is separated

from the processing of messages.

Integrated or capsule; Lndividuals inject programs into the network. Each packet

contains both a program fragment and a piece of data.

These two active network architectures seem to be at the opposite extreme ends of the

design space. The programmable router approach introduces a little fiexibility into the

network, while rnaintaining the efficiency of packet forwarding. In contrast, the capsule

approach achieves almost full flexibility, but packet forwarding eficiency is sacrificed.

Hence there is a trade-off between programmability and efficiency. There is also a

paradox in using active network especially in the capsule approach. From one side, the

execution of each packet program increases the arnount of processing at the network. As

a consequence, the packets expenence longer per-hop latency. However, active networks

are expected to improve the network performance such as per-hop latency, and end-to-

end delay. Although longer per-hop latency and other effects of active networking may

appear to degrade the network performance, they may actudly lead to a better overail

performance, perhaps because of reducing requested bandwidth. creating less congestion

and etc.

The important point is that with active networks, the network services are to

support the user's req~irenents. Althugh some application of actir-, nerworlÿng h n c

shown great promise, the advantages of the active network approach are still not obvious.

More concrete results are needed to mswer the following questions:

What existing important network services will be significantly improved by active

network?

What useful new services will be enabled?

The answen to these questions can faIl under two categories:

Some network services can be irnproved or supponed using information that is only

available inside the network. In other words, the network may have information that

is required by the applications and by the users to fully optimize performance.

Especially the timely use of that information can significantly improve the

performance. Examples of these services are:

- Accessing accurate information about the curent state of congestion in the network

[20].

- Locating the network hot spots in the network where requests for an object are high,

in order to perform caching in network nodes [2 11.

- Moving management decision points closer to the node being managed [22].

- Finding the location of packet Ioss in rnulticast distribution trees 1231.

To improve the performance of some other services, the network needs the

infornation from the users and the applications. Examples of diis type of information

include:

- How important is the user data unit? In many applications, retransmission of a lost

packet degrades the network performance.

- Usability of the cached data, instead of real-time data.

In summary, the performance of active network should be evaluated in terms of

application-specific metrics.

3.2 Exploring the Network by Smart Packets

As it was mentioned in the introduction, congestion control and resource

management are prime candidates for active networking, since they are specifically an

intra-network events and are potentially fa. removcd from the applications. There are two

approaches of appl ying active networking technology to congestion control and resource

management: exploring the state of the network by the reports from entities inside the

network, and leming the network condition by some speciai packets generated by the

end points. These two methods are tightly coupled because without the active entities

inside the network sending rnonitor packets are not beneficial.

Having some congestion control policy implemented by entities inside the

network can have several advantages [24]:

Easy collection of necessary information from strategically placed network entities

for making congestion control decision.

Removing the corresponding round trip delay in accessing the relevant information

and hence faster response to changes.

End points are definitely responsible to eventually enforce the congestion control

decision. In contrast with traditional congestion control mechanisms, entities in the

network can give an explicit feedback to the end points.

Examples of active processing in the entities inside the network can be found in several

current congestion control mechanisms. We describe severai proposais based on active

entities inside the network.

Early packet drop, proposed in [17] and [Ml, is a type of active network

processing that improves the performance of ATM network. In an ATM network, packets

are fragmented into cells. Transmitting cells of a fragmented packet that one of its cells

has already been lost in the switch is wasting the bandwidth. In [17], Romanow and

noyd propose an aggressive dropping mechanism to drop entire packets when the

congestion threshold is reached. Along the same line, Floyd and Jacobson in [18] propose

randorn early detection (RED) mechanism to randomly drop or mark the packet when the

average queue length exceeds a threshold. In both proposals, no smart packets are used,

instead, active entities implicitly return the congestion status to the end points.

Active networking congestion control (ACC) proposed in [25] is a system that

uses entities in each router to immediately rex t to congestion. . C C replaces the

conventionai end-to-end congestion control to a new hop-by-hop congestion control. In

time of congestion, each router reports the state of congestion back to the end point and

actively slows down the transmission rate as if the end point instantly reduces its sending

rate. This rnechanism has two advantages: first, the end points are informed of the

congestion much earlier because now, the feedback delay is less than a round trip delay;

and second. the intemal network nodes beyond the congested router do not expenence the

congestion. This can irnprove the aggregate throughput in the core network.

The second method for exploring the network is, generating some type of special

packets that are treated in a particular way in the network to gather the information about

the state of the network. These special packets have been called with several names in the

literature: sample packet, smart packet. control packet, rnonitor packet, scout packet,

sampling packet, probe packet, and so on. We use these names interchangeably. This

approacch has the extra advantage that it's easier to implement and more compatible with

the current Intemet architecture. It can also be integrated with other congestion control

means, resulting in an overall superior performance. The idea of using smart packets in a

computer network is not new and has already been proposed severai times to improve the

network performance. We describe severd proposals based on the smart packet idea. It

should be indicated that some active entities are still required inside the network to

update the information carried by smart packets.

In 1161, Kent and Mogul enurnerate the disadvantages of packet fragmentation in

the gateway. To avoid fragmentation, they propose a probing mechanism to discover the

minimum MTU (maximum transmission unit) dong the path the datagram will follow. In

this mechanism, the packet collects MTU information dong its route. We use a similar

rnechanism in our proposal to improve the performance of TCP-Vegas. As mentioned

earlier. sending probe packets without having active entities inside the network will be

futile, therefore each router must update the probe according to the MTU of the hop it is

about to take. Furthemore, the collaboration between the two end systems is also

required, since the MTU information should be retumed to the sender.

In 141, Hass proposes the mechanism, adaptive admission congestion control

(AACC), to monitor the congestion status of the network by using a periodic transmission

of time-stamped sampling packets through the network. The moni todsample packet

travels through the network like the regular packet. At the other end, the receiver

calculates the virtual delay, which is the difference between the time-stamp and the value

of the local clock. This virtual delûy is retumed to the sender via the acknowledgement

packet. Then the sender compares the value of virtual delay for the last two consecutive

acknowledgment packets that is the difference between actual delays. Based on the

difference, the sender adjusts its congestion parameters. The larger difference means

higher level of congestion in the network.

In [26], Hicks et al describe a routing method using scout packet which is very

similar to the ATM VC-switching. Usen periodically send scout packets that explore the

network to search for a better route, and direct the flow of the regular data packets.

Furthermore, the scout packets comrnunicate with each other by leaving part of their

states at each router they visited. These states consist of some metric to represent the best

path so far. If the scout packet reaches the destination with the best metric, it announces

the best path by retuming aiong its route to the sender and installing an entry in the

routing table of each router in the path. As a result, the regular data packets are sent aiong

the path just set up by the reuiming scout packets.

3.2.1 In-Band and Out-Band Monitor Packets In the previous section, we describe and enumerate the advantages of using

monitor packets to explore the network. In this section, we see how these monitor packets

will be processed inside the network. There are two approaches for monitor packets: in-

band and out-band.

In the in-band approach, monitor packets are treated the same way as regular data

packets. They receive the same level of forwarding, routing and other services as other

data packets. Therefore, both the data and monitor packets are subject to being delayed

or even dropped when the node is congested. This might be an advantage or

disadvantage, depending on the application. For exarnple, if we are interested in

rneasuring and analyzing the packet delay distribution, the in-band approach will be a

suitable choice. Furthermore, by examining the arriva1 of the monitor packets, one c m

aiso obtain a sarnple of the current congestion level of the data-forwarding path, as

proposed in [4]. On the other hand, if propagation delay mesurement is important, using

the in-band approach may not be appropriate. In a simple word, for control algorithms

that are sensitive to queuing delay, the in-band approach cannot be used. However, this

approach is simple to implemented, and has less overhead compared to the out-band

method.

In the out-band approach, the control packets receive special, usually better

services, than the regular data traffic, at the intenor node. It requires a special

mangement in the forwarding module of an interior node. such as quick forwarding of

the out-band monitor packets. For example, in resource reservation protocol (RSVP) [27].

the PATH. RECV. and other control messages receive special treatment to reserve

resources dong a path inside the network. The special treatment for the PATH message

is, fonvarding them if it originates from a new source to request a reservation. With out-

band packets, it's possible to schedule different flows through the interior node so that

any kind of specific requirement can be met.

As mentioned above, the out-band approach with a priority scheduling

mechanism is more appropriate for a round-trip-delay sensitive control algorithm.

However, there could be a scaling problem at the active network node since potentially

there may be millions of monitor packets waiting to receive special treatment. This

problem also affects the in-band monitor packets when they collect interior node status

for management and other purposes. We will summarize the major drawbacks of the

active network approach in the next section.

3.3 Major Drawbacks of the Active Networking

Approach

It has been demonstrated that active networking technology c m provide a better

performance and enable new services, but what is the price for these improvements?

What are the problerns created by this new prcposal? mat does this new tcchnology

degrade to improve its strength? In this section, we briefly answer to these questions.

Processing power

Perhaps the primary role of the active network is communication, from which

resource management and congestion control are trernendous win for the active network

approach. In addition, active networks allow the computation to move from the edge of

the network into the intenor node where the important information can be timely

accessed. Computation inside the network requires extra processing power in the network

intenor nodes, which is not neressary for a simple forwarding in the traditional network.

Therefore, the major drawback of the active networking approach is the extra processing

overhead introduces. In the active network paradigrn, not only is bandwidth vaiuable, but

also the processing power has a price. Najafi and Leon-Garcia [28] propose a mode1 to

include the cost of processing as well as bandwidth. The new cost mode1 can be used to

find the optimum openting point for trade-off between the processing power and the

bandwidth.

Safety and security

Another difficulty that is concerned with the active network approach, is safety

and security. As an illustration, if an active node is to allow any arbitrary C program to be

executed in the node, then it will be a vulnerable point for malicious attacks.

Irnplementing safety and security in active network requires run-time checking

mechanism. For example, in the srnart packet paradigrn, the authentication of the packet

should be checked to venfy the identity of the user, and Iimit the resource access. This in

turn adds more burdens on the processing requirements.

Scaiability

Scalability is another bottieneck in active network technology. Generally

speaking, active network is not scalable because there may be huge number of active

packets from thousands of users requesting special services and treatments in the active

nodes. For example, the integrated services IP with RSVP control mechanism is not

scaiable because a path needs to set up for every flow, and keeping the state of each flow

inside the network is riot possible when the nrimber of usen is e x ~ n e l y largc.

Interoperability

To be deployed in the cumnt IP network, the active networking technology

should perform in a heterogeneous environment. To achieve this goal, active network

proposals should minimize the amount of global agreement required.

To conclude this chapter, it's worth mentioning that in this research we focus

mainly on the idea of smart/monitor/scout packets to explore the network state; the other

extreme of active network technology. capsule approach. is not our main interest. In

Chapter 4. we describe Our proposed active TCP-Vegas that takes advantage of the

monitor packets to obtain better estimation of the propagation delay of the connection

path.

Chapter 4

Active Congestion Control

The main conclusion frorn chapter 2 is that TCP-Vegas is unfair to old

connections, and creates persistent congestion in the buffer. The reason for these

drawbacks is basically in over-estimation of minimum round trip delay. In the previous

chapten, it has been shown that the over-estimation of the propagation delay for a given

connection leads to an increase in its rate, and results in unfairness. Furthemore, over-

estimation causes more packets to be accumulated in the buffer.

Perhaps, the main weakness of TCP-Vegas is its propagation delay estimation.

Hasegawa et al [13] propose enhanced TCP-Vegas to overcome this failure. In the first

part of this chapter, we will review their proposal and based on the simulation results,

major weakness of this method will be descnbed.

Other researchers [29] as well as Hasegawa [30] propose DRR (Deficit Round

Robin) as a rnechanism to increase the faimess of TCP-Vegas. In the second part of this

chapter, we bnefly overview this proposal and enumerate its problem.

In the third part of this chapter, we introduce Our own methodology, Active

Vegas, to resolve the unfaimess of TCP-Vegas.

4.1 Introduction

Flow control, which regulates the amount of traff~c, is a distributed algorithm. It

can be divided into two parts: the link algorithm that is executed inside the network, and

the source algorithm that is executed at the e d p of the network. These two algonthms are

tightly related to each other, and they provide the required framework for Bow control.

Flow sontrol can be saan as a generai mechanism ùiat covers most of conuol mec hanisms

that maintain good system performance. For example. in a differentiated services

network, there are control mechanisms on the boundary of the network, e.g., rnarking,

policing and shaping (source algonthms), and there are other control mechanisms at the

core network, e.g.. queuing and scheduling (link algorithms).

The main duties of the link algonthm are congestion detection inside the network

and retuming this information to the source algorithm. It is worth mentioning that in the

curent Intemet, there is no explicit way for the link algorithm and the source algorithm

to communicate with each other. After receiving congestion information from the link

algonthm implicitly, the source algorithm adjusts the rate of sending trdtic. As an

illustration, FIFO, RED, and RED with ECN are link algorithms. RED detects congestion

by setting a threshold on the average queue size and informs the sender implicitly by

dropping a packet randornly when average queue size is greater than the threshold. RED

with ECN detects congestion the same way as RED does, but instead of dropping the

packet randomly, it marks the packet randomly to explicitly inform the sender.

As far as the source algorithm is concerned, there are many TCP fiow control

algorithms to adjust the rate at which traffic is injected into the network in response to

congestion feedback notification from the link algorithm. To that extent, coordination

between the link alprithm and source algorithm is very crucial. For example, RED and

TCP- Reno cm work together because Reno rate adjustment is based on packet drop and

RED detects congestion and notifies Reno by packet drop. In contrast, RED is not

compatible with TCP-Vegas because Vegas rate adjusment is not only based on packet

drop but also on round trip measurement. Therefore, to improve the overall performance

of Vegas, either source algorithm should be changed or compatible link algorithm should

be investigated or both.

4.2 Enhanced TCP-Vegas

Probably the most innovative feature of TCP-Vegas is its congestion detection

mechanism during congestion avoidance, which leads to a stable window size that does

not oscillate enormously like TCP-Reno. Hasegawa [13] claims that the condition of

unchanging window sizes results in most of TCP-Vegas drawbacks, particularly in

uniaimess to old connecuons and creaung persistent congestion inside the network.

Hasegawa [13] proposes an algorithm, Enhanced TCP-Vegas, to intentionally force the

window size to oscillate around the appropriate value and prevent the convergence of the

window size to a stable point. Their aigorithm is based on setting a equd to fl in TCP-

Vegas. Intuitively, the idea came from the fact that TCP-Reno and its ancestor. TCP-

Tahoe. do not penalize the old connections after a new connection is established.

Therefore, making the window size performance closer to TCP-Reno can be helpful.

Referring to Equation (U), making a equd to fi introduces more oscillation in

congestion window size. As previousl y mentioned, Vegas tries to keep the backlogged

queue size in a lirnited interval, namel y between a and /? . When the bac klogged queue

size is less than a , congestion window size is increased by one and when backlogged

queue size is greater than /? , congestion window size is decreased by one. Having set a

equal to /? , the congestion window will oscillate around the convergence point.

4.2.1 Fairness Evaluation of Enhanced Vegas

Enhanced Vegas proposai only focused on the simple network topology, a single-

hop network with onIy two connections where they are homogenous. The homogenous

connections have the same propagation delays. Back to Equation (2.8), it can be seen that

any over-estimation of the propagation deiay for a connection still leads to an increase in

window size. In the next part, we will see from the simulation results that the congestion

window size oscillation, proposed by Hasegawa, is not able to cornpensate this effect.

Here, we conducted two sets of simulation with different values of a and 1 to

evaluate the performance of the Enhanced Vegas algorithm. The network mode1 is

shown in Figure 2.3. The following parameters were used: al1 link bandwidths and

propagation delays ,except bottleneck link's, are l Mbps and jms, respectively. The

bottleneck link capacity and propagation delay are OSMbps and 20msec respectively.

Packet sizes for al1 connections are the sarne and are equal to 1Kbyte. Each simulation

lastcd for 50 sscondst Sourzzs SI and S2 eatablinli tlieir co~mections aiter 1 second and

connection 3 joins to the network d e r 10 seconds. As it can be seen in Figure 4.1, the

allocated bandwidth to the new connection is almost three times greater than the old

connections. Furthemore, before the third co~ec t ion joins the network, the congestion

windows of the old connections oscillate, but after that the queue size reaches steady state

and there will be no longer enough oscillation in the congestion window sizes. This is

unexpected from the algorithm.

9 - c

I m r i c

L a m p

I C I U l i I

i

Figure 4.1 Comection goodputs and instantaneous queue lengths for enhanced Vegas

Why is congestion window not oscillating in our simulation result? The reason is

that; when diff is equal to a . the congestion window remains unchanged (Equation

(2.1)). In order to remove the stability from window size, this condition should be

removed from window update algorithm (the equali ty condition should be included either

in the increasing or in the decreasing sections). To that extent. the enhanced Vegas

algorithm has been changed and implemented in ns-2 as follows:

WindowSize + 1 diff < a PVindo rvSke =

Part (b) of the above simulation (a = p = 2) is repeated with this new algorithm. based

on Equation (44 , and the results are shown in Figure 4.2. Despite the oscillation in the

window size, due to over-estimation of BaseRn, the last connection still receives more

bandwidth than its fair share. Even worse, the actual throughput is slightly smaller than

that of original Vegas implementation. with different threshold on window size, due to

window size oscillation.

Lower throughput for enhanced Vegas can be expected because its oscillating

window size behavior is very similar to TCP-Reno, which has less goodput than Vegas.

The main conclusion from this section is that oscilIation in window size cannot

compensate the over-estimation of BaseRIT.

nt t . 1

î *#) ' a m 8 '. r . . . . *

8 . C .. Y . . . . A . .

r r n m c

S L I P - .- - *¶m-

1mPC

4-+

*m.-

1 P P -

I I l L P . -

Figure 4.2 Comection goodputs and congestion windows for old and new connections

4.3 Vegas and DRR at the Gateway

Perhaps one of the simplest link algorithms to provide isolation between flows to

achieve faimess in terms of throughput is weighted round robin (WRR). The buffer at the

router is divided into multiple queues and each queue is served in round robin order. To

be as fair as possible, the WRR server must know the average packet size before hand.

This is not pncticai when packet size is variable. To eliminate this drawback, deficit

round robin (Dm) [31] keeps the state of each queue to measure the "deficit" or past

unfairness by use of a deficit counter for each queue. The queuing discipline

corresponding to each queue is FIFû. However, a new algorithm is proposed in [30],

DRR+, that combines RED and DRR. In other words, the queue management in each

logical queue is RED instead of FIFO.

Deploying Vegas, and choosing a better scheduling algorithm at the gateway

might be a remedy for the intrinsic unfairness of Vegas. In this way, by using DRR

gateways to provide the necessary isolation among different usen, the unfaimess of

Vegas c m be resolved [29], [30]. To provide perfect isolation arnong the users. a separate

logical queue will be required for each user, which is alrnost impossible for a large

nurnber of different users. In other words, as the number of connections increases, it is

impossible that DRR can assign a separate logical queue to each connection: therefore,

severai connections should share one logical DRR queue. Eventually, we return to the

sarne problem of original Vegas and a shared FIF0 buffer, which leads to an unfair

allocation of bandwidth among those connections sharing a single queue. Moreover, to

provide fairness, DRR needs to keep the state of each logical queue. This will incur ex tn

processing time overhead. In a nutshell, using complicated scheduling algorithm, e.g.,

weighted fair queuing (WFQ) and DRR, usually ends up in scalability problems. Using

DRR+ at the gateway does not help to irnprove the fairness problem of Vegas because of

the incompatibility between RED and TCP-Vegas.

4.4 Active Vegas

As repeated several times in the literature. Vegas has achieved a better throughput

than TCP-Reno, but it fails to keep the faimess arnong different usen, and creates

persistent congestion. In the previous sections, we enumerated recent proposals to

overcome this problem and explained their weaknesses. We believe, for TCP-Vegas to be

deployed in the future Intemet, it requires to be modified to improve the unfaimess

arnong different users. In this section, we introduce Our own algorithm, called "Active

Vegas".

From the previous simulations and proposals, it c m be observed that a better

estimation of the BaseRïT is highly required. On the other hand, using a separate queue

for each flow to achieve faimess is not practical from the point of view of scalability. A

better estimation of propagation delay can be achieved if queuing delay can be removed

from the BaseRïT rneasurement.

To remove the delay corresponding to the backlog queue in the gateway from the

measurement, we propose a new algorithm. It is based on the active network approach

described in chapter 3, to combine a modification in Vegas and a prionty queuing in the

gateway. In other words. our proposal is based on out-band monitor packets. Active

extension of Vegas allows us to employ an strategy in which the gateway is enhanced

dynarnically to improve the faimess of the Vegas. In the default implementation, there is

a single Drop Tai1 queue shared by al1 of the connections. By receiving an activelmonitor

packet. the gateway changes its queuing mechanism to a priority queue. To provide an

out-band path, it creates a fast route for the monitor packet. The buffer at the gateway is

logically divided into two separate queues, one with higher priority than that of the other.

In this way, high priority packets will be forwarded without expenencing any queuing

delay at the gateway. Every time TCP-Vegas enters the congestion avoidance phase

(congestion window greater than slow start threshold), it transmits the first packet as a

high priority packet, an out-band urgent packet, to monitor the network. Its

acknowledgement can provide a better estimation of propagation delay. Further, we

assume no delayed acknowledgment to be installed at the receiver. The expedited packet

can be either a regular data packet or a separate monitor/control packet. If the regular data

packet is used to perform monitoring operation, it has the advantage of reducing the

overhead of consuming extra bandwidth for sending separate monitor packet. This cm

mitigate active Vegas scheme overhead. Moreover, if the high pnority packet is lost, we

presume that active Vegas is intelligent enough to send another monitor packet until it

receives its acknowledgement. Since there is no packet loss in our simulation, we did not

implement this mechanism in our simulation.

Slow Start Congestion Avoidance A

Sender -1

Figure 4.3 Packets in transit for active Vegas

As mentioned in chapter 2, every tirne a new comection is established. due to

delay corresponding to backlog queue. last connection over-estimates BuseRTT, which

results in opening its window size more than its share. Now with Active Vegas, there will

be no or very minor queuing delay for the marked expedited packets. Therefore, there

will be very small over-estimation in BaseRn, and more faimess will be achieved.

Flow from al1 connections 1

I I . . . I I

D m Low priority I I I 1 1 1 1 1 0 0 1 - - 1 - - - 1 1

Figure 4.4 Prionty queue at the gateway

Figure 4.4 shows our dynamic queue modification in response to monitor active

Vegas packet. It is also possible to send urgent packet penodically to check if there is any

change in the routing path. We will discuss this in ~e future research section. In the next

part, we run the fint preliminary simulations to see the algonthm problems and

weaknesses.

Simulations

In the firçt part of this section, we use the network topology shown in Figure 4.5 to

examine the performance of Active Vegas. Instead of drop tail, a pnonty packet

scheduling has been implemented in R1 and al1 sources send priority packets when they

enter to congestion avoidance phase after a slow start. It is worth mentioning that cr and

/3 are set to 1 to 3, respectively. The propagation delay for al1 links are jmsec and

packet size is 1 Kbyte. The first and second connections are established afier one second

and the third c o ~ e c t i o n joins the network after I O seconds.

4

S1 Dl

Figure 4.5 Network mode1

It can be seen kom Figure 4.6 that the faimess has been achieved in this simple

simulation. However, extensive simulation is required to evaluate this proposal.

Therefore, as a first step, in the next part we run the simulation with 4 Active Vegas

sources.

Figure 4.6 Comection goodputs for 3 Active Vegas sources

In Table 4.1, three algorithms (original Vegas. enhanced Vegas and active Vegas) are

compared from the point of view of achieved average goodput (packet/second, packet

size = 1KByte). The network mode1 and simulation scenario are the same for three

algorithms.

Table 4.1 Comection throughput for 3 different version of Vegas

1'' comection znd comection zrd connection

In the next part, a forth connection joins the network after 20 seconds. Recall

from the simulation in Chapter 2 (Figure 2.6) that later connection establishment of S4

leads to a very unfair allocation of bandwidth, particularly for co~ect ions 1 and 2. The

results with active Vegas simulation are s h o w in Figure 4.7. As it cm be observed, not

only faimess has been irnproved , the number of packets backiogged in the gateway

buffer has also been decreased to 12 packets, the number that we expected. Each

connection keeps at most 3 packets in the bufEer: so, 12 packet is the maximum number

of packets in the buffer.

Original Vegas a 4 . P = 3

20.0667 20.1333

25.52

Enhanced ~ e g & a = p = 2

19.93 3 19.1

26.86

Active vegas a 4, f l =3

2 1.65 22.45 20.8

Figure 4.7 C o ~ e c t i o n goodputs and instantaneous queue size for 4 Active Vegas sources

it is worth mentioning that buffer size is large enough to avoid any drop. From

Figure 4.7, it can be seen that a better faimess has been achieved with Active Vegas

compared to original Vegas im plementation.

4.4.2 Single Bottleneck Network Simulations

In this section. we simulate Our proposa1 with many senders and receiven. The

simple network mode1 that was simulated in this section is the one descrîbed in Figure

4.8. I t consists of a single bottleneck s h e d by 30 point-to-point connections. For all

cases, the shared link has 1Mbps bandwidth. Also, a lMbps access link connects each

hosts to the router such that a flow is setup fiom host Sx to host Dx and is identified by a

flow Idx. Packet size for dl of the connections is 1Kbyte. The Propagation delay of the

access links and the shared link are 5ms and 20ms, respectively. Six sets of simulations

were conducted under this topology with different TCP algorithms at senders and

different packet scheduling algorithms at the router. To compare some of the simulation

results, different buffer sizes were also investigated. Successive connections (i= 1. . . . .30) joins the network every one second. Al1 of the senders tum off after 100 seconds and

each simulation lasted for 1 10 seconds.

Bandwidth = lMbps

Access Links Bandwidth = LMbps

Delay = Srnsec

Figure 4.8 Network topology

In the fint four mns of simulation, buffer size is set to 100 packets and the following

scenarios are simulated.

Scenario 1: Active Vegas, a = 1 and #? = 3

Scenario 3: Active Vegas, a = ,î3 = 2

Scenario 3: Original Vegas with Drop Tail, a = L and /? = 3

Scenario 4: Original Vegas with Drop Tail, a = /? = 2

In Figure 4.9, the average and the standard deviation of each connection goodput

are shown. Each figure corresponds to a different scenario. The x-axis and y-axis

represent the index of the co~ec t ion and the corresponding goodput, respectively. The

stars and vertical bar; correspond to the mean and the standard deviation of the rate

oscillation. The number of received packets in one second is calculated as the connection

goodput.

V

O 5 10 15 20 25 30 connedon index

l5 i+ alpha- 1 ,beta=3,PropTail 4.

alpha. b a . 2 . PQ

L O 5 10 15 20 25 30

connectior. index

-" O 5 10 15 20 25 30

connedon index

-" O 5 10 15 20 25 30

connection index

Figure 4.9 Goodput vs. connection index for 4 different scenarios

Figure 4.9 illustrates that active Vegas provides a better faimess among the

connections. Setting equal values for a and ,û gives an even better fairness than two

different values for a and 8. However, it produces more variation in the window size

and hence in the throughput. On the other hand, cr # ,û will converge to a settiing point

very quickly and therefore less oscillation in the window size and the throughput.

The performance of original Vegas with Drop Tai1 at the gateway is very poor and

surpnsingly, it does not behave as we expected. From the previous simulations, we

expect that original Vegas with Drop Tai1 at the gateway allocates more bandwidth to the

new connections than old connections. This is not the case in the current simulation. To

find out the reason, buffer occupancies of active Vegas and original Vegas with Drop

Tai1 are shown in Figure 4.10.

Figm 4.10 Buffer occupancies for the different scenarios, buffer size 100 packets

Active Vegas not only improves the faimess but also leads to less nurnber of

packets logged in the buffer. Figure 4.10 answers our question about the behavior of

original Vegas and Drop Tai1 buffer at the gateway. Potentially, originai Vegas induces

persistent congestion and creates large queues at the gateway up to the point that Reno

like behavior kicks in; buffer overflows, packet drops, and the window size is halved.

Since 100-packet buffer size is not enough for original Vegas, after 30 sec, packets start

to dm:, and the perfommce hecomes sirnilx to Reno.

As mentioned in [ I l ] , [2], if the buffer is not suficiently large. equilibrium

cannot be achieved, loss cannot be avoided, and Vegas will go back to Reno. It's worth

mentioning that creating less congestion is another advantage of active Vegas compared

to original Vegas.

To have a better view of the two algorithms in the same conditions. we repeated

the last two scenarios of the simulation with enough buffer size to avoid any loss. which

resulted in no packet drop. To remove al1 of the start up transient fiom the calculations.

mean goodputs for al1 connections under different scenarios were calculated in the last 60

seconds of simulation and they are shown in Figure 4.11.

+Active Vegas, alphacbeta -t- Active Vegas, alpha=beta -t- Original Vegas, alpha~beta +Original Vegas, alpha=beta

Figure 4.1 1 Mean goodput of each connection under different Vegas algonthms

Figure 4.1 1 confirmed our previous observation that active Vegas with equal a and /9 results in a fair allocation among the connections. To investigate the nurnber of

backlogged packets for active Vegas and original Vegas, we conducted the simulation

again for original Vegas with infinite buffer size. Buffer occupancy for original Vegas is

show in Figure 4.12. Comparing Figure 4.12 and Figure 4.10, it c m be observed that

active Vegas creates Iess congestion in the buffer compared to original Vegas.

Figure 4.12 Buffer occupancy for original Vegas (infinite buffer size)

Afier al1 connections join the network, original Vegas back logged 170 packets in

the buffer while active Vegas back log queue size is about 65 packets. This is two and a

haif times less than original Vegas. One of the features of TCP-Vegas is that Vegas

attempts to keep at l e s t a packets but no more than f l packets in the queues. Active

Vegas, by its monitor packets and prionty queuing, cm provide and support this feature

with a good approximation. Frorn the simulation with 30 connections, we expect the

backlog queue size to be Iess than 90 packets ( f l = 3). This was satisfied by active

Vegas. In contrast, as it cm be observed, original Vegas cannot achieve this feature and

creates more congestion in the network.

4.4.2.1 Round Trip Time Measurement

To compare BaseRn measurement and estimation in both active Vegas and

original Vegas, we show the values of BaseRïT measured by different connections in

Figure 4.13. Again. the x-axis represents the connection index.

1st rtt measurement i 2nd rtt update A 3rd rtt update

Active Vegas, alpha=l , beta=3 Active Vegas, alpha=beta=2

Connection index

Original Vegas, al pha=l , beta=3

O 10 20 30 40

Connectron index

Connection index

Original Vegas, alpha=beta=2

O 1 O 20 30 40

Connection index

Figure 4.13 BaseRïT of connections for al1 scenarios

The main point behind Figure 4.13 is that original Vegas cannot estimate the

BasRm properly. End-to-end delay c m be calculated as follows:

(End-to-end delay) = (Transmission delay) + (Propagation deZay)

+ (Packet Processing delay) + (Querting delay)

Considering the access link transmission delays in our calculation, the BrrreRTT should

be around 80 to 100msec. Since the packet size is IKbyte, and link capacities are lMbps,

the transmission delays corresponding to the bottleneck link and the access links wiil be

around 8msec. Moreover the propagation delay is 30msec; hence, adding al1 together, the

BaseRTT will be 84msec. As it can be seen in Figure 4.13, active Vegas gives a very

accurate value (88msec) for BaseRlT for al1 connections. In contrast. original Vegas

BaseRlT estimates becorne wone and worse when a new connection joins the network

such thl3t the BaseRTTestimate for the last connection is more than 1 second.

4.4.2.2 Fairness Evaluation

The faimess criterion has been widely studied in the litenture but no single

metric has been comrnonly accepted [32]. In this research, we consider a "fair share per

link" metric; when n users share a bottleneck link, each flow bas the right to llnth of the

capacity of the bottleneck bandwidth. lain's faimess index [33] is applicable in this

context. For n flows, with flow i being allocated an xi on the bottleneck link, the faimess

index is defined as follows:

Fairness

+Active Vegas, alpha<beta +Active Vegas, alpha=beta

+Original Vegas, alphacbeta +Original Vegas, alpha=beta

Bottleneck link propagation delay

Figure 4.15 Faimess vs. bottleneck link propagation delay (link capacity = 4Mbps)

Observation frorn Figure 4.15 reveals that original Vegas faimess increases with

an increase with propagation delay. In other words, original Vegas perfoms better in

long TCP co~ection. The reason is that in long TCP co~ect ion, the share of the queuing

delay in the total end-to-end delay becomes smaller. As a result, the over-estimation error

in Base R 7T becomes smaller; and consequently unfaimess will be reduced.

4.4.2.3 Buffer Occupancy Although active Vegas does not suggest a way to completely eliminate persistent

congestion, it mitigates the level of the congestion in the network. As we discussed in

section 2.5.2, the over-estimation of BaseRTT results in larger backlog. The simulation

results with 30 sources and destinations also confirm the fact that original Vegas undcr-

estimates the number of packets in the burer, due to over-estimation of BaseRlT for new

connections. On the other hand, active Vegas achieves the expected number of packets in

the buffer, which is n p for n active connections. Comparing Figure 4.10 and Figure 4.12

reveals that active Vegas mitigates the level of congestion in the network with less

number of backlogged packets. Table 4.2 summarizes the buffer occupancies.

Table 4.2 Buffer occupancies for 4 different implementations of Vegas

Buffet Occupanc y

(pac ke t)

4.4.2.4 Summary of Simulation Results

in this section, we surnrnarize Our simulation results for the single buttleneck

network with 30 sources and destinations in tabular format, which is shown in Table 4.3.

1 1 Active Vegas 1 Active Vegas 1 Original Vegas 1 Original Vegas

Onginal Vegas

c r = F = z

Osciliatory

L 62

Active Vegas

a=i.j3=3

Stable

68

Active Vegas

a = B = 2

Oscillatory

64

Miuirnurn faimess

metric

Minimum fairness

me tric

Original Vegas

a = i , p = 3

Stable

170

Maximum BaseRTï

(mil tisecond)

CU =i. f l =3

1

0.998 1

Minimum BuseRn

(milliseconci)

Maximum mean

throughput

( pac ketkec)

throughput

(pac ketfsec)

I

a =/? = 2

0.9805

0.9166

1120 I I

84.96

6.42

Minimum mean 1 1

Table 4.3 Summary of simulation results

92.64

1

L

B uffer occupancy

1 (packet)

a =i. /? =3

0.9 106

0.785

84.96

4.26

= p = ?

0.8333

0.6673

92.64

68

1 180.64

84.96

9.1

84.96

10.54

64 170 162

4.4.3 Multi-hop Network Simulations

To M e r study the fairness performance of active Vegas under situations where

connections experience different round trip delay and traverse different number of

gateways, a rnulti-hop network topology, as illustrated in Figure 4.16, is implemented in

ns-2.

Figure 4.16 Multi-hop network mode1

S 1 SS S7

Core links

Bandwidth = lOMbps Bandwidth = lMbps

It consists of 8 connections. The shared links have lMbps capacities. Also, a lOMbps

access link connects each host to the router. A flow is setup from host Sx to host Dx and

is identified by a flow Idx. Packet size for al1 of the connections is 1Kbyte.The

propagation delays of the access links and shared links are 5msec and SOmsec,

respectively. Two sets of simulations were conducted under this topology with active

Vegas (a = f l = 2), and original Vegas ( a = /3 = 2) with no packet &op. Successive

Delay = Smsec Delay = 50msec . D8

:

connections (i = 8,7, .. ., 1) join the network every two seconds. Al1 of the senders turn

off after 100 seconds and each simulation lasted for 1 10 seconds.

Fim of d l , Ict us look at average congestion windows during congestion

avoidance phase for difiierent connections under active Vegas and original Vegas, which

is s h o w in Figure 4.17.

Ri Active Vegas Mriginai Vegas

a 1 2 3 4 5 6 7 8 Connection index

Figure 4.17 Average congestion windows for different connections

Since the First four co~ec t i ons are homogeneous, they should have the sarne window

size to share the bandwidth equaily. Figure 4.17 illustrates that original Vegas cm not

achieve this. The average congestion window sizes are 10.2 1 and 18.39 for c o ~ e c t i o n 4

and c o ~ e c t i o n 2 respectively, which has a difference of 7 packets. On the other hand,

active Vegas can force al1 of the homogenous connections to have the sarne window size,

e.g. average congestion window sizes are 1 1.47 and 11.84 for connection 4 and

connection 1 respectively

Another important thing io note, which is cornmon to both active and onginai

Vegas, is that Vegas in general does not penalize long TCP connections. As it can be seen

in Figure 4.1 7, short TCP connections have smaller window size compared to long TCP

connections (connection 8 has the smallest propagation delay and connections 1,2,3, and

4 have the longest propagation delays). This will result in fair allocation of bandwidth

among al1 of the connections. To have a quantitative idea, Figure 4.18 describes the

connection goodputs versus different connections.

i L I I

1 l

20.=[ , :

20

L

19.5 O 2 4 6 6

connection index

Origine) Vegas

+ l

I &

t 1

' * - e l .

r * C

connedan index

Figure 4.18 Goodput vs. connection index for active and original Vegas

Same as single bottleneck topology, the average and the standard deviation of

each c o ~ e c t i o n goodput is illustrated in Figure 4.18. To remove the startup transient

from the calculation, mean goodputs were averaged in the last 80 seconds of the

simulation. Again, active Vegas performs better and allocates the bandwidth more fairly

than original Vegas. The differences between maximum and minimum mean values of

the connection goodputs are 2.46, and 11.77 packetfsec for active and original Vegas

respectively, which shows a better f ahess with active Vegas. As mentioned before,

active Vegas does not penalize long TCP connections, i.e. co~ec t i ons 1, 2, 3 and 4 with

longer propagation delays receive the same share as connection 8 with srnailest

propagation delay.

To compare BaseRTT measurement and estimation in both active Vegas and

original Vegas, we show the values of BuseRï7' in millisecond measured by different

connections in Table 4.4. In this table blank slots mean that BoseRTT was not ,updated

during the simulation, because the first measurement had the smallest value.

Table 4.4 BaseRTT estimation for active and original Vegas

For the fint 4 homogeneous connections, the minimum possible value for BaseRTT

theoretically is 561.6msec. The transmission delay for the R1, R2, R3, R4, and R5 is

âmsec, for R6 is O.8msec. The propagation delay is 260msec. As it c m be seen in Table

4.4, active Vegas estimates BaseRn with a much better accuracy than original Vegas.

BmeRTT measured by active Vegas is 571.008msec and BaseRn measured by original

Vegas is 721.024msec. The error comsponding to original Vegas is almost more than

LOOmsec, which leads to a very unfair bandwidth allocation.

BaseRTT Fit mesurement Last update

Ori_eind

665.024

72 1.024

633.024

585.024

526.704

364.384

256.08

129.983

Active

- -

568

568

- 346.642

240

-

Active Onginal

-

672

- - 480

360

340

-

1

7 - 3

4

5

6

7

8

573.632

565.632

571.008

571.008

462.703

348.384

240.08

129.984

Chapter 5

Conclusion

In this thesis, we have first focused on evaluating the essential characteristic of

TCP, mainly TCP-Vegas. We have obtained the results that TCP-Vegas sometimes fails

to keep faimess among connections. From those results, we have proposed the active

version of TCP-Vegas, which has better fairness than the original Vegas. This proposai is

based on a probe mechanism to better estimate the propagation delûy and measure the

smallest round trip delûy.

To what extent is the network performance improved? What is the ptice of this

improvement? What are the problems created by this new proposal? This chapter answers

the above questions, and concludes the thesis with a summary of areas for future work,

and research contributions.

5.1 Discussion We noted that the measurement of the propagation delay in TCP-Vegas is not

robust .The primary goal of this thesis is to decrease the effect of enor in round trip delay

estimation from TCP-Vegas algorithm (a delay-based congestion control mechanism). In

the generd case where the route remains the sarne during the connection time, the

propagation delay can only be over-estimated because of the queuing delay. We observed

that over-estimation of propagation delay leads to unfair resource allocations of the old

connections. As stated previously. TCP-Vegas intends to keep between a and

packets in the buffen. However, with the over-estimation of BoseRTT. although it

beiieves that it has between a and #3 packets in transit, in fact it has many more packets

in transit. In other words, biased BaseRTTcauses more packets to be backlogged in the

buffers. This induces persistent congestion in the network.

Throughout the course of this research, we leamed that a probing mechanism is

useful for discovering the path characteristics, such as deiay, queue length and

bandwidth. Along the same Iine, we propose active TCP-Vegas which consists of an

extension to original Vegas. This change requires support both frorn gateways and host IP

la yen.

Assuming no changes in the connection route, we have demonstrated that sending

a high pnority probe packet by TCP-Vegas, and fast forwarding rnechanisms at the

gateways resolve the problem of BaseRZT over-estimation. Our simulation results with a

single bottleneck and multi-hop networks are shown that this simple enhancement to

TCP-Vegas improves the fairness and reduces the number of packets in transit; and hence

level of the congestion in the network.

The most innovative feature of TCP-Vegas is the window control mechanism

based on delay measurement, which results in stable window size and higher throughput.

However, a packet loss would still lead to a congestion window reduction by a large

amount. Therefore to achieve the improvements proposed by TCP-Vegas, packet drops

should be prevented. Otherwise, the Reno behavior will kick in, congestion window will

oscillate, many retransmissions will occur, and throughput will be decreased.

Since the performance improvernent heavily depends on probe packet

acknowledgement, we further assume that probe packets and their acknowledgments are

not dropped or lost. Moreover, we assume that probe packets cary the same T'OS and IP

security options as the regular data packets will. In a nutshell, we assume the sarne route

for data and probe packets.

We believe that we c m encourage TCP-Reno usea with a proper incentive to

switch to -4ctive TCP-Vegas by using a simple priority drop tail scheduling mechmism a:

the gateways to provide better throughput and less delay. The only concern is the

incompatibility of Vegas and Reno, 3s stated in the literature [12]. In other words, Reno

and any variant of Vegas, enhanced or active, c m not share the s m e buffer. To remedy

this problem, a DRR scheduling mechanism to provide the necessary isolation between

the Reno usen and the Vegas users cm be proposed.

5.2 Drawbacks The important drawbacks of Our proposa1 are as follows:

Coordination and support between Active TCP-Vegas and IP layer is required, since

the active Vegas probe packet should be marked at IP header. For Ipv4. this c m be

done using the IP type of service (TOS) octet, although six bits of this byte has

already been assigned to differentiated services code point (DSCP). In Ipv6, there is

an IP priority field that cm be used for marking active Vegas probe packet.

Support from gateways should be provided to probe packets. The major drawback of

our proposal is that a priority &op tail scheduling algorithm should be implernented

at every gateway in the network. This condition might not always be fulfilled.

One of the advantages of Our proposal is that no changes in the current receiver

implementation are required. However, receivers can not install delayed

acknowledgement. This condition is not limited to our proposal and is comrnon

among dl delay-based congestion control mechanisms.

5.3 Future Research Directions Although this research has addressed an important problem of delay-based

congestion control mechanism, particularly TCP-Vegas, several issues remain for further

investigations. The following lists some of the future research directions.

1. Rerouting also causes error in propagation delay estimation. When the route

changes dunng the connection time, the propagation delay may increase. The

sender believes, this increase in round trip delay is a result of congestion in the

network, and consequently decreases its transmission rate. This will result in a

poor utilization of the network. With rerouting, we face an under-estimation of

BaseRTT. One solution to this problem is sending probe packets periodically. One

of the problems that needs further investigation is: what is the effect of the

frequenc y of probe packets?

II. We did not consider the effect of other traftic in the performance of active TCP-

Vegas. What types of mechanisms are required to isolate the active Vegas flows

from other trafic? Can active Vegas and Reno share the same buffer? if so, what

type of queue mniagement and scheduling is appropriate in the buffer? These are

the questions that need to be addressed.

III. Although the in-band approach is simple to implement, and creates less overhead,

in Our proposai, probe packets are out-band. It is also possible that instead of fast

forwarding of the probe packets (out-band approach), they are forwarded like

regular data packets (in-band approach). in the in-band approach, gateways dong

the route must update the probe packet the queuing delay it experiences.

Furthemore, in-band mechanism also requires support from the peer hosts. Once

the probe packet reaches the end of its route, the queuing delay information it has

collected must be piggybacked in the acknowledgement.

N. Since TCP-Vegas was aiready implemented as a patch to the LENLTX kernel,

active Vegas extension c m also be implernented in the LINUX kernel to further

study the performance in a reai network.

5.4 Research Contributions This thesis has addressed a number of issues regarding the congestion control

mechanism of TCP-Vegas. With simulation analysis, this thesis has presented severai

drawbacks of TCP-Vegas, which have their root in round trip delay estimation. To

improve the faimess performance of TCP-Vegas, this thesis proposed a mechanism based

on the active networking approach. We develop "Active TCP-Vegas" that efficiently

estimates the path propagation delay. The simulation results from Our work lead to the

following observations and conclusions.

Faimess Performance. We investigate the faimess performance of active TCP-Vegas

and compare it with original TCP-Vegas. We expenment with both a single

bottleneck and a multi-hop network. Our results show that active Vegas fairly

allocates the bandwidth arnong the connections, and does not penalize the old

connections in favor of the new connections.

Persistent Congestion. We examine the buffer occupancy of active Vegas. We show

that better estimation of BaseRTT leads to a smaller number of packets backlogged in

the buffers compared to original Vegas. This decreases the level of congestion in the

network.

References

T. V. LAshman, U. Madhow, "Performance Analysis of Window-Based Flow

Control using TCPIIP: The Effect of High Bandwidth-Delay Products and Random

Loss", Proceeding of HPN'94. 1994.

L. S. Brakmo, L.L. Peterson, 'TCP Vegas: End-to-End Congestion Avoidance on a

Global Internet", IEEE Journal on Selected Areas in Commiinications, Oct. 1995,

pp. 1465-1480.

J.S.Ahn, P.B. Danzig, 2. Liu and L. Yan, "Evaiuation of TCP Vegas: Emulation

and Experiment", Compziter Cornmrinication Review. ACM SIGCOMM, 1995, pp.

185-195.

Z. Haas, " Adapti ve Admission Control". Cornputer Cornmcinications Review. ACIM

SIGCOMM, Jan. 1991, pp. 58-67.

V. Jacobson, "Congestion Avoidance and Control", SIGCOMM Symposiirm on

Communications Arcltitectzires and Protocols, 1988, pp. 3 14-329.

V. Jacobson, "Modified TCP Congestion Avoidance Algorithm". Technical report,

Apr. 1990, ftp://ftp.ee.lbl.gov/email/vanj.90apr30.txt.

R. Jain, "A Delay-Based Approach for Congestion Avoidance in Interconnected

Heterogeneous Computer Networks", Cornputer Communications Review. ACM

SIGCOMM, Oct. 1989, pp. 56-71.

K.K. Rarnakrishan, R. Jain, "An Explicit Binary Feedback Scheme for Congestion

Avoidance in Computer Networks with a Connectionless Network Layer", Proc.

ACM SIGCOMM, Aug. 1988, pp. 303-3 13.

2. Wang, J. Crowcroft, "A New Congestion Control Scheme: Slow Start and Search

Uri-S)", Computer Commiinication Review, ACM SIGCOMM, 1990.

[IO] L. S. Brakmo, S. W. O'Malley and L. L. Petenon, 'TCP Vegas: New Techniques

for Congestion Detection and Avoidance", Proc of ACM SIGCOMM'94, Oct. 1994.

pp.24-35.

[Il] T. Bonald, "Comparison of TCP Reno and TCP Vegas via Fluid Approximation",

Technicd Report, m, 1998. Availabie at

http://www .dmi .en~.fr/%7ernistraYtcpworkshop. html.

[12] J. Mo, R.J.La, V. Anathararn and J. Walrand, "Analysis and Comparison of TCP

Reno and Vegas", Globecomm '99.

[13] G. Hasegawa, M. Murata and H. Miyahara, "Faimess and Stability of Congestion

355,262, Control Mechanisrn of TCP", 11''' ITC Specinlis? Seminar, Ocr. 1998, pp. -dd

[14] S. H. Low, L. Peterson, and L. Wang, "Understanding TCP Vegas: Theory and

Practice", TR 6 i 6-00, Feb. 2000. AvaiIabIe at

h ttp://www .ee.mu.oz.au/staff/slow/papers/vegas.ps.

[15] ns-2 Simulator, Available at http://www-rnash.cs.berkeley.edu/ns/.

[16] C. Kent and J. MoguI, "Fragmentation Considered Harmful", K M SIGCOMM87,

Aug. 1987.

[17] A. Romanow and S. Floyd, "Dynamics of TCP Trflic over ATM Networks", IEEE

Journal of Selected Area in Communications, May 1995.

[18] S. Floyd, snd V. lacobson, "Random Early Detection Gateways for Congestion

Avoidance", IEEUACM Transactions on Networking, Aug. 1993, pp. 397413.

[19] D. L. Tennenhouse, and D. J. Watherall, "Towards an Active Network

Architecture", ACM Cornputer Cornmirnication Review, Apr. 1996, pp. 5- 18.

[20] S. Bhattachajee, K. Calvert and E. Zegura, "An Architecture for Active

Networking". froc. High Perf: Networking 97, 1997.

[21] D. L. Wetherall, U. Legedza and J. Guttag, "Introducing New Internet Services:

Why and How", IEEE Nenvork Magazine, JulyIAugust 1998.

[22] B. Schwartz, A. W. Jackson, W. T. Strayer, W. Zhou, D.Rockwel1 and C. Partridge,

"Smart Packets for Active Networks", OPENARCH'99, Mar. 1999.

[23] S. Kasera, S. Bhattacharyya, M. Keaton, D. Kiwior. J. Kurose, D. TowsIey, and S.

Zabele, "Scalable Fair Reliable Multicast Using Active SeMces", IEEE Nework

Magazine (Special Isszte on Mirlticast), January/Fe bruary 2000.

[24] E. Takahashi, P. Steenkiste, J. Gao, A. Fisher, "A Programming Interface for

Network Resource Management", OPENARCH'99, Mar. 1999.

(251 T. Faber, "ACC: Using Active Networking to Enhance Feedback Congestion

Control Mechansirns", IEEEE Neîwork Magazine, MayIJune 1998.

[26] M. Hicks et al. "PLANET: An Active Intemetwork", INFOCOMM '99, Mar. 1999.

[27] L. Zhang, S. Deering, D. Estrin, S. Shenker and D. Zappala, "RSVP: A New

Resource ReSerVation Protocol", IEEE Network Magazine. Sept. 1003, pp. 8- 18.

[28] K. Najafi, A. Leon-Garcia, "A Novel Cost Mode1 for Active Networks",

Proceeding of ?nt. Conf. on Communication Technologies, World Cornputer

Congress 2000.

[29] R.J.L. and V. Anatharam, "A Case Study for TCP Vegas and Deficit Round-

Robin Gateways", Available at http://www.path.berkeley.edu/-hyongla. Dec. 1998.

[30) G. Hasegawa, T. Matsuo, M. Murata and H. Miyahara, "Cornparison of Packet

Scheduling Algorithms for Fair Service among Connections on the Intemet".

Proceedings of iWS'99. Feb. 1999, pp. 193-200.

[31] M. Shreedhar and G. Varghese, 'Efficient Fair Queuing Using Deficit Round

Robin", Proceeding of ACM SIGCOMM'95, Sept. 1995.

[32] S. Floyd, "Connections with Multiple Congested Gateways in Packet-Switched

Networks, Pari 1 : One-way Traffic", ACM Computer Communications Review, Oct.

1991, pp. 30-47.

[33] D. Chiu and R. Jain, "Analysis of the increase and Decrease Algorithm for

Congestion Avoidance in Computer Networks". Computer Networks and ISDN

Systems, 17(1989) pp. 1-14

Appendix

In this section, the simulation tool and experiment setup topology are descnbed

(most of the information in this part is brought from ns web page [15]).

A.l ns network Simulator

One of the goals of this thesis is to implement Our active Vegas proposai in ns

network simulation environment. ns is an event-driven network simulator. An extensible

simulation engine is implemented in C++ that uses MIT'S Object Tool Language, OTCL

as the command and configuration interface. The interface to ns must be written in TCL.

To run the simulator, one needs to be able to program TCL. It's usually convenient to

represent the results in xgraph form. However, some of graph is sketched with

MATLAB.

A network topology is realized using three primitive building blocks: nodes, links

and agents. The Simulator class has methods to createkonfigure each of these building

blocks. Agents are the objects that actively drive the simulator. Agents can be thought of

as the processes andor transport entities that run on nodes that may be hosts or routers.

TrafXc sources and sinks are al1 example of agent.

A. 1.1 Links

Links are created between nodes to form a network topology. In this project, we

used a duplex-link method that sets up a bi-directional link. Link bandwidth and

propagation delay c m be set. Since we are investigating Priority-Queuing scheduling

algorithm, PQ objects are attached to each link. PQ objects are a subclass of Queue

objects that implement priority based queuing. The only parameter that could be set is

limited to the queue size in packet. Default value of queue size is 50.

A.1.2 Source and Traffic Types

T ~ c object created data for a transport protocol to send. In our simulations we

used TCP as transport protocols. However, we implemented active Vegas as a subclass of

TCP agent. Al1 of TCP parameters are set as default (e.g.. window size is 20 packets). For

TCP protocol, FTP and TELNET source object c m be used. FlT objects produce bulk

data for a TCP object to send.

A.1.3 Trace and Monitoring

There are a number of ways of collecting output or trace data on a simulation.

Generaily trace data is either displayed directly during execution of the simulation. or

(more cornmonly) stored in a file to be post-processed and analyzed. The trace records

each individual packet as it arrives, deparis, or is dropped at link or queue. We can also

open files in the program to read / wnte any particular variable frodto it.

A.2 Measured Statistics

In this thesis, we mainly focused on two parameters, the end-to-end delay per

flow for each source (since al1 f sources transport one type of flow, it could be assumed

as end-to end delay for each source) and allocated bandwidth for each source, i.e. source

goodput.

As explained earlier, trace data is stored in a file. This file includes almost dl the

information from the simulation. Since the amount of data is very large, it's impractical

to inspect the records without some tools. Therefore several scripts for processing the

records was developed in AWK.

The raw trace records are first sorted by unique-id of each packet. Then fiom this

sorted file we extract those record belong to each flow and then we can calculate end-to-

end delay by subtracting the enque time from the source node and receiving time by the

destination node. After extracting necessary data h m the trace record, this information

will be ported to MATLAB to find the delay histogram and mean and variance for the

corresponding distribution.