Self-Similar through High-Variability:Statistical Analysis of Ethernet LAN Traffic at the Source Level

Walter Willinger, Murad S. Taqqu, Robert Sherman, Daniel V. Wilson

Bellcore, Boston University

SIGCOMM’95

Outline

Introduction

Self-similarity through high-variability

Ethernet LAN traffic measurements at the source level

Implications of the Noah Effect in practice

Conclusion

Introduction

Actual traffic exhibits correlations over a wide range of time scales (i.e. has long-range dependence).

Traditional traffic models focus on a very limited range of time scales and are thus short-range dependent in nature.

Introduction

Two problems that cause the resistance toward self-similar traffic modelingWhat is a physical “explanation” for the

observed self-similar nature of measured traffic from today’s packet networks?

What is the impact of self-similarity on network and protocol design and performance analysis?

Introduction

The superposition of many ON/OFF sources whose ON-periods and OFF-periods exhibit the Noah Effect produces aggregate network traffic that features the Joseph Effect.Noah Effect: high variability or infinite varianceJoseph Effect: self-similar or long-range

dependent

Self-Similarity through High-Variability

Idealized ON/OFF modelAn ON-period can be followed by an ON-

period and an OFF-period can succeed another OFF-period.

The distributions of the ON and OFF times may vary.

Idealized ON/OFF Model

Reward sequence

{W(l ), l = 0,1,2,…}{W(l )} is a 0/1-valued discrete time stochastic

process.W(l ) = 1 or 0 depends on whether or not there

is a packet at time l.{W(l )} consists of a sequence of 1’s (“ON-

periods”) and 0’s (“OFF-periods”)

Idealized ON/OFF Model

The lengths of the ON- and OFF-periods are i.i.d. positive random variables, denoted Uk, k = 1,2,…

Let Sk = S0 + U2 + … + Uk , k 0 be the corresponding renewal times.

,...2,1,0 ),1())(()( 10 uuUPUEuSP

Idealized ON/OFF Model

Suppose there are M i.i.d. sourcesThe mth source has its own reward sequence {W

(l ), l 0}Superposition reward (“packet load”)

b: non-overlapping time blocksj: the aggregation block number

,...2,1,0 ,)()()1(

1 1

)(*,

jlWjWjb

bjl

M

m

mbM

Idealized ON/OFF Model

Suppose that U has a hyperbolic tail distribution,

as M and b , adequately normalized is fractional Gaussian noise

, which is self-similar with Hurst parameter ½ H <1

(1) ,21 , as ~)( ucuuUP

}{ *,bMW

}0 ),({ , ttGH

Idealized ON/OFF Model

Property (1) is the infinite variance syndrome or the Noah Effect.

2 implies E(U2) = > 1 ensures that E(U) < , and that S0 is not

infinite

Idealized ON/OFF ModelTheorem 1. For large enough source Number M and Block aggregation size b, the cumulative load

behaves statistically as

where and . More precisely,

where Llim means convergence in the sense of the finite-dimensional distributions (convergence in law)

}0 ),({ *, jjW bM

)(2

1,

2/1 jGMbbM HH

2

3 H

)3)(2)(1(2)(4

12

UE

)(2

)(limlim ,*

,2/1 jG

bMjWMb HbM

H

Mb

LL

Ethernet LAN Traffic Measurements at the Source Level

Location Bellcore Morristown Research and Engineering Center

The first set The busy hour of the August 1989 Ethernet LAN measureme

nts About 105 sources, 748 active source-destination pairs 95% of the traffic was internal

The second set 9 day-long measurement period in December 1994 About 3,500 sources, 10,000 active pairs Measurements are made up entirely of remote traffic

Textured Plots of Packet Arrival Times

Textured Plots of Packet Arrival Times

Checking for the Noah Effect

Complementary distribution plots

Hill’s estimateLet U1, U2,…, Un denote the observed ON-(or

OFF-)periods and write U(1) U(2) …U(n) for the corresponding order statistics

uucuUP as ),log()log(~))(log(

(3) ,)log(log1

ˆ11

0)()1(

ki

iknnn UU

k

A Robustness Property of the Noah Effect

As far as the Noah Effect is concerned, it does not matter how the OFF-periods have been defined.

The similar investigation of sensitivity of the ON-period distributions to the choice of threshold value reveals the same appealing robustness feature of the Noah Effect.

(4) 21 ,~)|(

t

utUuUP

Self-Similarity and the Noah Effect: 1989 Traffic Traces

181(out of 748) source-destination pairs generated more than 93% of all the packets are considered.

The data at the source-destination level are consistent with ON/OFF modeling assumption Noah Effect for the distribution of ON/OFF-periods

-values for the ON- and OFF-periods may be different.

Self-Similarity and the Noah Effect: 1994 Traffic Traces

Non-Mbone traffic300 (out of 10,000) pairs responsible for 83% o

f the traffic are considered.Self-similarity property of the aggregate packet

stream is mainly due to the relative strong presence of the Noah Effect in the OFF-periods.

Self-Similarity and the Noah Effect: 1994 Traffic Traces

Mbone trafficOnly an analysis of the aggregate packet stream

is performed.The strong intensity of the Joseph Effect becom

e obvious only after aggregation levels beyond 100ms.

There is no Noah Effect for ON-periods.Reason: The use of unsophisticated compression alg

orithms resulted in packets bursts separated by comparatively large idle periods.

Traffic Modeling and Generation

Although network traffic is intrinsically complex, parsimonious modeling is still possible.Estimating a single parameter (intensity of

the Noah Effect) is enough.

Performance and Protocol Analysis

The queue length distributionTraditional (Markovian) traffic: decreases expo

nentially fastSelf-similar traffic: decreases much more slowl

y

Protocol design should be expected to take into account knowledge about network traffic such as the presence or absence of the Noah Effect.

Conclusion

The presence of the Noah Effect in measured Ethernet LAN traffic is confirmed.

The superposition of many ON/OFF models with Noah Effect results in aggregate packet streams that are consistent with measured network traffic, and exhibits the self-similar or fractal properties.