Parameter Estimation and Performance Analysis of Several Network Applications Sara Alouf Ph.D. defense - November 8, 2002 Advisor: Philippe Nain

Parameter Estimation and Performance Analysis

of Several Network Applications

Sara Alouf

Ph.D. defense - November 8, 2002

Advisor: Philippe Nain

Thesis topics

Adaptive unicast applications Background: network does not offer guarantee Objective: estimate network internal state

Large audience multicast applications Background: need for membership estimates Objective: efficiently track membership

Mobile code applications Background: existence of several mechanisms

for objects communication Objective: determine fastest among two of them

Thesis topics


Challenges: efficient congestion control, good QoS

Two distinct approaches: adding intelligence to network adding intelligence to applications

acquire some knowledge on network change application policy accordingly

Adaptive unicast applications

Application

Poisson probes

data packetsSink

Methodology:source probes networkhaving feedback from destination, source measures

some performance metrics (e.g. loss probability, end-to-end delay, conditional loss probability, etc.)

K

given model for connection, metrics are expressed in terms of network internal state

given performance metrics, source infers network internal state

Adaptive unicast applications

Main contributions: Detailed analysis of the M+M/M/1/K queue

(expressions for 5 metrics of interest, including loss-related conditional probabilities)

New analysis of the M+M/D/1/K queue (explicit information on stationary distribution; expressions for 3 metrics of interest)

Identification of “best” way of inferring network internal characteristics:

use loss rate and network response time

given by M+M/M/1/K queue model

Thesis topics





Large audience multicast applications

Motivation - Objective

Kalman filter

Wiener filter

Least square estimation

Extension



Kalman filter

Wiener filter


Extension

Motivation

Interesting multicast applications (distance learning, video-conferences, events, radios, televisions (?), live sports(?), etc.)

Membership is required for: feedback suppression (RTP, SRM) tuning amount of FEC packets for reliability pricing stopping transmission when no more receivers

and especially for radios and future TVs, to: adapt transmission content, advertise, ...

Previous work#ACKsneeded

Previousestimate

BiasFeedbackimplosion

Bolot, Turletti &Wakeman

at leastone

no possibleno if

N 216

Nonnenmacher &Biersack

at leastone

yes yes no

Friedman &Towsley

at leastone

no no no

Liu &Nonnenmacher

at leastone

no possible possible

Need for unbiased estimator that efficiently uses previous estimates

Methodology

Source: periodically requests from receivers to send

ACK with probability p every S secondsReceivers:

each S seconds, send ACK to source with prob. pSource:

stores Yn number of ACKs received at time nS

Objective: use noisy observation Yn to estimate membership Nn N(nS)

Naive estimation

Drawbacks:

very noisy (s.l.l.n. lim N Y/N = p) no profit from correlation (no use of previous

estimate)

p

YN n

n ˆ

Naive estimation : p 0.01

Naive estimation : p 0.50

EWMA estimation

Advantages: use of previous estimate no a priori information needed

Drawbacks: what value for ? estimator does not depend on ACK interval S

10

1ˆˆ,1,

p

YNN n

nn

EWMA estimation

Objective

Use optimal filtering techniques to find estimator

Notation

Ti join time of participant i

Ti+Di leave time of participant iN(t) number of participants at time t

Occupation process in the G/G/ queue… not much is known about it …

iiii

DTtTtN 1

1



Kalman filter

Wiener filter


Extension

M/M/ model - heavy traffic case

Assumptions: Poisson arrival process, intensity T exponential on-times, parameter

Occupation process in the M/M/ queue

average membership: T

T

if T , ZT(t) Ornstein-Ühlenbeck process

udBeeXtXt utt 0

20

{B(t), t 0} standard Brownian motion

Define normalized membership T

TtNtZ T

T

Optimal estimation - Kalman filter

Ornstein-Ühlenbeck process in discrete time

udBenSXeSnXSn

nS

uSnS 1 121

nnn w 1

udBew

enSXSn

nS

uSnn

Sn

1 12

,

and

with

wn are white noise with variance Q = (12)

Number of ACKs at step n: Yn

Define normalized measurement

T

nSpNY

T

TnSNp

nT

TpYM

TnT

nn

,1,0,

Weak limit T : nnn vpm


vn are white noise with variance R = p(1p)

ZT(nS) VT(n)


Stationary version

Optimal filter minimal mean-square error

System dynamics n+1 n wn

Measurement mn pn vn

wn and vn white noisevariances Q and R

Error variance P = ([ 2 P + Q]1 + p2 / R)1

Filter gain K = Pp/RState estimator

])ˆ[(ˆˆ11 nnnn pmK

actualizationprediction


KpTKYNKpNnnn

11ˆ1ˆ1

known assumed and

step nobservatio th at ACKs of amount

Finally

of estimator Define

of estimator

T

nY

TTN

NN

n

nn

nn

nn

ˆˆ

ˆ

ˆ

p

YNN n

nn 1ˆˆ,1,

EWMA estimator

Kalman filter

To summarizeEstimation

KpT

KYNKpNnnn

11

ˆ1ˆ1

])ˆ[(ˆˆ11 nnnn pmK

])ˆ[(ˆˆ11 nnnn pMK ZT(t)

X(t)

NT(t)

Continuous

time

System state

norm

al iz

ew

eakl

y

Zn = ZT(nS)

n X(nS)n+1 n + wn

Nn NT(nS)

Discretetime

weakl

yn

orm

al iz

e

Mn = p Zn + VT(n)

mn p n + vn

nY

Measurement

weakl

yw

eakl

y

weakl

yn

orm

al iz

e

Simulations

Objective: validate model

Assumptions made in theory Poisson arrivals Exponential on-times Heavy-traffic regime

Simulations: 2 regimes investigated: light load/heavy-load 2 distributions: Exponential/Pareto

8 different scenarios simulated

Validation with real traces

Objective: further validate modelRobustness to “real” distributions? Independence-related assumptions are

violated

Distribution of traces investigatedBest fi t f or inter-arrivals sequence

Best fi t f or on-times sequence

Short audio Weibull Weibull

Long audio Lognormal Lognormal

Membership in real traces vs. time

Objective

Find optimal estimator under more general assumptions



Kalman filter

Wiener filter


Extension

M/G/ model

Assumptions:Poisson arrival process, intensity on-times have common probability distribution

D denotes a generic random variable

Occupation process in the M/G/ queue

Characteristics of N(t) in steady-state:Poisson random variable, Mean Variance D

Autocorrelation function

Notation:

duuDPhtNtNh

,Cov

SknNnSNkN ,CovCov

Optimal estimation - Wiener filter

yn Wiener filterHo(z)

Optimal linear filter minimal mean-square error

Noisy observation Yn

Optimal estimation - Wiener filter

k

ky

k

knn

zkzSkz

zkzSy

yy

yy

CovCov

Cov

, of transform-

, of spectrum power

ppkkpk

kpk

y

y

10CovCov

CovCov2

1

Compute

ion,factorizat Canonical

zGzHzH

zG

zSzH

zGzGzS

oy

y

1

1

Introduce:

We have:

Application to M/M/ model

pp

p

ppp

B

A

AzBzH

Sk

μD

o

kν

1221111

122111211

2

2222

2222

11

expCov

Exp~

where

find We

When

,

nnn ByAAzBzH

o

1

1

ˆˆ1

response Impulse

function Transfer

Application to M/M/ model

nnn ByA 1ˆˆ :processes Centered

pBABYNAN nnn 1ˆˆ1

ABp

NNE nn

11

ˆ 2

min

error square Mean

Non-centered processes:

Estimators are the same!

ButKalman filter M/M/ queue, heavy trafficWiener filter M/M/ queue

we relaxed one assumption

Kalman filter vs. Wiener filter



Kalman filter

Wiener filter


Extension

Optimal first-order linear filter

0

0

Cov

2122

ˆ

ˆ

ˆˆ1,0

2

2

21

1

k

k

kzzg

pApgApBg

yAB

E

ByA

BA

k

knk

n

nn

nnn

ApB

Minimize

minimized error square-mean

that such and Find

where

state-Steady

Optimal first-order linear filter

solving Numerical

filter Wiener as solution same

unique is Solution

solve to System

LipLonentialexpHyperD

ExpD

ii

B

A

1,,~

~

,

0

0

Validation with real traces

Distribution of inter-arrivals and on-times

Best fi t f or inter-arrivals sequence

Best fi t f or on-times sequence

video1 Lognormal Weibullvideo2 Lognormal Weibullvideo3 Weibull Lognormalvideo4 Weibull Weibull

Mean & Variance of the error

MeanVariance min, min

video12ˆ

ˆ

Hn

En

N

N 0.11210.0469

12.6641

12.8508

13.9424

12.11981.1504video2

2ˆ

ˆ

Hn

En

N

N 0.0062

0.0188

0.4947

0.7851

1.4068

0.39553.5570video3

2ˆ

ˆ

Hn

En

N

N 0.0373

0.0194

0.2065

0.2291

0.7370

0.20843.5365video4

2ˆ

ˆ

Hn

En

N

N 0.0523

0.0651

0.9105

1.4231

1.5656

0.67552.3177

nn NN ˆ

theoreticalempirical

And the winner is …

Advantages: optimal for M/M/ queue efficient over real traces only two parameters required

Drawbacks: a priori knowledge needed

EnN̂Estimator !



Kalman filter

Wiener filter


Extension

Extension

) (initially as estimates

(MLE) as estimates

message hello th of receipt of time records

prob. with message hello"" send arrival, on

) (recall and estimate

:Source

:Receivers

? and estimate to How

pYENE

tq

m

mt

q

nn

m

m

ˆˆˆ

ˆ


Main contributions:

Proposition of several unbiased estimators that efficiently track membership

Validation through simulated and real traces

Identification of “best” estimator among those proposed

Proposition of estimators for a priori parameters

Thesis topics





Mobile code applications

Code mobility paradigm

Forwarders mechanism

Centralized mechanism

Simulations & experiments

Contributions


Definition: components of application might change host (migrate) during execution

Utility: load balancing data mining (data available on different hosts) e-commerce (find the cheapest airline fare)

Issue: ensure communications with mobile objects


Two widely used solutions:

distributed approach (use forwarders) centralized approach (use server)

Objective: identify best approach in terms of response time

Forwarders mechanism: description

S

Host A

O

Host B Host C Host D

S : SourceO : mobile ObjectF : Forwarder

reference


S

Host A

Host B

O

Host C Host D


reference

Message

Forwarding ForwardingF OF

Migrating Migrating


Host B

F

Host C

O

Host D


reference

Update

F

S

Host A


Host B

F

Host C

O

Host D


reference

F

S

Host A

Subsequent messages use new reference

Centralized mechanism: description

S

Host A

O


S : SourceO : mobile Object

referenceServer


S

Host A

Host B

O

Host C Host D


reference

Migrating

Server

Update


S

Host A



reference

Message

MigratingO

Server

UpdateFail


S

Host A



reference

O

ServerQuery

location

Reply

Message

Object may have moved in the meantime

!

Centralized mechanism: the server

may need to send Reply after processing request from Source

S

O S

send Reply

S O


Forwarders mechanism: infinite state-space Markov chain expression for expected response time TF

expression for expected number of forwarders

Centralized mechanism: finite state-space Markov chain expression for expected response time TS

Models validated through simulations and experiments (LAN & MAN)

0

50

100

150

200

250

Experiments Model

Forwarder LAN (100 Mb/s)

= 10

= 1

= 5

1 2 3 4 5 6 7 8 9 10 11

Mean response time (ms) vs. communication rate

migration rate

0

20

40

60

80

100

120

Experiments Model

Server LAN (100Mb/s)

= 5

= 1

= 10

1 2 3 4 5 6 7 8 9 10 11


0

500

1000

1500

2000

2500

3000

Experiments Model

Forwarder MAN (7Mb/s)

= 10

= 5 = 1

1 2 3 4 5 6 7 8 9 10 11


0

500

1000

1500

2000

2500

3000

Experiments Model

Server MAN (7Mb/s)

= 10

= 5

= 1

1 2 3 4 5 6 7 8 9 10 11


Overall performance is fair

models can safely be used for performance evaluation


Main contributions: Proposition of Markovian models for two

communication mechanisms

Validation through simulations and experiments (LAN & MAN)

Theoretical comparison: prediction of fastest mechanism in general

Conclusion

General methodology Propose mathematical models for system at

hand Derive metrics of interest or estimators under

models assumptions Validate models via simulations and/or

experiments

Simple tools applicable over wide range of applications

Conclusion

Optimal filtering techniques estimation of RTT in TCP protocol estimation of average queue size in RED routers …

Performance analysis tools very useful in design of mobile code

applications (high cost of implementation) protocol evaluation …

Thank you!

Documents

Parameter Estimation and Performance Analysis of Several Network Applications Sara Alouf Ph.D. defense - November 8, 2002 Advisor: Philippe Nain