View
31
Download
0
Category
Tags:
Preview:
DESCRIPTION
Two Samples are Enough: Opportunistic Flow-level Latency Estimation using NetFlow. Myungjin Lee †, Nick Duffield‡, Ramana Rao Kompella† †Purdue University, ‡AT&T Labs–Research. Per-hop Measurements are important. AS 1. 100 ms. R2. IPTV/VoIP/ VoD Server. R1. R3. AS 2. - PowerPoint PPT Presentation
Citation preview
Two Samples are Enough:Opportunistic Flow-level Latency Es-
timation using NetFlow
Myungjin Lee†, Nick Duffield‡, Ramana Rao Kompella†
†Purdue University, ‡AT&T Labs–Research
2
Per-hop Measurements are important
R3R2
R1IPTV/VoIP/VoD
Server
Which router causes the prob-lem??
100 ms
AS 2
AS 1
Why 100 ms?!
3
Aggregate vs. Per-Flow
R3R2
R1IPTV/VoIP/VoD
Server5 ms10 ms
AS 2
AS 1
Why 100 ms?!
flow-level latency measurements on a per-hop basis
Aggregate la-tencies look all right. Why?
4
Existing Approaches Active probes and tomography
Chen et al. [SIGCOMM’04], Duffield et al. [IMC’03] Problems
Problem formulation is under-constrained No per-flow latency measurements
Lossy Difference Aggregator (LDA) Kompella et al. [SIGCOMM’09] Problems
Require hardware modification No per-flow latency measurements
5
Basic Framework: NetFlow A measurement framework widely deployed in
routers Maintains per-flow state in the form of a flow record
Packet and byte counts Flow duration (flow start and end timestamps)
Usage Normally used for accounting, traffic matrix estimation,
etc. Does not support per-flow latency measurements
Goal: Enable per-flow latency estimation Harness flow start and end timestamps in NetFlow
framework
6
Obtaining Two Delay SamplesFlow
IDStart
TSEnd TS
1 1
B
A
Flow ID
Start TS
End TS
2 2
12
Flow ID
Start TS
End TS
2 4−
−
=
= Delay2
Delay1
Two delay samples / flowEnd TS
4Start
TS2
R2R1
Flow ID
Start TS
End TS
1 2
End TS2
Start TS1
R3R2
R1IPTV/VoIP/VoD
Server
AS 2
AS 1
7
Problem 1: Independent Packet Sam-pling
Flow ID
Start TS
End TS
1 1
B
A
Flow ID
Start TS
End TS
4 4
12
Only up-date
1st packet
Only up-date2nd
packet
R2R1No coordination between NetFlow instances
8
Solution: Hash-based SamplingFlow
IDStart
TSEnd TS
1 1
B
A
Flow ID
Start TS
End TS
2 2
12R2R1
Hash Space
Sampling Space
Hash-based sampling achieves coordination
Sampled at both NetFlow
Instances
Not sampled at both Net-
Flow In-stances
1 2
9
Problem 2: Packet LossFlow
IDStart
TSEnd TS
1 1
B
A
Flow ID
Start TS
End TS
2 2
12
Flow ID
Start TS
End TS
1 2
R2R1
X
Updateboth
packets
Only up-date
1st packet
Packet losses may cause inconsistencies
10
Solution: Packet DigestsFlow
IDStart
TSEnd TS
1 1
B
A
Flow ID
Start TS
End TS
2 2
Flow ID
Start TS
End TS
1 2
Start PD
End PD
0x01 0x01
Start PD
End PD
0x01 0x01
Start PD
End PD
0x01 0x02
12
X Detect un-usable
timestamp
Detect unus-able
timestampR1 R2Packet digest achieves packet association
11
Consistent NetFlow (CNF) Architecture Issue I: No coordination between NetFlow instances
Different packets are sampled on different NetFlow instances Solution: Hash-based sampling (filtering) in RFC 5475
Same packets are selected on different NetFlow instances IETF PSAMP working group
Issue II: Packet losses Discrepancy in selected packets due to packet losses
Solution: Maintaining packet digests Hash of the invariant packet contents Use timestamps iff packet digests match at the two routers
12
Trivial Estimator: Endpoint Use two delay samples belonging to the same
flow Obtain accurate latency estimates for small flows
Problem: Accuracy penalty for large flows Solution: Multiflow estimator
Time
Endpoint estimator Avg ( , )=
Flow ID
13
Better Estimator: Multiflow Key insight: Packets experiencing same queu-
ing busy periods will experience similar delays Use background delay samples from other flows Use only delay samples between the start and end
of a flow
Time
Multiflow estimator Avg ( , , , )=
Flow ID
14
Evaluation Simulation Setting
Endpoint vs. Multiflow estimators
Comparison with Trajectory sampling
15
Simulation Setting Modified YAF
Simulate a queuing model with RED active queue management policy
Dataset CHIC trace
1 min. trace collected from an OC-192 backbone link About 13M packets and 1M flows
16
Multiflow vs. Endpoint Estimators
1 10 100 1000 100000
0.10.20.30.40.50.60.70.80.9
1MultiflowEndpoint
Flow size
Med
ian
rela
tive
err
or o
fde
lay
mea
n es
tim
ates
Endpoint obtainsgood accuracy
Multiflow performsbetter than Endpoint
17
Trajectory Sampling Shares some similarity with CNF architecture
Routers use consistent hash function to sample packets
Facilitates direct observation of packet trajectories
Requires flow ID and timestamps for per-flow latency estimation Aggregate all sampled packets with same flow key Compute their average latency
18
Comparison with Trajectory Sampling
0.001 0.01 0.1 1 100
0.10.20.30.40.50.60.70.80.9
1MultiflowTrajectory
Relative error of delay mean estimates
CDF
Packet sampling rate = 0.01
Multiflow is 2-3xbetter than Trajectory
19
Summary Our approach retrofits per-flow latency esti-
mates in the NetFlow framework
Two main ideas Consistent NetFlow architecture ensures that dif -
ferent routers record the same set of flows
Multiflow estimator achieves significantly accurate estimates of per-flow latencies compared to prior approach
20
Questions?
Recommended