1

Every Microsecond Counts: TrackingFine-Grain Latencies with a Lossy Difference AggregatorAuthor: Ramana Rao Kompella, Kirill Levchenko, Alex C. Snoeren, and George VarghesePublisher: SIGCOMM’09Presenter: Yun-Yan ChangDate: 2012/02/22

2

Motivation Many network applications have stringent end-to-end

latency requirements, even microsecond variations may be intolerable.

VoIP, automated trading, high-performance computing, etc.

Propose instrumenting routers with a hash-based primitive that called Lossy Difference Aggregator (LDA) to measure latencies down to tens of microseconds and losses as infrequent as one in a million.

Introduction

3

LDA (Lossy Difference Aggregator) A measurement data structure that supports measuring

the average delay and standard deviation of delay.

Both sender and receiver maintain an LDA.

At the end of measurement period, the sender send its LDA to receiver, and the receiver computes the statistics.

Tight time synchronization Consistent packet ordering

Introduction

4

No loss◦ Avg. delay

Difference of timestamp between sender and receiver, divided by number of packets.

Low loss Maintain an array of several timestamp accumulators and

packet counters. Each packet hash to one of the accumulator-counter pairs,

and update the corresponding one. By using the same hash function on sender and receiver, we

can determine the number of packets hash to each pair and the number of loss packets. Assume the number of losses is L, and spilt the traffic into m

separate stream, the expected sample size is at least (1-L/m) received packets.

LDA (Lossy Difference Aggregator)

5

Example

LDA (Lossy Difference Aggregator)

Figure 2: Computing LDA average delay with one bank of four timestamp accumulator-counter pairs. Three pairs are usable (with 5, 2, and 1 packets), while the second is not due to a packet loss. Thus, the average delay is (60 + 22 + 8)/(5 + 2 + 1).

25.11

125

6141537120-180 delay avg.

6

Known loss rate Sample incoming packets to reduce the unusable rows. Hashing to compute the sampling probability. At sample rate p, expect the number of lost packets be pL,

the usable rows is at least m-pL. Arbitrary loss rate

Use multiple LDA banks with different loss rates. Look the high-order bit to determine the updated bank. Example.

Consider three banks with sampling probabilities p1 = 1/23, p2= 1/25 and p3=1/27. Each packet hash to an integer. If the first five bits are zero, update bank 2. If the first seven bits are zero, update bank 3. If the first three bits are zero, update bank 1. Otherwise, the packet is not sampled.

LDA (Lossy Difference Aggregator)

7

Data structure

LDA (Lossy Difference Aggregator)

Figure 3: The Lossy Difference Aggregator (LDA) with n banks of m rows each.

Update procedure1. i ← h(x) // row2. j ← g(x) //bank sampling with pj

3. if j >0 then4. T[i, j] ← T[i, j] + τ5. S[i, j] ← S[i, j] + 16. end if

8

Hardware implementation

LDA (Lossy Difference Aggregator)

Figure 10: Potential LDA chip schematic

9

Validation◦ Use n = 1 bank of m = 1024 counters.◦ Simulate a 10-Gbps OC-192 link.

Evaluation

Figure 4: The sample size obtained by a single-bank LDA as a function of loss rate.

RL

m

1-1E[S]

S: expected sample sizem: number of rowsL: number of loss packetsR: number of received packets

10

Validation

Evaluation

Figure 5: Average relative error and 98% confidence bounds of the delay estimates computed by LDA as a function of loss rate. Actual mean delay is 0.2 μs in all cases. In (b), each curve represents an LDA with different random seed on the same trace.

(α , β)

Weibull distribution generated byPareto distribution generated by P(X ≤ x) = 1−(x/α)−β

α: scale parameter β : shape parameter

/1 xexXP

11

Validation

Evaluation

Figure 6: Average relative error of LDA’s standard-deviationestimator as a function of loss rate.

Weibull distribution generated byPareto distribution generated by P(X ≤ x) = 1−(x/α)−β

α: scale parameter β : shape parameter

/1 xexXP

(α , β)

12

Evaluation

Figure 7: The performance of various multi-bank LDA configurations.

13

Compare with active probes

Evaluation

Figure 8: Sample size, delay and standard deviation estimates obtained using a two-bank LDA in comparison with active probing at various frequencies. Log-scale axes.