WiScape - UW Computer Sciences User Pagespages.cs.wisc.edu/~yoonj/wiscape-talk.pdfMotivation • One way to monitorwireless performance ... monitoring of wide-area networks JongwonYoon

WiScapeA Case for Client-Assisted Approach

to Monitoring Wide-Area Wireless Networks

Sayandeep Sen, Jongwon Yoon, Joshua Hare,Justin Ormont, and Suman Banerjee

University of Wisconsin-Madison

Jongwon Yoon WiScape / IMC 2011

1

Motivation

• One way to monitor wireless performance

- Carrying out drive-by measurement test


2

Labor intensive

Motivation




3

Does not scale efficientlyfor large-area networks

Motivation




4

Client Assisted Approach

Client-Assisted Monitoring

• Collecting measured samples from multiple clients


5



MeasurementRequests


6

Server



MeasuredSamples


7

Benefits

• Leverages multiple users

- Covers more area more frequently

• Captures client experiences

• At locations clients care about

• Helps network operators

- Improves network performance

- Detects connectivity holes

→ in turn helps users


8

Challenges of Client-Assisted Monitoring


9


Measurements discrete in space

Aggregate in SPACE


10


in SPACE(Zone)?


Battery


11

Data Usage(Mbytes)


Aggregate in TIME(Epoch)?

Measurements discrete in time


12





13

Can we compose

measurements?

Measurements not from a single client


Aggregate in TIME

Measurements discrete


Aggregate in SPACE(Zone)?


14



Can we compose

measurements?


WiScape• Characterize the Wireless landscape using client-assistance

• Using small and infrequent amounts of measurement collected by different users

80 packets/ 1hr

100 packets/ 75mins


15

80 packets/ 1hr

120 packets/ 2hrs

90 packets/ 75mins

90 packets/ 80mins

100 packets/ 75mins

WiScapeServer

WiScape• Characterize the Wireless landscape using client-assistance

• Using small and infrequent amounts of measurement collected by different users

Corollary:


16

Corollary: Client-assisted monitoring can capture coarse grained (~1hr) network property

Contributions

• Present a framework for coarse grained monitoring system

• Present applications of coarse grained monitoring of wide-area networksmonitoring of wide-area networks


17

Outline

• Motivation

• Dataset

• Low overhead client-assisted monitoring

– Aggregation in space– Aggregation in space

– Aggregation in time

– Composing of measurements

• Applications

• Related work & Conclusion


18

Outline

• Dataset


19

Data Collection Methodology

Measurement Metrics1) TCP/UDP

Throughput2) Jitter

Net A

Net B


20

CellularBase Stations

Client

Throughput2) Jitter3) UDP Loss rate4) LatencyNet C

Snapshot of TCP performance


21

- City-wide area, NetB. Circle represents 1.1㎢ area

Vehicular Dataset

155 ㎢ city-widearea in MadisonWI


22

- Madison Metro buses for 11 month- Net B and C

Vehicular Dataset

240 km road stretch, Madison, WI to Chicago, IL

- Collected for 6 month- Net B and C

Madison, WI


23

Chicago, IL

Vehicular Dataset

20 km Road stretch, Madison, WI


24

- Active measurement. Collected over 3 month- Net A,B and C

Static & Proximity Dataset

New Brunswick, NJ

Princeton, NJ

Madison, WI

: Static


25

: Proximate

Static: 5 Locations in WI, 2 Locations in NJProximity: Vicinity of the static locations- 5 month in WI using Net A,B and C1 month in NJ using Net B and C

: Static

Dataset

Group Span Months Nets Location

Static 5 locations2 locations

51

A, B, CB, C

Madison, WINew Brunswick, Princeton, NJ

Proximity Vicinity of the static locations

51

A, B, CB, C

Madison, WINew Brunswick,


26

locations 1 B, C New Brunswick, Princeton, NJ

Vehicular 155 ㎢ city-wide 240 Km road155 ㎢ city-wide20km road stretch

6

123

B, C

BA, B, C

Madison, WIMadison-ChicagoMadison, WIMadison, WI

Outline

• Low overhead client-assisted monitoring

– Aggregation in space– Aggregation in space

– Aggregation in time

– Composing of measurements


27


Aggregate in TIME

Measurements discrete


Aggregate in SPACE(Zone)?


28



Can we compose

measurements?


Aggregating in SPACE (Zone)

Zone: a region which is small enough to ensure we have similar performance


29

to ensure we have similar performance but big enough to ensure sufficientmeasurements can be collected



30


Aggregate in SPACE(Zones)?

What zone size should be used?

Yes


31


250 meter zone radius 97% of the zone have relative std. dev. < 8%


32

97% of the zone have relative std. dev. < 8%

- TCP throughput for NetB, collected 155 ㎢ Madison(200~10000 measurements were collected per week at all zones)- Relative Standard Deviation = standard dev. of samples / mean of samples

WiScape Framework



Aggregate When to re-estimate

Yes

E.g., 250m radius

Yes


33


When to re-estimate the metric?

Yes

Measurement Metrics1) TCP/UDP throughput2) Jitter3) UDP Loss rate4) Latency

Aggregating in TIME (Epoch)

• 30 min bins (Coarse time scale)

- TCPthroughputWI

- Jitter, WI

Small degree of variation


34

- UDPThroughputWI

-UDP loss rateWI

• Static-WI/NJ, Net A, B and C

Small degree of variation at coarse time (30min) scale


• 10 sec bins (Fine time scale)

- Standard Deviation for 10sec/30min time bins

Net B - WI Net C - WI Net B - NJ Net C – NJCoarse30min

Fine10sec

Coarse30min

Fine10sec

Coarse30min

Fine10sec

Coarse30min

Fine10sec


35

30min 10sec 30min 10sec 30min 10sec 30min 10sec

TCP(Kbps)

33 102 36 96 126 408 167 414

UDP(Kbps)

39 82 38 94 153 429 182 365

Jitter(msec)

1.3 2.1 0.7 1.6 0.5 1.6 0.5 1.0

Loss(%)

~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0





Fine10sec

Coarse30min

Fine10sec

Coarse30min

Fine10sec

Coarse30min

Fine10sec


36


TCP(Kbps)

33 102 36 96 126 408 167 414

UDP(Kbps)

39 82 38 94 153 429 182 365

Jitter(msec)

1.3 2.1 0.7 1.6 0.5 1.6 0.5 1.0

Loss(%)

~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0High degree of variation at fine time (10sec) scale





Fine10sec

Coarse30min

Fine10sec

Coarse30min

Fine10sec

Coarse30min

Fine10sec


37


TCP(Kbps)

33 102 36 96 126 408 167 414

UDP(Kbps)

39 82 38 94 153 429 182 365

Jitter(msec)

1.3 2.1 0.7 1.6 0.5 1.6 0.5 1.0

Loss(%)

~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0Degree of variation is zone specific


• Zone specific epochs

- Coherence interval of a metric (e.g. TCP throughput,

Jitter, etc.) in a given zone.

- We re-estimate the metric once every time epoch


38


• Allan deviation

- Determining intervals for over which given metric is

most stable

- Finding the time (coherence) interval at which the difference between bins is minimumdifference between bins is minimum


39


• Allan deviation

Time duration with the lowest Allan dev.


40

Time duration with the lowest Allan dev. determines the Zone Specific Epoch


• Zone specific epochs


41

- Proximate-WI/NJ, UDP throughput, Net B

75 min, WI 15 min, NJ

WiScape Framework



Aggregate What time epoch

Yes

e.g., 250m radius

Yes


42


What time epoch should be used?

How many client-sourced measurements?

Zone specific, O(10s of min)

Can we compose

measurements?

Yes

Yes

WiScape Framework




Yes

e.g., 250m radius

Yes


43




Can we compose

measurements?

Yes

Yes

Zone specific: 100s of packets/ epoch (~10s of min)/zone

[details in paper]


WiScape Framework




Yes

1) Radius 2) For each zone:

calculate

Zone specific

Yes

e.g., 250m radius


44




calculate time epoch

3) For each zone:calculate packet quantity

Yes

Zone specific: 100s of packets/ epoch (~10s of min)/zone


Can we compose

measurements?Yes

Outline

• Applications


45

• Multi Network Interface Systems

Improving Multi-interface Applications

NetA,B and C


46

TCP T

hro

ughput

(kbps)

Zone1 2



NetB?NetA,B and C

NetA?


47

TCP T

hro

ughput

(kbps)

Zone1 2

Observations

TCP T

hro

ughput

NetA

NetB NetB

NetC

Persistently better performance

5%

95%


48

TCP T

hro

ughput

(kbps)

Zone1 2

NetB

NetC

NetA

NetB

Observations

• Persistently better performance for a zone


49

- 20 km road stretchMadison, WI

- Vehicular tracesMadison-Chicago




50

TCP T

hro

ughput

(kbps)

Zone1 2

NetA

NetB

NetCNetA

NetB

NetC



Proportionally use NetA more


51

TCP T

hro

ughput

(kbps)

Zone1 2

NetA

NetB

NetCNetA

NetB

NetC



Proportionally use NetC more


52

TCP T

hro

ughput

(kbps)

Zone1 2

NetA

NetB

NetCNetA

NetB

NetC



Multisim-WiScape and MAR-WiScape


53

TCP T

hro

ughput

(kbps)

Zone1 2

NetA

NetB

NetCNetA

NetB

NetC

Multisim-WiScape and MAR-WiScape(~30-40% improvement)

[details in paper]

Observations

• Identifying locations with variable performance

• Detecting flash crowd

- Game day,


54

- Game day, Camp Randall, Madison WI

Application for Network Operators

• Identifying regions of high overload quickly

• Detecting flash crowd

- Game day,


55

- Game day, Camp Randall, Madison WI

Detecting overall conditions quickly with small number of measurements

Outline

• Related work & Conclusions


56

Related Work

• Monitoring Cellular Networks

- Mark the Spot (AT&T)

- 3G Test (University of Michigan)

- RootMetrics


57

Future Work

• Full scalable deployment

• Monitoring dense deployments (NYC, LA, etc.)


58

Conclusion

• Present a framework for coarse grained monitoring of wide-area network

- Validate it with data (datasets is available at

www.cs.wisc.edu/~yoonj/wiscape/IMC11_Data.html)

• Applications

- Improving client performance

- Helping operators


59

Thank You!

• Questions?


60

Backup slides

• Backup


61


TCP T

hro

ughput

(kbps)


62

TCP T

hro

ughput

(kbps)

Timebin size


TCP T

hro

ughput

(kbps)

Average, Std.


63

TCP T

hro

ughput

(kbps)

Timebin size

30 min bins: Coarse time scale10 sec bins: Fine time scale


• Allan Deviation

TCP T

hro

ughput

10 min

5 5 5


64

5 10 15 20 25 30

TCP T

hro

ughput

(Mbps)

Time (min)

5 5

5 5

5


• Allan Deviation

TCP T

hro

ughput

5 min

5 5 5


65

5 10 15 20 25 30

TCP T

hro

ughput

(Mbps)

Time (min)

5 5 5


• Allan Deviation

TCP T

hro

ughput

15 min

5 10


66

5 10 15 20 25 30

TCP T

hro

ughput

(Mbps)

Time (min)

510

Documents

WiScape - UW Computer Sciences User Pagespages.cs.wisc.edu/~yoonj/wiscape-talk.pdfMotivation • One way to monitorwireless performance ... monitoring of wide-area networks JongwonYoon