XPRESS: A CROSS-LAYER BACKPRESSURE ARCHITECTURE FORWIRELESS MULTI-HOP NETWORKS

Rafael Laufer, Theodoros Salonidis, Henrik Lundgren, Pascal Le Guyadec, MobiCom2011

2011/12/19 study group by Neight

OUTLINE

Motivation XPRESS Backpressure Scheduling & Routing Cross-Layer Protocol Stack Contorl Plane Congestion control Optimal Schedule Computation

Wireless multi-hop networks operate below capacity Poor coordination across layers Poor coordination among transmitting

nodes How to achieve the network capacity?

MOTIVATION

NEW IDEA

A radical alternative by advocating cooperation among the multiple layers of the protocol stack

algorithm which, in theory, achieves the network capacity

NEW IDEA

• Backpressure scheduling & Routing

• Cross-Layer cooperation• Select optimal link set for

transmission• TDMA MAC protocol

NEW IDEA

XPRESS

Mesh access point (MAP) Sends queue lengths Executes the schedule Cross-layer protocol stack

Mesh controller (MC) Receives flow queue lengths Computes schedule Disseminates schedule

XPRESS VIEW

Internet

MC

MAP

GW

…

Frame k

DSCS

…… …

Practical challenges1. Time slots: TDMA MAC in multi-hop networks2. Link sets: Knowledge of non-interfering links3. Protocol overhead: Queue backlogs known at each

slot4. Computation overhead: Exhaustive search over links

sets5. Link scheduling: Backpressure schedules links, not

nodes6. Hardware constraints: Memory limitations at

wireless cards Backpressure so far a theoretical concept

No real system implementing backpressure to date

8

CHALLENGES

ALGORITHM

Backpressure Scheduling

Flow schedule and Routing For each link ( i,j ), select the flow with

the maximum queue differential backlog

Link Schedule Compute the weight of link as Select links to maximize

Transmit chosen flows on the selected links

BACKPRESSURE SCHEDULING & ROUTING

fjfi

fij qqw max ji,

),(

* maxargji

ijijμ

w

*,i jf

*, argmax f fi j i j

f Ff q q

BACKPRESSURE SCHEDULING & ROUTING

3

7

6

2

65

81

13

4

5

4

3

7

2

A B

C D

Transmission:During the time slot, a selected link ( i,j ) transmits a packet from flow using rate*

*,i jf

IMPLEMENT

Backpressure Scheduler

During frame , compute the schedule for frame

BACKPRESSURE SCHEDULER

MC

MAP Frame k Frame 1kCS

DS

Execution of

)(kS

Estimate)1( kS

)1( kS

)1( kQ

CS

DS

Compute

Execution of

Estimate)2( kSCompute

)1( kS

)(kS

)(kQ

k 1k

Challenge: compute optimal schedule per slot Knowledge of queue backlogs at each slot

Speculative scheduling: estimate queue backlogs

Challenge: schedule computation takes time

)1( kQ )2( kQ

IMPLEMENT

Cross-Layer cooperation

Per-LinkQueues

15

XPRESS CROSS-LAYER PROTOCOL STACK

A1

An

User Kernel Firmware Wireless

.

.

.

Time

PreQ FlowQ LinkQ

FlowClassifier

Congestion Control

PacketScheduler

Per-FlowQueues

LinkClassifier

Slot t+1

Flow Schedule

Link Schedule

PacketScheduler

Local

ForwardCongestion control ensures flow ratesare within the capacity regionFlow queues at the kernel address

the limited memory in the firmwareFlow scheduler enforces schedule and avoids overflows at the firmwareLink queues required for link

schedulingLink scheduler enforces schedule,

respecting TDMA slot boundaries

XPRESS CROSS-LAYER PROTOCOL STACK XPRESS flows:• General and can easily accommodate other flow definitions.• compared to the usual 5-tuple flow definition of source and

destination IP addresses, source and destination transport ports, and transport protocol (i.e., TCP or UDP),

• this design decision reduces processing and communication overhead in XPRESS at the expense of flow granularity

Congestion control and flow scheduling:• Each flow has two individual queues, namely, a pre-queue

(PreQ) and a flow queue (FlowQ)• Congestion control is performed according to and depends only

on the length of the local FlowQ. • A longer FlowQ reduces the allowed input rate, while a shorter

FlowQ allows a higher rate.

XPRESS CROSS-LAYER PROTOCOL STACK Link scheduling:• The MAC protocol keeps an individual queue for each neighbor

in order to enable link scheduling, which allows a higher spatial reuse than node scheduling.

• The slotted MAC, realized by a TDMA MAC protocol maintains network-wide node synchronization, and ensures that transmissions occur strictly within slot boundaries

Packet reception and forwarding:• Once a packet is received, it is first filtered based on the

destination MAC address and then inserted into a single receive queue (RxQ) at the firmware.

• The packet is delivered to the network layer at the kernel, where it is routed and tagged for local delivery or forwarding

IMPLEMENT

Contorl Plane

CONTORL PLANE

IMPLEMENT

Congestion Control

CONGESTION CONTROL

* 1

0argmax ( ) ' ( )

f

f ff f f f s f s

xx U x x q U q

fx

fsq

1' ( )ff sU q

(.)fUfsq

( ) *log( ), 100f f fU x k x k

A simple distributed congestion control algorithmwhere the source s of each flow f adjusts the flow rate as

where is the queue backlog for flow f at the source s and is the inverse of the first derivative of the utilityfunction at the point

In XPRESS

* / ff sx k q

MORE IMPLEMENT PROBLEM

Optimal Schedule Computation

For each slot, exhaustive search over all link sets Find link set which maximizes the sum of weights

Binary interference in TDMA MAC over 802.11 PHY Links have either low or high PDR Maximum weighted independent set (MWIS) MWIS computation takes 100 µs for testbed

23

OPTIMAL SCHEDULE COMPUTATION

Conflict graph

Finding the link transmission sets and their capacities is a challenge because each link capacity depends on both the channel condition and the interference created by the other links in the set. A direct approach requires O ( N r × 2L)

A DIRECT APPROACH

Define: The capacity μ ij of each link ( i,j ) is

estimated on a TDMA frame time scale as Pij × Rij

• Pij is the packet delivery ratio (PDR) and

• Rij is the PHY rate of link ( i,j ) during the TDMA frame.

BINARY INTERFERENCE EXPERIMENT

In order to understand how interference manifests in our TDMA system, we perform experiment for all link pairs that do not share a node in testbed

The links of each pair simultaneously transmit backlogged UDP traffic for 1 minute using broadcast packets at 24 Mbps PHY rate. During this time, the receivers measure the received signal strength (RSS) and PDR values at each TDMA frame

BINARY INTERFERENCE EXPERIMENT

BINARY INTERFERENCE

MORE IMPLEMENT PROBLEM

In Backpress scheduler: Binary interference MWISmaximum weight independent set (MWIS) computation in system due to binary interference. The MWIS computation is an NP-hardimplementation is based on an algorithm for enumerating maximal independent sets at the beginning of each frame. We then find the MWIS using a linear search over the independent sets. Heap structureFor efficiency, these sets could be stored in a heap structure keyed by their weights.At each slot, queue lengths change, which triggers a heap update. After the update, the new MWIS can be found as the root of the heap

MWIS

http://www.csie.ntnu.edu.tw/~u91029/Matching.html

GENERATE CONFLICT GRAPH

Use RSS measurement: This results in one RSS measurement per frame for

each link. Based on these measurements, we estimate the signal-to-interference ratio (SIR) of a link ( i,j ) under the interference of a transmitting node k as the difference S ij − S kj , between S ij , the RSS of link ( i,j ), and S kj , the RSS of link ( k,j ), both in dBm. If the SIR of the link exceeds a threshold, which depends on the PHY rate, the link PDR is estimated“high.”

GENERATE CONFLICT GRAPH

RSS BASE PROBLEM:• RSS values reported by 802.11 wireless cards

can be highly inaccurate due to the type of hardware, poor calibration, environmental conditions, location, temperature, multi-path effects, and external interference.

• it relies on the RSS of decoded packets and hence cannot detect hidden interferers which are within interference range, but not within communication range.

Solution:• use PDR measurements

IMPLEMENT CONFLICT GRAPH

The MC uses a conflict graph to represent interference in the network. A vertex vij in the conflict graph corresponds to the link ( i,j ) in the network graph.

An edge between vertices vij and vkl denotes interference between links ( i,j ) and ( k,l ) in either the DATA or ACK directions.

The vertex independent sets in the conflict graph correspond to the link transmission sets.

The conflict graph update mechanism is executed at each TDMA frame, after the MC receives the RSS and PDR measurements.

The conflict graph construction occurs in two stages.

CONFLICT GRAPH

STAGE1

If Sij is not measured, the transmission from i to j is estimated outside of communication and interference range.

MC creates a vertex vij in the conflict graph if S ij ≥ the RX sensitivity threshold of receiver j at PHY rate R . MC adds an edge between each pair of vertices

vij and vkl in the conflict graph if either they share a common node or if the SIR of DATA or ACK directions is less than the SIR receiver threshold at PHY rate R

Rj

Rj

STAGE2

For each link ( i,j ) selected by the first stage, the MC checks its reported PDR value Pij . If Pij ≥ 90%, the link remains in the conflict graph. Otherwise, MC finds hidden interferers.

The MC identifies the hidden interferers using the connectivity graph.

we assume that the hidden interferers of link ( i,j ) are those transmitters in Iij which are two-hop neighbors of either i or j . For each such node k , the algorithm adds an edge between vertex vij and vertex vkl in the conflict graph.

If link ( i,j ) fails again in the same set I ij during the next TDMA frame, the three-hop neighbors of i and j can be considered, until the hidden interferers are detected.

LIKE FOUR COLOR THEOREM

http://www.csie.ntnu.edu.tw/~u91029/Coloring.html

IF NEED SOME REFERENCE

http://caterpillar.onlyfun.net/Gossip/AlgorithmGossip/HeapSort.htm

Knowledge of interference to build conflict graph

Naive approach: measure each link set at all rates Measurement complexity

RSS measurements taken on each TDMA frame Control packets used to measure RSS Link RSS used to compute SIR threshold per PHY

rate Measurement complexity reduced to

RSS limited only to decoded packets PDR measurements also taken on each TDMA frame Detection of hidden interferers

38

INTERFERENCE ESTIMATION

)(NO

)2( LRO

Link queue:• Head-of-blocking• This problem may occur if we have

wireless losses and the packet at the head of the queue is destined for a different neighbor than the one assigned for the slot, resulting in no packet transmitted during that slot.

OTHER ISSUES

HEAD-OF-BLOCKING

當封包流入的總和比交換 switch 的速度快封包會被儲存在 input queue 等待交換。

Head-of-the-Line (HOL) blocking: 在等待交換的封包影響後來的封包進行交換 (queue 的特性 ) 。

擷取自計算機網路概論 課程投影片

Design and implementation of XPRESS First throughput-optimal backpressure system

Backpressure challenges addressed1. Time slots: multi-hop TDMA MAC & time

synchronization2. Link sets: RSS-based interference estimation3. Protocol overhead: Multi-slot framing and

speculation4. Computation overhead: Binary interference MWIS5. Link scheduling: Individual link queues at the MAC6. Hardware constraints: Network/MAC queue

coordination

41

CONTRIBUTIONS

42

802.11A INDOOR TESTBED

MAP node 1.6 GHz CPU, 512 MB RAM Linux OS / BP kernel module 802.11 Technicolor card (5 GHz) Customized firmware (TDMA/link scheduling)

Mesh controller 2.7 GHz CPU, 16 GB RAM

Ability of XPRESS to exploit multiple paths One flow between extreme nodes XPRESS allowed to use every link available 802.11 uses the shortest ETX path

43

MULTI-HOP: MULTI-PATH TOPOLOGY

http://nccur.lib.nccu.edu.tw/bitstream/140.119/32668/6/97101206.pdf

Throughput test Use Iperf

MULTI-HOP

45

MULTI-HOP: MULTI-PATH TOPOLOGY

Coordination & path diversity higher network throughput

Select short path (less hops) Delay is linear increase after meet

throughput

MULTI-HOP

47

QUEUE BACKLOG ESTIMATION ERROR

Accurate predictions XPRESS reaches network capacity

MWIS computation for optimal schedules In theory, MWIS is NP-hard In practice, polynomial with the number of links

48

OVERHEAD: COMPUTATION

49

OVERHEAD: COMPUTATION

MWIS computation is feasible for practical network sizes

Each frame Queue backlogs sent from the MAPs to the MC Computed schedule sent from the MC to MAPs

Time to exchange this on the control subframe

50

OVERHEAD: PROTOCOL

51

OVERHEAD: PROTOCOL

(50 nodes, 10 ms)

Control exchange feasible for practical network sizes

Design and implementation of XPRESS Cross-layer backpressure architecture First throughput-optimal backpressure scheduling XPRESS integrates backpressure with TDMA MAC

XPRESS achieves the network capacity High throughput gains in practice

Feasible for practical network sizes

52

CONCLUSIONS