Mobile Netw Appl (2009) 14:124–133

DOI 10.1007/s11036-008-0137-2

Harnessing Traffic Uncertainties in Wireless MeshNetworks—A Stochastic Optimization Approach

Yang Song · Chi Zhang · Yuguang Fang

Published online: 3 February 2009c© Springer Science + Business Media, LLC 2009

Abstract In this paper, we investigate the routing optimiza-tion problem in wireless mesh networks. While existingworks usually assume static and known traffic demand, weemphasize that the actual traffic is time-varying and difficultto measure. In light of this, we alternatively pursue a stochas-tic optimization framework where the expected networkutility is maximized. For multi-path routing scenario, wepropose a stochastic programming approach which requiresno priori knowledge on the probabilistic distribution of thetraffic. For the single-path routing counterpart, we developa learning-based algorithm which provably converges to theglobal optimum solution asymptotically.

Keywords wireless mesh networks · network utilitymaximization · routing

1 Introduction

Wireless mesh networks provide last mile broadband Internetaccess with low cost yet high bandwidth. The mesh routerscommunicate with each other via wireless links. Meanwhile,several edge mesh routers provide wireless access to theclient users where all the traffic is directed to the gateway

Y. Song (B) · C. Zhang · Y. FangDepartment of Electrical and Computer Engineering,University of Florida, Gainesville, 32611 FL, USAe-mail: yangsong@ufl.edu

C. Zhange-mail: zhangchi@ufl.edu

Y. FangNational Key Laboratory of Integrated Services Networks,Xidian University, Xi’an, Chinae-mail: fang@ece.ufl.edu

node. Due to the multi-hop nature, the routing in wirelessmesh networks is an important and interesting topic and hasattracted significant attention from the community [1].

However, the existing routing schemes in wireless meshnetworks usually assume that the traffic demands aggre-gated at edge routers are static and known. In practice,apparently, the instantaneous traffic demand may fluctuatedramatically due to the client users’ behavior and the mobil-ity of users. Therefore, the uncertainty of the traffic demandsat edge routers is not negligible and needs to be consideredmeticulously in wireless mesh networks.

Traffic estimation is proposed as a feasible solution toharness the traffic uncertainty. Several prediction models areproposed in the literature [2–4]. However, we emphasizethat the acquisition of accurate traffic information is by nomeans trivial and usually computationally expensive. Evenworse yet, the estimation techniques yield poor performancein a highly dynamic environment. Therefore, routing designin wireless mesh networks with uncertain traffic demand isremarkably challenging.

To circumvent the burden of route recalculation, oblivi-ous routing schemes are suggested [5–9]. A traffic-obliviousrouting protocol requires limited traffic information andachieves worst-case performance guarantee. In other words,by sticking to a fixed routing strategy, the worst-case perfor-mance is optimized. Therefore, the instability and prohibitiveoverhead of rerouting are avoided and the performance isacceptable if the traffic pattern varies within a certain range.However, one noticeable drawback of oblivious routingschemes is the computational complexity [9]. In addition,the optimization on the worst-case performance is usuallyover-conservative. Also, limited knowledge is attained whenthe traffic pattern varies drastically and exceeds the tolerancebound. In light of these concerns, we propose a stochasticoptimum routing strategy in wireless mesh networks where

Mobile Netw Appl (2009) 14:124–133 125

the expected network utility is optimized, rather than theworst-case performance. Our schemes differ from previouswork in several aspects. First, our algorithm does not requirea priori knowledge of the traffic distribution. Therefore, thecomputational overhead of traffic estimation can be avoided.Secondly, our scheme does not require that the traffic pat-tern varies within a certain range. Thirdly, our proposedalgorithms are amenable to decentralized implementations.

The remainder of this paper is organized as follows.Section 2 briefly outlines the related work. The system modelused in this paper is introduced in Section 3. The routingoptimization problems with multi-path and single-path rout-ing constraints are investigated in Section 4 and Section 5,respectively. A numerical example is provided in Section 6and Section 7 concludes this paper.

2 Related work

Routing plays a critical role in wireless mesh networks. Inthe literature, the routing problem is usually formulated asan optimization problem where heuristic or decomposition-based solutions are proposed. The existing works usuallyassume that the traffic input from each edge router is staticand known. Meanwhile, the empirical studies based on realtraffic traces reveal that the instantaneous traffic fluctuatesfrom time to time and is difficult to predict. To address theuncertainty of traffic demand in mesh networks, two lines ofresearch effort have been discussed. The first category, whichis in analogy of robust optimization techniques, achievesoptimal worst-case performance. Oblivious routing schemesfall into this category. The second category, which is in anal-ogy of the general stochastic optimization techniques, alter-natively pursues the expected utility maximization [10–14].Our work falls into this category. In [13], the power schedul-ing problem with time-varying channel is investigated. In[11], the impact of noisy feedback is analyzed, where the traf-fic demand is assumed to be sufficiently large and a rate con-trol algorithm, based on the dual decomposition approach,is implemented to adjust the ingress traffic. Nevertheless,in our work, the traffic demand is dynamic and may evenappear zero sometime. Another related work is [10] wherethe throughput maximization routing is considered. The traf-fic dynamic in wireless mesh networks is well addressed.However, the authors assume the traffic statistics are known,i.e., the probability distribution of random traffic demand isassumed to be a priori. In our scheme, such information isnot required, as will be clarified in following sections.

3 Model

We consider a wireless mesh network depicted in Fig. 1. Themesh network consists of several edge routers, intermediate

InternetGateway Node

Relay Router

Relay Router

Relay Router

Edge Router Edge RouterEdge Router

Fig. 1 Wireless mesh network topology

relay routers and a gateway node W which provides theconnection to the Internet. Each edge router provides wire-less access and is associated with a number of client users.Denote the set of edge routers as M and the aggregated traf-fic demand at the m-th edge router as zm . Naturally, zm isa random variable which is determined by the behaviors ofclient users. We assume that the instantaneous traffic demandat an edge router forms a stationary stochastic process. In thispaper, we restrain ourselves to wireless mesh networks witha single gateway node. The extension to multiple gatewaysis left as future work.

There are two types of routing protocols in the literature,i.e., arc-routing and path-routing. In arc-routing protocols,the decision variable is the amount of traffic allocated oneach link of the network, whereas the path-routing protocolsadjust the fraction of traffic allocated on each distinct path.We follow the latter approach due to its practical merit [7].For example, we can utilize the path diversity provided byOSPF/IS-IS and MPLS techniques. More specifically, OSPFdivides the traffic evenly among all paths with the same costwhereas MPLS enables more flexible manipulation on theflows on all available paths. In our scenario, one edge router,say m, has Pm acyclic paths to the gateway node W . Theinstantaneous traffic zm is distributed among all Pm pathsand each path possesses a fraction, denoted by rk

m where1

m = 1, · · · ,M and k = 1, · · · ,Pm . We have

Pm∑

k=1

rkm ≤ 1,∀m ∈ M. (1)

Define rm as the fraction vector and r = [r1, · · · , rM],which is the decision variable in our optimization framework

1 In this paper, we slightly abuse the notation by using the same symbolfor a set and its cardinality for notation brevity.

126 Mobile Netw Appl (2009) 14:124–133

throughout the paper. We define

δm =Pm∑

k=1

rkm,∀m ∈ M (2)

for notation succinctness.We denote the set of wireless links in the mesh network

as E . We assume in this paper that a scheduling scheme isavailable for the medium access where each link possessesa time share of the channel access. For each link e ∈ E ,there is a link cost, denoted by le, which is a function of theinstantaneous aggregated flow and the achievable data rateof the link, denoted by fe and ce, respectively. Note that ce

is given by

ce = c̃e × γe (3)

where c̃e denotes the nominal capacity of the link and γe is thefraction of time that link e is active following the scheduling.Without loss of generality, we assume the power of each linkis fixed.

Each edge router has a utility function, denoted by U (xm),which reflects the degree of satisfaction by transmitting a flowof xm = zm × δm . In light of the link costs, the surplus of them-th edge router, a.k.a., net reward, is given by

Om = Um(zm × δm) −Pm∑

k=1

zmrkm

∑

e∈Pkm

le( fe) (4)

where e ∈ Pkm represents all links along the k-th path of

Pm . We assume that the link cost function le( fe) is a non-decreasing and differentiable function of fe. Two such well-known examples are

le( fe) = 1

ce − fe(5)

which represents the average waiting time on link e, and

le( fe) = fe

(ce − fe)ce(6)

which reflects the average queueing time on link e, followinga model of M/M/1 [15]. Note that the cost is considered as+∞ if fe ≥ ce.

Define a topology matrix Hm for the m-th edge routerwhere the element Hm

e,k = 1 represents that link e is on thek-th path of Pm and zero otherwise. Therefore, we have

fe =M∑

m=1

Pm∑

k=1

Hme,k zmrk

m . (7)

Note that in wireless mesh networks, the edge routers andrelay routers are usually static. Therefore, the topology

matrix, i.e., Hm , is a fixed binary matrix and can be acquiredeasily. However, the values of fe, zm and le are randomdue the uncertain traffic demand, which makes the routingproblem more challenging. In Section 4, we formulate themulti-path routing optimization problem of wireless meshnetworks in a stochastic programming framework where adistributed algorithmic solution is derived. The single-pathrouting scenarios are investigated in Section 5, where alearning-based algorithm is introduced to achieve the globaloptimum solution asymptotically.

4 Multi-path routing scenario

In this section, we consider the cases where multi-path rout-ing is allowed for fault tolerance and load balancing purpose,i.e., the aggregated flow of each edge router is distributedamong several available paths.

In the network, each edge router has a surplus given byEq. 4. From the network’s perspective, our objective is tomaximize the overall surplus of the network, i.e.,

∑Mm=1 Om .

However, due to the randomness induced by the trafficdemand uncertainty, the overall surplus itself is a randomvariable. Therefore, we alternatively pursue a stochastic opti-mum solution of r∗, which maximizes the expected overallsurplus of the network. Mathematically, the routing problemin the wireless mesh network is formulated as

maxr

E

⎛

⎝M∑

m=1

Om

⎞

⎠

s.t.

E( fe) ≤ ce ∀e ∈ E (8)

Pm∑

k=1

rkm ≤ 1 ∀m ∈ M (9)

fe =M∑

m=1

Pm∑

k=1

Hme,k zmrk

m ∀e ∈ E (10)

Om = Um(zm × δm) −Pm∑

k=1

zmrkm

∑

e∈Pkm

le( fe) ∀m ∈ M

(11)

where E(.) is the expectation operator. We assume that therandom traffic demands of all edge routers can be discretizedinto arbitrarily many yet finite states, where each state isrepresented by s ∈ S. The time is slotted and within oneslot, the traffic demands on all edge routers correspond toone of the states and remain unchanged within the currentslot. Note that Om , fe, le and zm are all state-dependent vari-ables and hence we will add a superscript s in the followingformulations. Denote the stationary probability distribution

Mobile Netw Appl (2009) 14:124–133 127

of state s as πs . We can rewrite the routing optimizationproblem as

maxr

∑

s∈S

πs

⎛

⎝M∑

m=1

Osm

⎞

⎠

s.t.∑

s∈S

πs( f se ) ≤ ce ∀e ∈ E (12)

Pm∑

k=1

rkm ≤ 1 ∀m ∈ M (13)

f se =

M∑

m=1

Pm∑

k=1

Hme,k zs

mrkm ∀e ∈ E (14)

Osm = Um(zs

m × δm) −Pm∑

k=1

zsmrk

m

∑

e∈Pkm

lse( f s

e ) ∀m ∈ M

(15)

We can verify that if the stationary probability distribu-tion πs is known as a priori, the optimization problem is adeterministic convex optimization problem and the Slater’scondition is satisfied. However, the actual value of πs is eitherdifficult to measure in practice, or needs significant compu-tational overhead to estimate. Next, we propose a distributedsolution which converges to the global optimum solution yetrequires no information about the underlying probabilisticdistribution.

We first obtain the Lagrangian as

L(r, λ, μ) =∑

s∈S

πs

{∑

m∈MUm(zs

m × δm)

−∑

m∈M

∑

k∈Pm

zsmrk

m

⎛

⎝∑

e∈Pkm

lse( f s

e )

⎞

⎠

⎫⎬

⎭

+∑

m∈Mλm

⎛

⎝1 −∑

k∈Pm

rkm

⎞

⎠

+∑

e∈Eμe

(ce −

∑

s∈S

πs f se

)

=∑

s∈S

πs

{∑

m∈M(Um(zs

m × δm) + λm)

+∑

e∈Eμece −

∑

m∈M

∑

k∈Pm

×⎛

⎝zsmrk

m

⎛

⎝∑

e∈Pkm

(lse( f s

e ) + μe) + λmrkm

⎞

⎠

⎫⎬

⎭

(16)

Note that μ is the link congestion price similar as [16–18].The dual variables of λ ensure that the summation of fractionvariables is less or equal to unity. Define

Qs = supr

{∑

m∈M(Um(zs

m × δm) + λm) +∑

e∈Eμece

−∑

m∈M

∑

k∈Pm

(zsmrk

m

⎛

⎝∑

e∈Pkm

(lse( f s

e ) + μe) + λmrkm

⎞

⎠

⎫⎬

⎭

(17)

Therefore, the dual function is given by

g(λ, μ) =∑

s∈S

πs Qs where λ ≥ 0, μ ≥ 0. (18)

To achieve the minimum value of Eq. 18, or equivalently,to obtain the stochastic optimum value of r∗, we utilize thestochastic primal-dual approach [17]. The dual variables, i.e.,λ and μ are updated as

λm(n + 1) = [λm(n) − αm(n)ζm(n)]+ ∀m ∈ M (19)

μe(n + 1) = [μe(n) − αe(n)ξe(n)]+ ∀e ∈ E (20)

where [x]+ denotes max(x, 0).Similarly, the primal variable,i.e., rk

m , is updated as

rkm(n + 1) = [rk

m(n) + αm,k(n)ηm,k(n)]10 (21)

where [x]ba denotes max(min(b, x), a) and we use symbol

α(n) to represent the corresponding stepsizes, i.e., αm(n),αe(n) and αm,k(n), generally. Note that the instantaneouslink flow is considered as fixed when the fraction variable rk

mis updated.

The stochastic subgradients, i.e., ζm(n), ξe(n) andηm,k(n), can be attained by the Danskin’s theorem [19],following a similar approach as in [13, 14]

ζm(n) = 1 −∑

k∈Pm

rkm∀m ∈ M (22)

ξe(n) = ce −∑

m∈M

∑

k∈Pm

zsmrk

m Hme,k ∀e ∈ E (23)

ηm,k(n) = −⎡

⎣λm + zsm

⎛

⎝∑

e∈Pkm

(lse + μe)

⎞

⎠

⎤

⎦ ∀m ∈ M.

(24)

The distributed implementation of the algorithm is sum-marized as follows.

128 Mobile Netw Appl (2009) 14:124–133

Algorithm:

Repeat:

– Each link measures

ξe(n) = ce −∑

m∈M

∑

k∈Pm

zsmrk

m Hme,k . (25)

– Each link updates the link congestion price as

μe(n + 1) = [μe(n) − αe(n)ξe(n)]+. (26)

– Each edge router measures

ζm(n) = 1 −∑

k∈Pm

rkm (27)

and

ηm,k(n) = −⎡

⎣λm + zsm

⎛

⎝∑

e∈Pkm

(lse + μe)

⎞

⎠

⎤

⎦ . (28)

– Each edge router updates the fraction variable as

rkm(n + 1) = [rk

m(n) + αm,k(n)ηm,k(n)]10. (29)

– Each edge router updates the dual variable λ as

λm(n + 1) = [λm(n) − αm(n)ζm(n)]+. (30)

Until:

– The difference between successive iterations, i.e.,

ε = |rkm(n + 1) − rk

m(n)| (31)

is within a predefined convergence threshold ε′.

End

Note that the information needed in the algorithm is eitherlocally attainable or acquirable by the feedback along thepaths. Therefore, the algorithm is amenable to distributedimplementation. The convergence of the algorithm followsthe results of [11] and thus we provide the following theoremwithout proof.

Theorem 1 The algorithm converges to the global optimumwith probability one, provided that the following constraintsare satisfied [11, 20]: α(n) > 0,

∑∞n=0 α(n) = ∞, and∑∞

n=0(α(n))2 < ∞, ∀m ∈ M and e ∈ E , where α

generally represents all stepsize parameters in (19), (20)and (21).

5 Single-path routing scenario

In this section, we consider a wireless mesh network wheresingle-path routing strategy is adopted. Therefore, the frac-tion variable rk

m is a binary number in this scenario. Morespecifically, rk

m = 1 denotes that the m-th edge router selectsthe k-th path to deliver traffic and the fraction variables onother paths are zeros. The integral property of the fractionvariables complicates the routing optimization problem. Wefirst express the routing optimization problem, with singlepath routing constraint, as

maxr

∑

s∈S

πs

⎛

⎝M∑

m=1

Osm

⎞

⎠

s.t.∑

s∈S

πs( f se ) ≤ ce ∀e ∈ E (32)

Pm∑

k=1

rkm ≤ 1 ∀m ∈ M (33)

f se =

M∑

m=1

Pm∑

k=1

Hme,k zs

mrkm ∀e ∈ E (34)

Osm = Um(zs

m × δm) −Pm∑

k=1

zsmrk

m

∑

e∈Pkm

lse( f s

e ) ∀m ∈ M

(35)

rkm ∈ {0, 1} (36)

Apparently, this is a stochastic integer programming whichis difficult and computationally demanding to solve. Fora survey on the algorithmic solutions of stochastic integerprogramming problems, refer to [21]. Distinguishing fromprevious work, we next propose a learning-based algorithm,which asymptotically converges to the global optimum of theaforementioned stochastic integer programming problem.First, we briefly overview the learning automata technique,based on which our algorithm is proposed.

5.1 Learning automata

Learning automata techniques have been broadly investi-gated in the networking community [22–27]. As one ofthe stochastic learning schemes, learning automata was firstintroduced in the control community for stochastic systems.As depicted in Fig. 2, the basic single user scenario wherelearning automata techniques can be applied consists of arandom environment, a set of finite actions and a rationaldecision maker. At a time instance, the decision maker selectsone of the actions according to the selection probability

Mobile Netw Appl (2009) 14:124–133 129

Random Environment

Decision Maker

One of the actions

Random Output

?

Fig. 2 Structure of a single user learning automata

vector p. The random environment responds with a stochas-tic output based on which the learning algorithm updatesthe selection probability vector and the iteration continues.In a stationary random environment, the standard learningautomata algorithms asymptotically converge to an actionwhich is stochastically optimal in the sense that the expectedobjective is maximized [28].

In our scenario, each edge router is an independent deci-sion maker. The action space corresponds to the availablepaths of each edge router. Meanwhile, the random envi-ronment is the uncertain traffic demands at all edge routersjointly. To incorporate a multi-agent learning scenario, weform all M edge routers as a learning team, which targets tothe stochastic optimum solution collectively. In addition, weutilize the gateway nodeW of the wireless mesh network as ateacher. After each decision maker selects an action, the gate-way node multicasts a feedback signal, denoted by β, basedon which each edge router updates the selection probabilityvector and the algorithm iterates until convergence.

5.2 Learning-based algorithm

Each edge router, say m, maintains a selection probabilityvector pm and an inner-state vector um . We use the notation(m, k) to indicate the k-th path of the m-th edge router, i.e.,the k-th action from the action space of the decision maker m.

At the initialization phase, the value of um is randomlygenerated and pm = [1/Pm, · · · , 1/Pm]. At time instancen, the algorithm executed at the m-th edge router is describedas follows.

Algorithm:

Repeat:

– Selects a routing path from Pm , say j , according tothe selection probability vector pm(n) and starts thetransmission.

– After receiving the feedback signal from the gatewaynode W , denoted by β(n), the inner state vector um isupdated as

um,k(n + 1) =[

um,k(n) + θ(n)β(n)

×(

1 − eum,k

∑k∈Pm

eum,k

)

+ √θ(n)ωm,k(n)

]L

0

, for k = j (37)

um,k(n + 1) =[um,k(n) + √

θ(n)ωm,k(n)]L

0

for k = j (38)

Recall that j denotes the selected action, i.e., the chosenpath.

– The selection probability vector pm is then updated,following

pm,k = eum,k

∑k∈Pm

eum,k∀k ∈ Pm . (39)

Until:

– max(pm(n)) > B where B is a predefined convergencethreshold.

In the algorithm, θ(n) is the learning parameter of thealgorithm satisfying 0 < θ(n) < 1. L is a positive numberwhich keeps the inner state value bounded. The sequence ofωm,k(n) is a set of random variables with zero mean anda variance of σ 2(n). The global feedback signal β(n) iscalculated by the gateway node W as

β(n) =∑M

m=1 Osm

J (40)

where J is a sufficiently large number to normalize the out-put. In other words, the value of β(n) is deliberately tunedwithin [0, 1]. We emphasize that the introduced noise param-eter ω restrains the algorithm from being trapped in an ineffi-cient equilibrium. Note that the value ofβ(n) can be informedby efficient multicast algorithms, e.g., [29], initiated by thegateway node W . The team learning is then executed ina decentralized fashion. The steady state behavior of thelearning-based algorithm is given in the following theorem.

Theorem 2 The proposed learning-based algorithm con-verges to the global optimum solution of the single-pathrouting optimization problem with probability one, if the fol-lowing conditions are satisfied [30]: (1) limn→∞ θ(n) = 0and (2) limn→∞ σ(n) = 0.

130 Mobile Netw Appl (2009) 14:124–133

Proof The proof generalizes the results in [30]. First, weobserve that the set of edge routers implicitly form an N-person game. Moreover, according to Eq. 40, all the playerscollectively maximize a common utility function followingthe global feedback from the gateway node. Therefore, thegame can be viewed as an identical interest game. Note thatthe strategy space of each player is simply the routing paths,which are determined by the path selection probability vec-tors. Following Eq. 39, we hereby utilize the inner state vectoru as the variable in our analysis.

We first define an indicator function sm,k = 1 representingthe event that the m-th edge router selects the k-th path in Pm

and sm,k = 0 otherwise. Note that

∂E(β|u)

∂um,k= ∂

∑k pm,kE(β|u, sm,k)

∂um,k

=∑

k

eum,k

∑k eum,k

(1 − eum,k

∑k eum,k

)E(β|u, sm,k).

Also,

E

(β

(1 − eum,k

∑k eum,k

)|u

)

=∑

k

pm,k

(1 − eum,k

∑k eum,k

)E(β|u, sm,k)

=∑

k

eum,k

∑k eum,k

(1 − eum,k

∑k eum,k

)E(β|u, sm,k)

= ∂E(β|u)

∂um,k.

Therefore, from [31] (Chap.6), we conclude that the abovedynamic weakly converges to the following SDE

du = ∇E(β|u) + σdQ (41)

for a sufficiently small θ → 0 where Q is a standard WienerProcess. Note that the SDE (Eq. 41) falls into the category ofLangevin Eq. [32] which is well-known that the probabilitymeasure concentrates on the global maximum solution ofthe objective function, i.e., E(β|u), for a sufficiently smallσ → 0 [30, 32]. Therefore, we conclude that in this identicalinterest game, the aforementioned learning-based algorithmwill converge to the global optimum of the objective functionasymptotically and thus completes the proof. �

6 Numerical example

We consider an illustrative wireless mesh network shown inFig. 3. Among all available paths, we assign a set of paths

1

2 3 4

5 6 7

Gateway

Relay Router

Edge Router

TrafficTrafficTraffic

Fig. 3 Illustration of Available Paths

for each edge router and the corresponding links2 are shownin Fig. 3.

The gateway node is indicated as node 1. Three edgerouters are marked as node 5, 6 and 7, respectively, whichconsistently deliver the aggregated traffic to the gatewaynode. The available paths for each edge router are providedin Table 1.

Note that each edge router has two acyclic paths to reachthe destination. The link cost function follows Eq. 5. The traf-fic is randomly generated at each edge router with Gaussiandistribution. The mean is 5 and the variance is 1. In addition,we limit the traffic to be in the range of [0, 10]. We emphasizethat these settings do not involve any loss of generality. Bysetting the traffic of each router with identical statistical char-acteristics, the stochastic optimum solution coincides withthe load-balancing solution. The computational difficulty ofcalculating the global optimum of the stochastic integer pro-gramming problem can be avoided. Therefore, we can easilyverify the efficacy of our proposed algorithms. In addition,the actual achievable data rate is a function of the transmis-sion power and the scheduling algorithm. Without loss ofgenerality, we assume that the achievable data rate of eachlink is 10. We utilize the same stepsize for the stochasticgradient approach in Eqs. 19, 20 and 21, which is inverselyproportional to the number of iterations.

Table 1 Available paths for each edge router

Edge router 5 P15 : {5 → 2 → 1}

P25 : {5 → 3 → 1}

Edge router 6 P16 : {6 → 2 → 1}

P26 : {6 → 4 → 1}

Edge router 7 P17 : {7 → 3 → 1}

P27 : {7 → 4 → 1}

2 Note that the actual network topology can be much larger.

Mobile Netw Appl (2009) 14:124–133 131

We first investigate the multi-path routing scenario whereeach edge router can divide the incoming traffic on two avail-able paths. Restated, it is straightforward to verify that, giventhe current settings of link achievable rates and statisticalcharacteristics, the stochastic optimum solution coincideswith the load-balancing solution. In other words, the solu-tion of r1

5 = r25 = r1

6 = r26 = r1

7 = r27 = 1

2 maximizes theexpected overall network surplus and thus solves the routingoptimization problem. This observation is verified in Fig. 4.

Figure 4 depicts the evolution of 6 fraction variables ofedge routers where pair m − k denotes rk

m . At initialization,we manually set the fraction variables as r1

5 = 1, r25 = 0,

r16 = 1, r2

6 = 0 and r17 = 0, r2

7 = 1 which is remarkablybiased. Then, the distributed algorithm derived in Section 4is executed. As pictorially shown by Fig. 4, all 6 fraction vari-ables evolve with the iterations and converge to the stochasticoptimum solution gradually.

Next, we investigate the single-path routing scenario.The network settings are the same as in multi-path scenar-ios except that each edge router can only select one of theavailable paths to transmit. The learning parameter is set toθ(n) = 1/n where n is the index of the current iteration. Thenoise parameters are zero mean Gaussian random variableswith diminishing variances, e.g., σ(n) = 1/n. The boundingparameter L is 100 and J is 100. The convergence thresholdB is 0.9999. The learning-based algorithm is executed untileach edge router sticks to one of the available routing paths.The convergence behavior of the learning-based algorithm isdemonstrated in Fig. 5.

In Fig. 5, we denote pair m − k as the probability that them-th edge router picks the k-th path for next iteration. Forexample, 5 − 1 represents the probability that edge router5 selects the first path, i.e., P1

5 . As illustrated in Fig. 5,5 − 1, 6 − 2 and 7 − 1 promptly approach to unity whileothers diminish to null. From Fig. 3, it is apparent that sucha routing strategy is indeed a global optimum solution of

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Iteration

Fra

ctio

n V

aria

bles

515261627172

Fig. 4 Convergence of fraction variables

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Iteration

Pro

babi

lity 52

5161627172

Fig. 5 Convergence of selection probability vectors

the routing optimization problem with single-path routingconstraints. The learning-based algorithm finds this globaloptimum solution effectively. Note that this routing strat-egy is not the only global optimum solution. For instance,(p2

5 = 1, p16 = 1, p2

7 = 1) is another global optimumsolution due to the statistical symmetry of the network. Byadjusting the initial conditions, the learning algorithm findsother global optimum routing strategies as well. Duplicateresults are omitted.

7 Conclusions and future work

In this paper, we consider the routing optimization problemin wireless mesh networks under uncertain traffic demands.For multi-path routing scenario, we investigate the problemin a stochastic programming framework and a distributedalgorithmic solution is derived. For the single-path routingscenario, the problem is formulated as a stochastic integerprogramming where a learning-based algorithm is proposed.

In our work, we consider a single gateway in the wirelessmesh network. The extension to the multiple gateway scenar-ios remains as future research. In addition, the analysis forthe performance of our schemes under non-stationary trafficdemand is challenging and needs further investigation.

Acknowledgments This work was supported in part by the U.S.National Science Foundation under Grant DBI-0529012 and underGrant CNS-0721744. The work of Fang was also partially supportedby Changjiang Scholar Chair Professorship and the 111 Project underB08038.

References

1. Akyildiz IF, Wang X (2005) A survey on wireless mesh networks.IEEE Commun Mag 43(9):S23–S30

2. Zhang Y, Roughan M, Lund C, Donoho D (2003) An information-theoretic approach to traffic matrix estimation. ACM Sigcomm,pp 301–302

132 Mobile Netw Appl (2009) 14:124–133

3. Soule A, Lakhina A, Taft N, Papagiannaki K, Salamatian K,Nucci A, Crovella M, Diot C (2005) Traffic matrices: balanc-ing measurements, inference and modeling. ACM Sigmetrics,pp 362–373

4. Medina A, Taft N, Salamatian K, Bhattacharyya S, Diot C (2002)Traffic matrix estimation: existing techniques and new directions.ACM Sigcomm, pp 161–174

5. Applegate D, Cohen E (2003) Making intra-domain routing robustto changing and uncertain traffic demands: understanding funda-mental tradeoffs. ACM Sigcomm, pp 313–324

6. Wang H, Xie H, Qiu L, Yang YR, Zhang Y, Greenberg A (2006)Cope: traffic engineering in dynamic networks. ACM Sigcomm,pp 99–110

7. Li Y, Bai B, Harms J, Holte R (2007) Multipath oblivious routingfor Traffic Engineering - stable and robust routing in changingand uncertain environments. International teletraffic congress (ITC2007), 17–21 June 2007, pp 129–140

8. Li Y, Harms J, Holte R (2006) Traffic-oblivious energy-aware rout-ing for multihop wireless networks. Proc IEEE Infocom, pp 1–12

9. Wang W, Liu X, Krishnaswamy D (2007) Robust routing andscheduing in wireless mesh networks. IEEE SECON, pp 471–480

10. Dai L, Xue Y, Chang B, Cao Y, Cui Y (2008) Integrating trafficestimation and routing optimization for multiradio multi-channelwireless mesh networks. Proc IEEE Infocom, pp 71–75

11. Zhang J, Zheng D, Chiang M (2008) The impact of stochastic noisyfeedback on distributed network utility maximization. IEEE TransInf Theory 54(2):645–665

12. Lee JW, Mazumdar RR, Shroff NB (2007) Joint opportunisticpower scheduling and end-to-end rate control for wireless ad-hocnetworks. IEEE Trans Veh Technol 56:801–809, March

13. Lee JW, Mazumdar RR, Shroff NB (2006) Opportunistic powerscheduling for dynamic multi-server wireless systems. IEEE TransWirel Commun 5:1506–1515, June

14. Lee JW, Mazumdar RR, Shroff NB (2004) Opportunistic powerscheduling for multi-server wireless systems with minimum per-formance constraints. Proc IEEE Infocom 2:1067–1077

15. Kleinrock L (1975) Queueing systems, vol 1, 1st edn—theory.Wiley, New York

16. Chiang M (2004) To layer or not to layer: balancing transport andphysical layers in wireless multihop networks. Proc IEEE Infocom,pp 2525–2536

17. Chiang M, Low SH, Calderbank AR, Doyle JC (2007) Layeringas optimization decomposition: a mathematical theory of networkarchitectures. Proc IEEE 95:255–312, March

18. Kelly F, Maulloo A, Tan D (1998) Rate control in communicationnetworks: shadow prices, proportional fairness and stability. J OperRes Soc 49:237–252

19. Bertsekas DP (1999) Nonlinear programming. Athena Scientific,Nashua

20. Kall P, Wallace SW (1994) Stochastic programming. Wiley,New York

21. van der Vlerk MH Stochastic integer programming bibliography(1996–2007). http://mally.eco.rug.nl/biblio/sip.html

22. Xing Y, Chandramouli R (2007) Price dynamics in competitiveagile spectrum access markets. IEEE J Sel Areas Commun (JSAC)25:613–621, March

23. Haleem MA, Chandramouli R (2005) Adaptive downlink schedul-ing and rate selection: a cross layer design. IEEE J Sel AreasCommun 23:1287–1297, June

24. Haleem MA, Chandramouli R (2004) Adaptive transmission rateassignment for fading wireless channels with pursuit learningalgorithm. In: Proceedings of CISS, Princeton

25. Kiran S, Chandramouli R (2003) An adaptive energy-efficient linklayer protocol using stochastic learning control. IEEE Int ConfCommun (ICC) 2:1114–1118

26. Xing Y, Chandramouli R (2004) Distributed discrete power controlfor bursty transmissions over wireless data networks. IEEE Int ConfCommun (ICC) 1:139–143

27. Song Y, Fang Y, Zhang Y (2007) Stochastic channel selection incognitive radio networks. IEEE Globecom, pp 4878–4882

28. Thathachar MAL, Sastry PS (2002) Varieties of learning automata:an overview. IEEE Trans Syst Man Cybern 32:711–722, December

29. Zeng G, Wang B, Ding Y, Xiao L, Mutka M (2007) Multicastalgorithms for multi-channel wireless mesh networks. IEEE ICNP,pp 1–10

30. Thathachar MAL, Sastry PS (2004) Networks of learning automata.Kluwer Academic, Dordrecht

31. Kushner HJ (1984) Approximation and weak convergence methodsfor random processes. MIT, New York

32. Aluffi-Pentini F, Parisi V, Zirilli F (1985) Global optimization andstochastic difference equations. J Optim Theory Appl 47:1–26

Yang Song received his B.E. and M.E. degrees in Electrical Engi-neering from Dalian University of Technology, Dalian, China, andUniversity of Hawaii at Manoa, Honolulu, U.S.A., in July 2004 andAugust 2006, respectively. Since September 2006, he has been workingtowards the Ph.D. degree in the Department of Electrical and ComputerEngineering at the University of Florida, Gainesville, Florida, USA.His research interests are wireless network, game theory, optimizationand mechanism design. He is a student member of IEEE a member ofGame Theory Society.

Chi Zhang received the B.E. and M.E. degrees in Electrical Engineer-ing from Huazhong University of Science and Technology, Wuhan,China, in July 1999 and January 2002, respectively. Since September2004, he has been working towards the Ph.D. degree in the Departmentof Electrical and Computer Engineering at the University of Florida,

Mobile Netw Appl (2009) 14:124–133 133

Gainesville, Florida, USA. His research interests are network anddistributed system security, wireless networking, and mobile com-puting, with emphasis on mobile ad hoc networks, wireless sensornetworks, wireless mesh networks, and heterogeneous wired/wirelessnetworks.

Yuguang Fang received a Ph.D. degree in Systems Engineering fromCase Western Reserve University in January 1994 and a Ph.D degreein Electrical Engineering from Boston University in May 1997. He was

an assistant professor in the Department of Electrical and ComputerEngineering at New Jersey Institute of Technology from July 1998to May 2000. He then joined the Department of Electrical and Com-puter Engineering at University of Florida in May 2000 as an assistantprofessor, got an early promotion to an associate professor with tenure inAugust 2003 and to a full professor in August 2005. He holds a Univer-sity of Florida Research Foundation (UFRF) Professorship from 2006to 2009 and a Changjiang Scholar Chair Professorship with NationalKey Laboratory of Integrated Services Networks, Xidian University,China, from 2008 to 2011. He has published over 200 papers in ref-ereed professional journals and conferences. He received the NationalScience Foundation Faculty Early Career Award in 2001 and the Officeof Naval Research Young Investigator Award in 2002. He is the recipientof the Best Paper Award in IEEE International Conference on NetworkProtocols (ICNP) in 2006 and the recipient of the IEEE TCGN BestPaper Award in the IEEE High-Speed Networks Symposium, IEEEGlobecom in 2002. Dr. Fang is also active in professional activities.He is a Fellow of IEEE and a member of ACM. He has served on sev-eral editorial boards of technical journals including IEEE Transactionson Communications, IEEE Transactions on Wireless Communications,IEEE Transactions on Mobile Computing and ACM Wireless Networks.He has been actively participating in professional conference organiza-tions such as serving as the Steering Committee Co-Chair for QShine,the Technical Program Vice-Chair for IEEE INFOCOM’2005, Tech-nical Program Symposium Co-Chair for IEEE Globecom’2004, and amember of Technical Program Committee for IEEE INFOCOM (1998,2000, 2003–2009).