View
1
Download
0
Category
Preview:
Citation preview
A Fundamental Approach for ProvidingService-Level Guarantees for Wide-Area Networks
by
Jeremy Bogle
Submitted to the Department of Electrical Engineering and ComputerScience
in partial fulfillment of the requirements for the degree of
Master of Engineering in Electrical Engineering and Computer Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2019
ยฉ Massachusetts Institute of Technology 2019. All rights reserved.
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Department of Electrical Engineering and Computer Science
May 24, 2019
Certified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Manya Ghobadi
Assistant ProfessorThesis Supervisor
Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Katrina LaCurts
Chair, Master of Engineering Thesis Commitee
2
A Fundamental Approach for Providing Service-Level
Guarantees for Wide-Area Networks
by
Jeremy Bogle
Submitted to the Department of Electrical Engineering and Computer Scienceon May 24, 2019, in partial fulfillment of the
requirements for the degree ofMaster of Engineering in Electrical Engineering and Computer Science
Abstract
To keep up with the continuous growth in demand, cloud providers spend millions ofdollars augmenting the capacity of their wide-area backbones and devote significanteffort to efficiently utilizing WAN capacity. A key challenge is striking a good balancebetween network utilization and availability, as these are inherently at odds; a highlyutilized network might not be able to withstand unexpected traffic shifts resultingfrom link/node failures.
I motivate this problem using real data from a large service provider and proposea solution called TeaVaR (Traffic Engineering Applying Value at Risk), which drawson financial risk theory to realize a risk management approach to traffic engineering(TE). I leverage empirical data to generate a probabilistic model of network failures,and formulate a Linear Program (LP) that maximizes bandwidth allocation to net-work users subject to a service level agreement (SLA). I prove TeaVaRโs correctness,and then compare it to state-of-the-art TE solutions with extensive simulations acrossmany network topologies, failure scenarios, and real-world traffic patterns. The re-sults show that with TeaVaR, operators can support up to twice as much throughputas other TE schemes, at the same level of availability.
I also construct a simulation tool that builds on my implementation of TeaVaRand simulates its usage in the data plane. This tool can be useful not only for testingTE schemes but also for capacity planning, as it allows network operators to see howtheir network is performing, where the bottlenecks are, and what kind of demandloads it can handle.
Thesis Supervisor: Manya GhobadiTitle: Assistant Professor
3
4
Acknowledgments
I would like to thank various people for their contributions and guidance on this
project: Ishai Menache and Michael Schapira for helping with the proofs in the op-
timization, Nikhil Bhatia, for helping with the evaluations, and Asaf Valadarsky for
doing preliminary work computing failure probabilities. Special thanks should be
given to my thesis advisor, Manya Ghobadi, for her professional guidance, project
vision, and continued support.
5
6
Contents
1 Introduction and motivation 13
1.1 Effects of failures in the WAN . . . . . . . . . . . . . . . . . . . . . . 14
1.2 Motivation for this work . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.3 My work: TeaVaR (Traffic engineering applying value at risk) . . . . 18
2 Probabilistic approach to risk management 19
2.1 Probabilistic risk management in finance . . . . . . . . . . . . . . . . 19
2.2 Probabilistic risk management in networks . . . . . . . . . . . . . . . 21
3 Optimization framework 25
3.1 Overview of WAN TE . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 TeaVaR: deriving the LP . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.1 Probabilistic failure model . . . . . . . . . . . . . . . . . . . . 27
3.2.2 Loss function . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2.3 Objective function . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2.4 Linearizing the loss function . . . . . . . . . . . . . . . . . . . 31
3.2.5 Routing (and re-routing) in TeaVaR . . . . . . . . . . . . . . 32
3.3 Scenario pruning algorithm . . . . . . . . . . . . . . . . . . . . . . . . 33
4 Evaluating TeaVaR 35
4.1 Experimental setting . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2 Throughput vs. availability . . . . . . . . . . . . . . . . . . . . . . . . 38
4.3 Achieved throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7
4.4 Mathematical guarantees with tunable ๐ฝ. . . . . . . . . . . . . . . . . 41
4.5 Impact of tunnel selection. . . . . . . . . . . . . . . . . . . . . . . . . 42
4.6 Robustness to probability estimates . . . . . . . . . . . . . . . . . . . 43
4.7 Sensitivity to scenario pruning . . . . . . . . . . . . . . . . . . . . . . 44
5 Simulating and visualizing results 47
5.1 Developing an API for the optimization . . . . . . . . . . . . . . . . . 48
5.2 Visualizing the output . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.3 Applications to capacity planning . . . . . . . . . . . . . . . . . . . . 49
6 Contributions 51
A Appendix 53
A.1 Calculating ๐ ๐๐ ๐ฝ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
A.2 Proof of theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
8
List of Figures
1-1 ๐ฟ๐๐๐2โs utilization is kept low to sustain the traffic shift when failures
happen. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1-2 (a) A network of three links each with 10Gbps bandwidth; (b) Under
conventional TE schemes, such as FFC [30], the total admissible traffic
is always 10Gbps, split equally between available paths. . . . . . . . . 16
1-3 (a) The same network as in Fig. 1-2(a), with added information about
link failure probabilities; (b) A possible flow allocations under TeaVaR
with total admissible traffic of 20Gbps 99.8% of time. . . . . . . . . . 17
2-1 An illustration of ๐ ๐๐ ๐ฝ(๐ฅ) and ๐ถ๐ ๐๐ ๐ฝ(๐ฅ). . . . . . . . . . . . . . . 20
3-1 A diagram of the scenario space pruning algorithm with assumptions
1) and 2). The algorithm recursively searches the tree depth first,
updating ๐๐ at each level until ๐๐ < ๐ upon which it regresses one level. 33
4-1 CDF of failure probabilities used in our experiments. The exact value
of ๐ in (a) is not shown due to confidentiality reasons. The shape and
scale parameters in (b) are 0.8 and 10โ4, respectively. . . . . . . . . . 36
4-2 Comparison of TeaVaR against various TE schemes under different
tunnel selection algorithms. All schemes in (a) and (b) use oblivious
paths, and all schemes in (c) and (d) use edge disjoint paths. The term
availability refers to the percentage of scenarios that meet the 100%
demand-satisfaction requirement. . . . . . . . . . . . . . . . . . . . . 38
9
4-3 Averaged throughput guarantees for different ๐ฝ values on ATT, B4,
and IBM topologies. TeaVaR is the only scheme that can explicitly
optimize for a given ๐ฝ. For all other schemes, the value on the x-axis
is computed based on achieved throughput. . . . . . . . . . . . . . . . 40
4-4 The effect of tunnel selection scheme on TeaVaRโs performance, quan-
tified by the resulting ๐ถ๐ ๐๐ ๐ฝ (or loss) for different values of ๐ฝ. TeaVaR
using oblivious paths has better performance (lower loss). . . . . . . . 42
4-5 Impact of scenario pruning on accuracy and runtime.I use 10โ4 as the
cutoff threshold for ATT and 10โ5 for all other topologies. (a) The
cutoff thresholds cover more than 99.5% of all possible scenarios. (b)
The error incurred by pruning scenarios is less than 5% in all cases.
(c) The cutoffs applied lead to manageable running times. . . . . . . 44
5-1 An image showing TE-VIS when two nodes are selected. The tunnels
between the two nodes along with their weights are displayed on the
top right, and the dashed line is the current selected path. . . . . . . 48
5-2 This image shows the utilization on each link. Utilization is defined as
the amount of traffic divided by that links total capacity. The labels in
red show where congestion would occur and traffic would be dropped
because of the link breaking capacity. . . . . . . . . . . . . . . . . . . 49
10
List of Tables
3.1 Key notations in the TeaVaR formulation. The original optimization
problem is minimizing (3.3) subject to (3.1) โ (3.2). Here, we show the
LP formulation; see ยง3.2 for its derivation. . . . . . . . . . . . . . . . 26
4.1 Network topologies used in the evaluations. For confidentiality reasons,
I do not report exact numbers for the X-Net topology. . . . . . . . . . 35
4.2 The mathematical bandwidth guarantees of TeaVaR and FFC aver-
aged across 3 topologies, and 10 demand matrices. Avg๐ represents the
average bandwidth for all flows, guaranteed with probability ๐% of the
time. Min๐ represents the minimum bandwidth of all flows. . . . . . . 41
4.3 The effect of inaccurate probability estimations on TeaVaRโs perfor-
mance. The decrease in throughout is caused by running TeaVaR
with the ground-truth probabilities {๐๐}. . . . . . . . . . . . . . . . . 43
11
12
Chapter 1
Introduction and motivation
Traffic engineering (TE), the dynamic adjustment of traffic splitting across network
paths, is fundamental to networking and has received extensive attention in a broad
variety of contexts [21, 23, 13, 7, 3, 2, 17, 19, 24, 39, 30, 27]. Given the high cost of
wide-area backbone networks (WANs), large service providers (e.g., Amazon, Face-
book, Google, Microsoft) are investing heavily in optimizing their WAN TE, leverag-
ing Software-Defined Networking (SDN) to globally optimize routing and bandwidth
allocation to users [17, 19, 30, 26, 29]. In addition, they have to purchase or lease
fiber to support wide-area connectivity between distant data center locations, and
they need to know how and when to purchase this fiber. The process of deciding
which fibers to buy and how to manage them is called capacity planning.
A crucial challenge faced by WAN operators is striking a good balance between
network utilization and availability in the presence of node/link failures [15, 33, 30,
5, 18]. These two objectives are inherently at odds; providing high availability re-
quires keeping network utilization sufficiently low to absorb shifts in traffic when
failures occur. To attain high availability, todayโs backbone networks are typically
operated at fairly low utilization so as to meet user traffic demands while providing
high availability (e.g., 99%+ [18]) in the presence of failures.
In this chapter, I review some previous work in the field of WAN failure analysis
and explain the motivations driving this work. Chapter 2 explains a similar problem
in financial risk modeling and portfolio optimization. Chapter 3 offers a solution to
13
Aug.
4
Aug.
6
Aug.
8 Rest of the yearAu
g. 2
Aug.
10
Aug.
12
Jul.
310
0.2
0.4
0.6
0.8
1
NormalizedUtilization
Link1Link2
Figure 1-1: ๐ฟ๐๐๐2โs utilization is kept low to sustain the traffic shift when failureshappen.
this problem where the loss in the network is cast as a linear program and minimized
as a TE optimization. Chapter 4 describes the evaluation setup and performance of
the solution proposed in Chapter 3. Chapter 5 covers the simulation and visualization
of results. In this chapter I also explain how this same TE approach can be used with
the simulation for capacity planning in an SDN.
1.1 Effects of failures in the WAN
Figure 1-1 plots the link utilization of two IP links in a backbone network in North
America with the same source location but different destinations. The utilization of
each link is normalized by the maximum achieved link utilization; hence, the actual
link utilization is lower than plotted. On August 4๐กโ, ๐ฟ๐๐๐1 failed, and its utilization
dropped to zero. This, in turn, increased the utilization of ๐ฟ๐๐๐2. Importantly,
however, under normal conditions, the normalized utilization of ๐ฟ๐๐๐2 is only around
20%, making ๐ฟ๐๐๐2 underutilized almost all the time.
This example illustrates a common problem with todayโs WANs. In Chapter 4,
I show how state-of-the-art TE schemes fail to maximize the traffic load that can
be supported by the WAN for the desired level of availability. Under these schemes,
the allocated bandwidth falls significantly below the available capacity, resulting in
14
needlessly low network utilization. A key insight of my research is that operators
should explicitly optimize network utilization subject to target availability thresholds.
Instead of explicitly considering availability, todayโs TE schemes use the number of
concurrent link/node failures the TE configuration can withstand (e.g., by sending
traffic on link-disjoint network paths) as a proxy for availability. However, the failure
probability of a single link can greatly differ across links, sometimes by three orders
of magnitude [14]. Consequently, some failure scenarios involving two links might
be more probable than others involving a single link. Alternatively, some failure
scenarios might have only negligible probability; thus, lowering network utilization to
accommodate them is wasteful and has no meaningful bearing on availability.
Fortunately, operators have high visibility into failure patterns and dynamics.
For example, it has been shown that link failures are more probable during working
hours [15] and can be predicted by sudden drops in optical signal quality, โwith a 50%
chance of an outage within an hour of a drop event and a 70% chance of an outage
within one dayโ [14]. I propose that this wealth of timely empirical data on node/link
failures in the WAN should be exploited to probe the probability of different failure
scenarios when optimizing TE.
1.2 Motivation for this work
In this section, I provide a more concrete example of my work motivation. As men-
tioned above, the number of concurrent node/link failures a TE configuration can
withstand is used as a proxy for availability. This can be manifested, e.g., in sending
user traffic on multiple network paths (tunnels) that do not share any, or share only
a few, links or in splitting traffic across paths in a manner that is resilient to a certain
number of concurrent link failures [30]. In what follows, I explain why reasoning
about availability in terms of the number of concurrent failures that can be tolerated
is often not enough, using a demonstration of the recently proposed Forward Fault
Correction (FFC) TE scheme [30] to substantiate my explanation.
FFC example. FFC maximizes bandwidth allocation to be robust against up to ๐
15
s d
10 Gbps
10 Gbps
10 Gbps
(a)
s d10/3 Gbps
10/3 Gbps
10/3 Gbps
(b)
Figure 1-2: (a) A network of three links each with 10Gbps bandwidth; (b)Under conventional TE schemes, such as FFC [30], the total admissible trafficis always 10Gbps, split equally between available paths.
concurrent link failures, for a configurable value ๐. To accomplish this, FFC opti-
mization formulation sets a cap on the maximum bandwidth ๐๐ each network flow ๐
(identified by source/destination pair) can utilize and generates routing (and rerout-
ing) rules such that the network can simultaneously support bandwidth ๐๐ for each
flow i in any failure scenario involving at most k failures.
Figure 1-2 shows an example of FFC, where the source node ๐ is connected to the
destination node ๐ via three links, each with a capacity of 10Gbps. Suppose that the
objective is to support the maximum total amount of traffic from ๐ to ๐ in a manner
that is resilient to, at most, two concurrent link failures. Figure 1-2(b) presents
the optimal solution under FFC: rate-limiting the (๐ , ๐)-flow to send at 10Gbps and
always splitting its traffic equally between all links that are intact; e.g., when no link
failures occur, traffic is sent at 103Gbps on each link, when a single link failure occurs,
each of the two surviving links carries 5Gbps, and with two link failures, all traffic
is sent on the single surviving link. Thus, this solution guarantees the flow reserved
bandwidth of 10Gbps without exceeding link capacities under any failure scenario
that involves at most two failed links. Observe, however, that this comes at the cost
of keeping each link underutilized (one third utilization) when no failures occur.
Striking the right balance. The question is whether high availability can be
achieved without such drastic over-provisioning. Approaches such as FFC are com-
pelling in that they provide strong availability guarantees; in Figure 1-2(b), the (๐ , ๐)
flow is guaranteed a total bandwidth of 10Gbps even if two links become permanently
unavailable. Suppose, however, that the availability, i.e., the fraction of time a link is
up, is consistently 99.9% for each of the three links. In this scenario, the network can
16
p(fail) = 10-3
p(fail) = 10-3
s d10 Gbps
10 Gbps
p(fail) = 10-110 Gbps
(a)
s d10 Gbps
10 Gbps
0 Gbps
(b)
Figure 1-3: (a) The same network as in Fig. 1-2(a), with added informa-tion about link failure probabilities; (b) A possible flow allocations underTeaVaR with total admissible traffic of 20Gbps 99.8% of time.
easily support 30Gbps throughput (3ร improvement over FFC) around 99.9% of the
time simply by utilizing the full bandwidth of each link and never rerouting traffic.
This example captures the limitation of failure-probability-oblivious approaches to
TE, such as FFC, namely that they ignore the underlying link availability (and the
derived probability of failure). As discussed elsewhere [15, 14], link availability greatly
varies across different links. Consequently, probability-oblivious TE solutions might
lead to low network efficiency under prevailing conditions to accommodate potentially
highly unlikely failure scenarios (i.e., with little bearing on availability). However, not
only might a probability-oblivious approach overemphasize unlikely failure scenarios,
it might even disregard likely failure scenarios. Consider a scenario where three links
in a large network have low availability (say, 99% each), and all other links have
extremely high availability (say, 99.999%). When the operatorโs objective is to with-
stand two concurrent link failures, the scenario where the three less available links
might be simultaneously unavailable will not be considered, but much less likely sce-
narios in which two of the highly available links fail simultaneously will be considered.
To explain the motivation for a risk-management approach, let us revisit the ex-
ample in Figure 1-2. Now, suppose the probability of a link being up is as described in
the figure, and the link failure probabilities are uncorrelated (I will discuss correlated
failures in Chapter 3). In this case, the probability of different failure scenarios can
be expressed in terms of individual linksโ failure probabilities (e.g., the probability
of all three links failing simultaneously is 10โ7). Under these failure probabilities,
the network can support 30Gbps traffic almost 90% of the time simply by utiliz-
ing the full bandwidth of each link and not rerouting traffic in the event of failures.
17
FFCโs solution, shown in Figure 1-2(b), can be regarded as corresponding to the
objective of maximizing the throughput for a level of availability in the order of 7
nines (99.99999%), as the scenario of all links failing concurrently occurs with prob-
ability 10โ7. Observe that the bandwidth assignment in Figure 1-3(b) guarantees a
total throughput of 20Gps at a level of availability of nearly 3 nines (99.8%).1 Thus,
the network administrator can trade network utilization for availability to reflect the
operational objectives and strike a balance between the two.
1.3 My work: TeaVaR (Traffic engineering applying
value at risk)
Instead of reasoning about availability indirectly in terms of the maximum number of
tolerable failures [30], network operators could generate a probabilistic failure model
from empirical data (e.g., encompassing uncorrelated/correlated link failures, node
failures, signal decay, etc.) and optimize TE with respect to an availability bound.
My collaborators and I implemented a TE optimization framework called TeaVaR.
This framework allows operators to harness the information required to fine-tune the
tradeoff between network utilization and availability, thus striking a balance that best
suits their goals. TeaVaR is the first formal TE framework that enables operators
to jointly optimize network utilization and availability. In the next chapter, I discuss
the idea for this framework; I explain how it relates to portfolio optimization and
financial risk modeling and show how this mindset can be applied to TE. In Chapter
3, I explain the Linear Program of TeaVaR in more detail.
Note that this approach to risk-aware TE is orthogonal and complementary to
the challenge of capacity planning. While capacity planning focuses on determining
in what manner WAN capacity should be augmented to provide high availability,
my goal is to optimize the utilization of available network capacity with respect to
real-time information about traffic demands and expected failures. I explain how this
approach can be applied to capacity planning in Chapter 5.1The probability of the upper and lower links both being up, regardless of the middle link, is (1โ10โ3)2 = 0.998.
18
Chapter 2
Probabilistic approach to risk
management
In this chapter, I relate the central concept of Value at Risk (VaR) in finance to
resource allocation in networks and, more specifically, to TE. I then highlight the
main challenges of and ideas underlying TeaVaRโa CVaR-based TE solution. A
full description of TeaVaR appears in Chapter 3.
2.1 Probabilistic risk management in finance
In many financial contexts, the goal of an investor is to manage a collection of assets
(e.g., stocks), also called a portfolio, so as to maximize the expected return on the
investment while considering the probability of possible market changes that could
result in losses or smaller-than-expected gains.
Consider a setting in which an investor must decide how many of each type of
stock to acquire by quantifying the return from the different investment possibilities.
Let ๐ฅ = (๐ฅ1, . . . , ๐ฅ๐) be a vector representing an investment, where ๐ฅ๐ represents the
amount of stock ๐ acquired, and let ๐ฆ = (๐ฆ1, . . . , ๐ฆ๐) be a vector randomly generated
from a probability distribution reflecting market statistics, where ๐ฆ๐ represents the
return on investing in stock ๐. In financial risk literature, vector ๐ฅ is termed the
control and vector ๐ฆ is termed the uncertainty vector. The loss function ๐ฟ(๐ฅ, ๐ฆ)
19
ฮพ = VaRฮฒ(x)Loss(x, y)
CVaRฮฒ(x) = E[Loss |Loss โฅ ฮพ]Pr
obab
ility
(x,y) A scenario
Figure 2-1: An illustration of ๐ ๐๐ ๐ฝ(๐ฅ) and ๐ถ๐ ๐๐ ๐ฝ(๐ฅ).
captures the return on investment ๐ฅ under ๐ฆ and is simply ๐ฟ(๐ฅ, ๐ฆ) = โฮฃ๐๐=1๐ฅ๐๐ฆ๐, i.e.,
the negative of the gain.
Investors wish to provide customers with bounds on the loss they might incur,
such as โthe loss will be less than $100 with probability 0.95,โ or โthe loss will be less
than $500 with probability 0.99.โ Value at Risk (VaR) [22] captures these bounds.
Given a probability threshold ๐ฝ (say ๐ฝ = 0.99), ๐ ๐๐ ๐ฝ provides a probabilistic upper
bound on the loss: the loss is less than ๐ ๐๐ ๐ฝ with probability ๐ฝ.
Figure 2-1 gives a graphical illustration of the concepts of ๐ ๐๐ ๐ฝ and ๐ถ๐ ๐๐ ๐ฝ
(described below). For a given control vector ๐ฅ and probability distribution on the
uncertainty vector ๐ฆ, the figure plots the probability mass function of individual
scenarios (๐ฅ, ๐ฆ), sorted according to the loss associated with each scenario. Assuming
all possible scenarios are considered, the total area under the curve amounts to 1. At
the point on the x-axis marked by ๐ =๐ ๐๐ ๐ฝ(๐ฅ), the area under the curve is greater
than or equal to ๐ฝ. Given a probability threshold ๐ฝ (say ๐ฝ = 0.99) and a fixed control
๐ฅ, ๐ ๐๐ ๐ฝ(๐ฅ) provides a probabilistic upper bound on the loss: the loss is less than
๐ ๐๐ ๐ฝ(๐ฅ) with probability ๐ฝ. Equivalently, ๐ ๐๐ ๐ฝ(๐ฅ) is the ๐ฝ-percentile of the loss
given ๐ฅ. Value at Risk (๐ ๐๐ ๐ฝ) is obtained by minimizing ๐ ๐๐ ๐ฝ(๐ฅ) (or ๐) over all
possible control vectors ๐ฅ, for a given a probability threshold ๐ฝ. The VaR notion
has been applied to various contexts, including hedge fund investments [36], energy
markets [10], credit risk [4], and even cancer treatment [31].
20
Note that ๐ ๐๐ ๐ฝ does not necessarily minimize the loss at the tail (colored in
red in Figure 2-1), i.e., the worst scenarios in terms of probability, which have total
probability mass of at most 1โ ๐ฝ. A closely related risk measure that does minimize
the loss at the tail is termed ๐ฝ-Conditional Value at Risk (๐ถ๐ ๐๐ ๐ฝ)[35]; ๐ถ๐ ๐๐ ๐ฝ
is defined as the expected loss at the tail, or, equivalently, as the expected loss of
all scenarios with loss greater or equal to ๐ ๐๐ ๐ฝ. VaR minimization is typically
intractable. In contrast, minimizing CVaR can be cast as a convex optimization
problem under mild assumptions [35]. Further, minimizing CVaR can be a good
proxy for minimizing VaR.
2.2 Probabilistic risk management in networks
Optimizing traffic flow in a network also entails contending with loss, which in this
context reflects the possibility of failing to satisfy user demands when traffic shifts as
link/node failures congest the network. This section presents a high-level overview of
how the VaR and CVaR can be applied to this context. See Chapter 3 for the formal
presentation of TeaVaR.
We can model the WAN as a network graph, in which nodes represent switches,
edges represent links, and each link is associated with a capacity. Links (or, more
broadly, shared risk groups) also have failure probabilities. As in prior studies [17,
30, 19], in each time epoch a set of source-destination switch-pairs (โcommoditiesโ or
โflowsโ) wish to communicate, where each such pair ๐ is associated with a demand ๐๐
and a fixed set of possible routes (or tunnels) ๐๐ on which its traffic can be routed.
Intuitively, under the formulation of TE optimization as a risk-management chal-
lenge, the control vector ๐๐,๐ก captures how much bandwidth is allocated to each flow
on each of its tunnels, and the uncertainty vector ๐ฆ specifies, for each tunnel, whether
the tunnel is available (i.e., whether all of its links are up). Note that ๐ฆ is stochastic,
and its probability distribution is derived from the probabilities of the underlying fail-
ure events (e.g., link/node failures). The aim is to maximize the bandwidth assigned
to users subject to a desired, operator-specified, availability threshold ๐ฝ.
21
However, applying ๐ถ๐ ๐๐ ๐ฝ to network resource allocation incurs three nontrivial
challenges.
Achieving fairness across network users. Avoiding starvation and achieving fair-
ness are arguably less pivotal in stock markets, because money is money no matter
where it comes from, but they are essential in network resource allocation. In par-
ticular, TE involves multiple network users, and a crucial requirement is that high
bandwidth and availability guarantees for some users not come at the expense of
unacceptable bandwidth or availability for others. In the formulation, this translates
into carefully choosing the loss function ๐ฟ(๐ฅ, ๐ฆ) so that minimizing the chosen notion
of loss implies that such undesirable phenomena do not occur. This also means we
must represent the control vector as 2-dimensional so that we can have a control
vector associated with every flow. I refer to the control vector as ๐๐,๐ก where ๐ refers
to the flows and ๐ก represents a tunnel for that flow. I show how these modifications
are accomplished in Chapter 3.
Capturing fast rerouting of traffic in the data plane. Unlike the above for-
mulation of stock management, in the TE context the consequences of the realiza-
tion of the uncertainty vector cannot be captured by a simple loss function, such as
๐ฟ(๐, ๐ฆ) = โฮฃ๐๐ก=1๐๐,๐ก๐ฆ๐ก because the CVaR-based optimization formalism must take into
account that the unavailability of a certain tunnel might imply more traffic having to
traverse other tunnels.
Providing high availability in WAN TE cannot rely on the online global re-
computation of tunnels as this can be too time consuming and adversely impact
availability [30, 38]. To quickly recover from failures, TeaVaR is required to re-
adjust the traffic splitting ratios on surviving tunnels via re-hashing mechanisms
implemented in the data plane [30, 38]. Thus, the realization of the uncertainty vec-
tor, which corresponds to a specification of which tunnels are up, impact the control,
capturing how much is sent on each tunnel.
Achieving computational tractability. A naive formulation of the CVaR-minimizing
TE machinery yields a non-convex optimization problem. Hence, the first challenge
is to transform the basic formulation into an equivalent convex program. We are,
22
in fact, able to formulate the TE optimization as a linear program through careful
reformulation using auxiliary variables. In addition, because the number of all pos-
sible failure scenarios increases exponentially with the network size, solving this LP
becomes intractable for realistic network sizes. To address this additional challenge,
I introduce an efficient pruning process that allows us to consider fewer scenarios.
I explain scenario pruning in ยง3.3 and we show how it substantially improves the
runtime with little effect on accuracy in Chapter 4.
23
24
Chapter 3
Optimization framework
In this chapter, I describe the TeaVaR optimization framework in detail. I formalize
the model and delineate the goals of WAN TE [19, 17, 27, 30] (ยง3.1). Then, I introduce
TeaVaRโs novel approach to TE, showing that it enables probabilistic guarantees on
network throughput (ยง3.2).
3.1 Overview of WAN TE
In this section, I give a brief overview of a typical approach to WAN TE. Table 3.1
shows these inputs, along with a the additional inputs and outputs of TeaVaR. The
table also shows the linear program for TeaVaR, which is described in more detail
later in this chapter.
Input. As in other WAN TE studies, I model the WAN as a directed graph ๐บ =
(๐,๐ธ), where the vertex set ๐ represents switches and edge set ๐ธ represents links
between switches. Link capacities are given by ๐ถ = (๐1, . . . , ๐|๐ธ|) (e.g., in bps) and
as in any TE formulation, the total flow on each link should not exceed its capacity.
TE decisions are made at fixed time intervals (say, every 5 minutes [17]), based on
the estimated user traffic demands for that interval. In each time epoch, there is a
set of source-destination switch-pairs (โcommoditiesโ or โflowsโ); each such pair ๐ is
associated with a demand ๐๐ and a fixed set of paths (or โtunnelsโ) ๐๐ โ ๐ on which
its traffic should be routed. TeaVaR assumes the tunnels are part of the input.
25
TE Input
๐บ(๐,๐ธ) Network graph with switches ๐ and links ๐ธ.๐๐ โ ๐ถ Bandwidth capacity of link ๐ โ ๐ธ.๐๐ โ ๐ท Bandwidth demand of flow ๐ โ ๐น .๐๐ โ ๐ Set of tunnels for flow ๐ โ ๐น .
AdditionalTeaVaR Input
๐ฝ Target availability level (e.g.,99.9%).๐ โ ๐ Network state corresponding to failure scenarios
of shared risk groups.๐๐ Probability of network state ๐.
Auxiliary Variables
๐ข๐ Total loss in scenario ๐ .๐ก๐ ,๐ Loss on flow ๐ in scenario ๐
๐๐ ,๐ก 1 if tunnel ๐ก is available in scenario ๐ , 0 otherwise๐๐ก,๐ 1 if tunnel ๐ก uses edge ๐, 0 otherwise
TE Output๐๐ Total bandwidth for flow ๐ .๐๐,๐ก Allocated traffic on tunnel ๐ก โ ๐๐ for flow ๐ โ ๐น .
AdditionalTeaVaR Output
๐ผ โLossโ (a.k.a the Value at Risk (VaR)).
minimize ๐ผ + 11โ๐ฝ
โ๏ธ๐ โ๐
๐๐ ๐ข๐
subject toโ๏ธ
๐โ๐น,๐กโ๐๐
๐๐,๐ก๐๐ก,๐ โค ๐๐ โ๐
๐ข๐ โฅ ๐ก๐,๐ โ ๐ผ โ๐, ๐
where ๐ก๐ ,๐ = 1โโ๏ธ
๐กโ๐ก๐๐๐,๐ก๐๐ ,๐
๐๐โ๐, ๐
Table 3.1: Key notations in the TeaVaR formulation. The original optimizationproblem is minimizing (3.3) subject to (3.1) โ (3.2). Here, we show the LP formula-tion; see ยง3.2 for its derivation.
In Chapter 4, I evaluate the impact of the tunnel selection scheme (e.g., ๐-shortest
paths, edge-disjoint paths, oblivious-routing) on performance. My evaluation shows
the TeaVaR optimization improves the achievable utilization-availability balance for
all considered tunnel-selection schemes.
Output. The output of TeaVaR, consists of two parts (see Table 3.1): (1) the total
bandwidth ๐๐ that flow (source-destination pair) ๐ is permitted to utilize (across all
of its tunnels in ๐๐ ); (2) a specification for each flow ๐ of how its allocated bandwidth
๐๐ is split across its tunnels ๐๐ . The bandwidth allocated on tunnel ๐ก is denoted by
26
๐๐,๐ก.
Optimization goal. Previous studies of TE considered optimization goals such
as maximizing total concurrent flow [37, 17, 6, 30], max-min fairness [34, 19, 11],
minimizing link over-utilization [27], minimizing hop count [28], and accounting for
hierarchical bandwidth allocations [26]. As formalized below, an appropriate choice
for the present context is selecting the ๐๐,๐กs (per-tunnel bandwidth assignments) in
a manner that maximizes the well-studied maximum-concurrent-flow objective [37].
This choice of objective will enable us to maximize network throughput while achiev-
ing some notion of fairness in terms of availability across network users. In ยง3.2, I
discuss ways to extend the framework to other optimization objectives.
Under maximum-concurrent-flow, the goal is to maximize the value ๐ฟ โ [0, 1] such
that at least an ๐ฟ-fraction of each flow ๐ โs demand is satisfied across all flows. For
example, ๐ฟ = 1 implies all demands are fully satisfied by the resulting bandwidth
allocation, while ๐ฟ = 13
implies at least a third of each flowโs demand is satisfied.
3.2 TeaVaR: deriving the LP
TeaVaRโs additional inputs and outputs are listed in Table 3.1. Given a target
availability level ๐ฝ, the goal is to cast TE optimization as a CVaR-minimization
problem whose output is a bandwidth allocation to flows that can be materialized
with probability of at least ๐ฝ. Doing so requires careful specification of (๐) the โcontrolโ
and โuncertaintyโ vectors, as described in Chapter 2, and (๐๐) a โloss functionโ that
provides fairness and avoids starvation across network flows. The formulation is shown
in Table 3.1. In this section, I describe in detail how this formula is derived.
3.2.1 Probabilistic failure model
I consider a general failure model, consisting of a set of failure events ๐. A failure
event ๐ง โ ๐ represents a set of SRGs becoming unavailable (the construction of a set
of SRGs is described elsewhere [38, 25], and ยง4.1). Importantly, while failure events
in my formulation are uncorrelated, this does not preclude modeling correlated link
27
failures. Consider a failure event ๐ง representing a technical failure in a certain link
๐ and another failure event ๐งโฒ representing a technical failure in a switch or a power
outage that will cause multiple links, including ๐, to become unavailable concurrently.
Note that even though link ๐ is inactive, whether ๐ง or ๐งโฒ is realized, ๐ง and ๐งโฒ, which
capture failures of different components, are independent events.
Each failure event ๐ง occurs with probability ๐๐ง. As described earlier, the failure
probabilities are obtained from historical data (see Chapter 4 for more details on
estimation techniques, as well as sensitivity analysis for inaccuracies in these esti-
mations). I denote by ๐ = (๐ 1, . . . , ๐ |๐|) a network state, where each element ๐ ๐ง is a
binary random variable, indicating whether failure event ๐ง occurred (๐ ๐ง = 1) or not.
For example, for a network with 15 SRGs, the possible set of events (๐) is a vector
with 15 elements, where each element indicates whether the corresponding SRG has
failed or not. For example, ๐ = (0, . . . , 0, 1) captures the network state in which only
SRG_15 is down. More formally, let ๐ be the set of all possible states, and let ๐๐ de-
note the probability of state ๐ = (๐ 1, . . . ๐ |๐|) โ ๐. Hence, the probability of network
state ๐ can be obtained using the following equation,
๐๐ = ๐ (๐ 1 = ๐ 1, . . . , ๐ |๐| = ๐ |๐|) = ฮ ๐ง
(๏ธ๐ ๐ง๐๐ง + (1โ ๐ ๐ง)(1โ ๐๐ง)
)๏ธ.
The uncertainty vector specifies which tunnels are up. I define ๐ฆ as a vector
of size |๐ |, where each vector element ๐ฆ๐ก is a binary random variable that captures
whether tunnel ๐ก is available (๐ฆ๐ก = 1) or not (๐ฆ๐ก = 0). This random variable depends
on realizations of relevant failure events. For example, ๐ฆ๐ก will equal 0 if one of the
links or switches on the tunnel is down. Since each random variable ๐ฆ๐ก is a function
of the random network state ๐ , we often use ๐ฆ๐ก(๐ ), and ๐ฆ(๐ ) to denote the resulting
vector of random variables. In the LP formulation, I pre-compute these values to
provide fast look-ups and store them in the matrix ๐๐ ,๐ก, where ๐๐ ,๐ก is a 1 if tunnel ๐ก is
available in scenario ๐ or 0 if it is not.
The control vector specifies how bandwidth is assigned to tunnels. Recall
that the output ๐ in my WAN TE formulation captures the traffic allocation for each
28
flow on each of its tunnels. This is the control vector for my CVaR-minimization.
As in TE schemes, such per tunnel bandwidth assignment has to ensure the edge
capacities are respected, i.e., satisfy the following constraint:
โ๏ธ๐โ๐น,๐กโ๐๐
๐๐,๐ก๐๐ก,๐ โค ๐๐, โ๐ โ ๐ธ. (3.1)
Here, this constraint must sum all overlapping flows on any given edge. The matrix
๐๐ก,๐ is a binary variable that represents the edges for each tunnel as a 1 if tunnel ๐ก
uses a certain edge ๐, and 0 otherwise. This matrix is pre-computed to allow fast
look-ups. To account for potential failures, I allow the total allocated bandwidth per
user ๐,โ๏ธ
๐กโ๐๐๐๐,๐ก, to exceed its demand ๐๐ .
3.2.2 Loss function
I define the loss function in two steps. First, I define a loss function for each flow,
called the flow-level loss. Then, I define a total loss as a function of the flow-level
losses for all flows, known as the network-level loss.
Flow-level loss. Recall that in the TE formulation, the optimization objective is
to assign the control variables ๐๐,๐ก (per-tunnel bandwidth allocations) in a manner
that maximizes the concurrent-flow, i.e., maximizes the value ๐ฟ for which each flow
can send at least a ๐ฟ-fraction of its demand. To achieve this, loss in the framework is
measured in terms of the fraction of demand not satisfied (i.e., 1โ ๐ฟ). My goal thus
translates into generating the per-tunnel bandwidth assignments that minimize the
fraction of demand not satisfied for a specified level of availability ๐ฝ.
In my formulation, the maximal satisfied demand for flow ๐ is given by the sum
of the allocations on the tunnels for that flow,โ๏ธ
๐กโ๐๐๐๐,๐ก๐๐ ,๐ . Thus, the loss for each
flow ๐ with respect to its demand ๐๐ is captured by[๏ธ1โ
ฮฃ๐กโ๐๐๐๐,๐ก๐๐ ,๐
๐๐
]๏ธ+, where [๐ง]+ =
max{๐ง, 0}; note that the [+] operator ensures that the loss is not negative (hence,
the optimization will not gain by sending more traffic than the actual demand). This
notion of per-flow loss captures the loss of assigned bandwidth for a given network
state ๐ .
29
Network-level loss function. To achieve fairness, in terms of availability, we define
the global loss function as the maximum loss across all flows, i.e.,
๐ฟ(๐, ๐ฆ) = max๐
[๏ธ1โ
ฮฃ๐กโ๐๐๐๐,๐ก๐ฆ๐
๐๐
]๏ธ+. (3.2)
While this loss function is nonlinear, I am able to transform into a linear constraint
and thus turn the optimization problem into a linear program (LP) in the next section.
3.2.3 Objective function
To formulate the optimization objective, I introduce the mathematical definitions of
๐ ๐๐ ๐ฝ and ๐ถ๐ ๐๐ ๐ฝ. For a given loss function ๐ฟ, the ๐ ๐๐ ๐ฝ(๐ฅ) is defined as ๐๐ฝ(๐) =
min{๐ | ๐(๐, ๐) โฅ ๐ฝ}, where ๐(๐, ๐) = ๐ (๐ | ๐ฟ(๐, ๐ฆ(๐ )) โค ๐), and ๐ (๐ | ๐ฟ(๐, ๐ฆ(๐ )) โค ๐)
denotes the cumulative probability mass of all network states satisfying the condition
๐ฟ(๐, ๐ฆ(๐ )) โค ๐. ๐ถ๐ ๐๐ ๐ฝ is simply the mean of the ๐ฝ-tail distribution of ๐ฟ(๐, ๐ฆ), or
put formally:
๐ถ๐ ๐๐ ๐ฝ(๐) =1
1โ ๐ฝโ๏ธ
๐ฟ(๐,๐ฆ(๐ ))โฅ๐๐ฝ(๐)
๐๐๐ฟ(๐, ๐ฆ(๐ )).
Note that the definition of ๐ถ๐ ๐๐ ๐ฝ utilizes the definition of ๐ ๐๐ ๐ฝ. To minimize
๐ถ๐ ๐๐ ๐ฝ, I define the following potential function
๐น๐ฝ(๐, ๐ผ) = ๐ผ +1
1โ ๐ฝ๐ธ[[๐ฟ(๐, ๐ฆ)โ ๐ผ]+]
= ๐ผ +1
1โ ๐ฝโ๏ธ๐
๐๐ [๐ฟ(๐, ๐ฆ(๐ ))โ ๐ผ]+. (3.3)
The optimization goal is to minimize ๐น๐ฝ(๐ฅ, ๐ผ) over ๐,โ, subject to (3.1) โ (3.2).
I leverage the following theorem, which states that by minimizing the potential func-
tion, the optimal ๐ถ๐ ๐๐ ๐ฝ and (approximately) also the corresponding ๐ ๐๐ ๐ฝ are ob-
tained.
Theorem 1 [36] If (๐*, ๐ผ*) minimizes ๐น๐ฝ, then not only does ๐* minimize the ๐ถ๐ ๐๐ ๐ฝ
30
๐ถ๐ฝ over ๐, but also
๐ถ๐ฝ(๐*, ๐ผ*) = ๐น๐ฝ(๐
*, ๐ผ*), (3.4)
๐๐ฝ(๐*) โ ๐ผ*. (3.5)
The beauty of this theorem is that although the definition of ๐ถ๐ ๐๐ ๐ฝ uses the
definition of ๐ ๐๐ ๐ฝ, we do not need to work directly with the ๐ ๐๐ ๐ฝ function ๐๐ฝ(๐)
to minimize ๐ถ๐ ๐๐ ๐ฝ. This is significant since, as mentioned above, ๐๐ฝ(๐) is a non-
smooth function which is hard to deal with mathematically. The statement of the
theorem uses the notation โ to denote that with high probability, ๐ผ* is equal to
๐๐ฝ(๐*). When this is not so, ๐ผ* constitutes an upper bound on the ๐ ๐๐ ๐ฝ. The actual
๐ ๐๐ ๐ฝ can be easily obtained from ๐ผ*, as discussed in Appendix A.1.
3.2.4 Linearizing the loss function
I am interested in minimizing
๐น๐ฝ(๐, ๐ผ) = ๐ผ +1
1โ ๐ฝ๐ธ[max{0, ๐ฟ(๐ฅ, ๐ฆ)โ ๐ผ}] (3.6)
= ๐ผ +1
1โ ๐ฝฮฃ๐๐๐ฆ[๐ฟ(๐ฅ, ๐ฆ(๐))โ ๐ผ]+, (3.7)
In what follows, I โlinearize" the objective function by adding additional (linear)
constraints. I introduce a new set of variables ๐ข = {๐ข๐ }, where ๐ข๐ represents the total
loss of a given network state. I rewrite the objective function as
๐น๐ฝ(๐ , ๐ผ) = ๐ผ +1
1โ ๐ฝโ๏ธ๐
๐๐ ๐ข๐ , (3.8)
and add the following constraints
๐ข๐ โฅ ๐ฟ(๐, ๐ฆ(๐ ))โ ๐ผ โ๐ (3.9)
๐ข๐ โฅ 0. โ๐ (3.10)
31
Observe that minimizing ๐น w.r.t. ๐ and ๐ผ is equivalent to minimizing ๐น w.r.t. ๐ , ๐, ๐ผ.
I have removed the initial max operator and + operator in (3.6). However, ๐ฟ(ยท, ยท) still
involves a max operator. We must rewrite (3.9) as ๐ข๐ + ๐ผ โฅ ๐ฟ(๐, ๐ฆ(๐ )); now we can
materialize the max operator through the following inequalities
๐ข๐ + ๐ผ โฅ 0, โ๐ (3.11)
๐ข๐ + ๐ผ โฅ ๐ก๐ ,๐ โ๐ , ๐, (3.12)
where
๐ก๐ ,๐ = 1โฮฃ๐กโ๐๐
๐๐,๐ก๐ฆ๐(๐ )
๐๐. (3.13)
We end up with an LP with decision variables ๐, ๐ผ, ๐ข๐ , ๐ก๐ ,๐ (๐ข๐ and ๐ก๐ ,๐ can be
viewed as auxiliary variables); the objective of the LP is minimizing (3.8) subject to
(3.1), (3.10)โ(3.13).
3.2.5 Routing (and re-routing) in TeaVaR
Bandwidth allocations. The total bandwidth that flow ๐ is permitted to utilize
(across all tunnels in ๐๐ ) is given by ๐๐ . Clearly, ๐๐ should not exceed flow ๐ โs
demand ๐๐ to avoid needlessly wasting capacity. However, I do not add an explicit
constraint for this requirement. Instead, I embed it implicitly in the loss function
(3.2). Once the solution is computed, each flow ๐ is given: (i) the total allowed
bandwidth (1 โ ๐๐ฝ(๐*))% of its demand; i.e., ๐๐ = (1 โ ๐๐ฝ(๐*))๐๐ ; and (ii) a weight
assignment ๐ค๐,๐ก, where ๐ค๐,๐ก =๐*๐,๐กโ๏ธ
๐กโ๐๐๐*๐,๐ก
.
Tunnel Weights As in other TE solutions (e.g., [30, 38]), I use a simple rule for flow
re-assignment in the event of network failures: the traffic of flow ๐ is split between
all surviving tunnels in ๐๐ so that each tunnel ๐ก carries traffic proportionally to ๐ค๐,๐ก.
Put formally, let ๐๐ โ ๐๐ be the subset of ๐ โs tunnels that are available. Then, each
tunnel ๐ก โ ๐๐ carries ๐ค๐,๐กโ๏ธ๐กโ๐๐
๐ค๐,๐ก๐๐ traffic (i.e., its proportional share of ๐๐ ). I henceforth
refer to this allocation rule as the proportional assignment rule.
Proportional assignment does not require global coordination/computation upon
32
[1,1,1]
[1,1,0] [1,0,1]
[1,0,0] [0,1,0]
[0,0,0]
[0,1,1]
[0,0,1]
1)
2)
Figure 3-1: A diagram of the scenario space pruning algorithm with assumptions 1)and 2). The algorithm recursively searches the tree depth first, updating ๐๐ at eachlevel until ๐๐ < ๐ upon which it regresses one level.
failures and can be easily implemented in the data plane via (re-)hashing. Because
the proportional assignment rule is not directly encoded in the CVaR minimization
framework, traffic re-assignment might result in congestion, i.e., violating constraint
(3.1). Nevertheless, it turns out that this rather simple rule guarantees such violation
occurs with very low probability (upper-bounded by 1โ ๐ฝ). Formally,
Theorem 2 Under TeaVaR, each flow ๐ is allocated bandwidth (1โ ๐๐ฝ(๐*))๐๐ and
no link capacity is exceeded with probability of at least ๐ฝ.
See Appendix A.2 for the proof.
3.3 Scenario pruning algorithm
As discussed earlier, applying TeaVaR to a large network is challenging because
the number of network states, representing combinations of link failures, increases
exponentially with the network size. To deal with this, I devise a scenario pruning
algorithm to efficiently filter out scenarios that occur with negligible probability. The
main idea behind the algorithm is to use a tree representation of the different scenar-
ios, where the scenario probability decreases with movement away from the root. The
tree is traversed recursively with a stopping condition of reaching a scenario whose
33
probability is below a cutoff threshold. The pruned scenarios are accounted for in
the optimization as follows. To give an upper-bound on the ๐ถ๐ ๐๐ ๐ฝ (equivalently,
a lower-bound on the throughput), I collapse all the pruned scenarios into a single
scenario with probability equal to the sum of probabilities of the pruned scenarios. I
then associate a maximal loss of 1 with that scenario. In ยง4.7, I evaluate the impact
of the scenario pruning algorithm on both running time and accuracy.
Recall that a scenario is represented as a set of failure events. I introduce the idea
of a scenario cutoff ๐, and prune out all scenarios that occur with probability ๐ < ๐.
Generally, I refer to failure events as independent events that together can make up a
network state. A network state can be modeled as a bitmap where each bit represents
whether or not an edge can carry traffic. I call this bitmap a scenario. Each failure
event, such as a fiber cut, a node outage, or a correlated link failure, can lead to a
scenario in which one or more bits in a network state is flipped.
The scenario pruning algorithm (see Figure 3-1 for an illustration), uses a tree
representation of the different scenarios, where the root note is the scenario where no
failure event occurs [0, 0, . . . , 0], and every child node differs from its parent by one bit.
The tree is constructed such that each flipped bit must be to the right of the previously
flipped bit to prevent revisiting any previously visited states. Assuming each failure
event occurs with probability less than 0.5, the scenario probability decreases as
we traverse away from the root. Accordingly, we traverse the tree in a depth-first
search until the condition ๐๐ < ๐ is met, at which point no further scenarios down
that path need to be searched. It is important to efficiently calculate the scenario
probabilities while traversing the tree. To do so, I update the probability of a child
scenario incrementally from its parent via ๐๐๐ โ ๐๐โ1๐
1โ๐๐ง๐๐ง
; this update rule follows
from immediately from the formula ๐๐ = ๐ (๐1 = ๐1, . . . , ๐|๐| = ๐|๐|) = ฮ ๐ง
(๏ธ๐๐ง๐๐ง + (1โ
๐๐ง)(1 โ ๐๐ง))๏ธ. Note that the tree is not balanced, because if we consider failures in
order, taking the leftmost path in the tree, we do not have to revisit any scenarios
where failure ๐ง has previously occurred.
34
Chapter 4
Evaluating TeaVaR
In this section, I present the evaluation of TeaVaR. I begin by describing the ex-
perimental framework and evaluation methodology (ยง4.1). The experimental results
focus on the following elements:
1. Benchmarking TeaVaRโs performance against the state-of-the-art TE schemes
(ยง4.2).
2. Examining TeaVaRโs robustness to noisy estimates of failure probabilities (ยง4.6).
3. Quantifying the effect of scenario pruning on the running time and the quality of
the solution (ยง4.7).
Topology Name #Nodes #EdgesB4 12 38IBM 18 48ATT 25 112X-Net โ 30 โ 100
Table 4.1: Network topologies used in the evaluations. For confidentiality reasons, Ido not report exact numbers for the X-Net topology.
35
0
0.2
0.4
0.6
0.8
1
1e-(m+3) 1e-(m+2) 1-e(m+1) 1e-m
CDF
Failure Probability
(a) Empirical data from X-Net
0
0.2
0.4
0.6
0.8
1
1e-006 1e-005 0.0001 0.001 0.01 0.1
CD
F
Failure Probability
(b) Weibull distribution
Figure 4-1: CDF of failure probabilities used in our experiments. The exact value of๐ in (a) is not shown due to confidentiality reasons. The shape and scale parametersin (b) are 0.8 and 10โ4, respectively.
4.1 Experimental setting
Topologies. I evaluate TeaVaR on four network topologies: B4, IBM, ATT, and
X-Net. The first three topologies (and their traffic matrices) were obtained from the
authors of SMORE [27]. X-Net is the network topology of a large cloud provider
in North America. See Table 4.1 for a specification of network sizes. The empirical
data from the X-Net network consists of the following: the capacity of all links (in
Gbps) and the traffic matrices (source, destination, amount of data in Mbps) over
four months at a resolution of one sample per hour. Data on failure events include
the up/down state of each link at 15-minute granularity over the course of a year, as
well as a list of possible shared risk groups. Data on ATT, B4, and IBM topologies
include a set of at least 24 demand matrices and link capacities, but per-link failure
probabilities are missing. Hence, I use a Weibull distribution derived from the X-Net
measurements and change its parameters over the course of the simulations.
Tunnel selection. TE schemes [24, 20, 17, 30] often use link-disjoint tunnels for
each source-destination pair. However, recent work shows performance improvement
by using oblivious tunnels (interchangeably also referred to as oblivious paths) [27].
Because TeaVaRโs optimization framework is orthogonal to tunnel selection, I run
simulations with a variety of tunnel-selection schemes, including oblivious paths, link
disjoint paths, and ๐-shortest paths. As I show later in the section, TeaVaR achieves
higher throughput regardless of the tunnel selection algorithm. We also study the
36
impact of tunnel selection on TeaVaR and find that combining TeaVaR with the
tunnel selection of oblivious-routing [27] leads to better performance (ยง4.2).
Deriving failure probability distributions. For each link ๐, I examine historical
data and track whether ๐ was up or down in a measured time epoch. Each epoch is a
15-minute period. I obtain a sequence of the form (๐1, ๐2, . . .) such thatโ each ๐๐ก spec-
ifies whether the link was up (๐๐ก = 1) or down (๐๐ก = 0) during the ๐กth measured time
epoch. From this sequence, another sequence (๐ฟ1, ๐ฟ2, . . . , ๐ฟ๐) is derived such that ๐ฟ๐ is
the number of consecutive time epochs the link was up prior to the ๐th time it failed.
For example, from the sequence (๐1, ๐2, . . . , ๐12) = (1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0), I de-
rive the sequence (๐ฟ1, ๐ฟ2, ๐ฟ3) = (2, 3, 1) (the link was up for 2 time epochs before the
first failure, 3 before the second failure, and 1 before the last failure). An unbiased
estimator of the mean uptime is given by ๐ =ฮฃ๐
๐=1๐ฟ๐
๐. I make a simplified assumption
that the link up-time is drawn from a geometric distribution (i.e., the failure proba-
bility is fixed and consistent across time epochs). Then, the failure probability ๐๐ of
link ๐ is simply the inverse of the mean uptime, that is, ๐๐ = 1๐. Note that we can
use this exact analysis for other shared-risk groups, such as switches.
Figure 4-1(a) plots the cumulative distribution function (CDF) for the failure
probability across the network links, derived by applying the above analysis method-
ology to the empirical availability traces of the X-Net network. The x-axis on the plot
represents the failure probability, parametrized by ๐. The exact value of ๐ is not
disclosed for reasons of confidentiality. Nonetheless, the important takeaway from this
figure is that the failure probabilities of different links might differ by orders of mag-
nitude. To accommodate the reproducibility of results, I obtain a Weibull probability
distribution which fits the shape of the empirical data. The Weibull distribution,
which has been used in prior study of failures in large backbones [32], is used here to
model failures over time for topologies for which I do not have empirical failure data.
I denote the Weibull distribution with shape parameter ๐ and scale parameter ๐ by
๐ (๐, ๐). In Figure 4-1(b), I plot the Weibull distribution used in the evaluation, as
well as the parameters needed to generate it. Throughout the experiments, I change
the shape and scale parameters of the Weibull distribution and study the impact of
37
1 1.4 1.8 2.2 2.6
3 3.4
97 98 99 100
Demand Scale
Availability (%)
Oblivious paths
(a) IBM topology
1 1.4 1.8 2.2 2.6
3 3.4
97 98 99 100
Demand Scale
Availability (%)
Oblivious paths
(b) X-Net topology
1 1.4 1.8 2.2 2.6
3 3.4
97 98 99 100
Demand Scale
Availability (%)
Edge disjoint paths
(c) B4 topology
1 1.4 1.8 2.2 2.6
3 3.4
95 96 97 98 99 100Demand Scale
Availability (%)
Edge disjoint paths
(d) ATT topology
Figure 4-2: Comparison of TeaVaR against various TE schemes under differenttunnel selection algorithms. All schemes in (a) and (b) use oblivious paths, and allschemes in (c) and (d) use edge disjoint paths. The term availability refers to thepercentage of scenarios that meet the 100% demand-satisfaction requirement.
probability distribution on performance.
Optimization. The optimization framework uses the Gurobi LP solver [16] and is
implemented using the Julia optimization language [8].
4.2 Throughput vs. availability
I examine the performance of different TE schemes with respect to both throughput
and availability.
Setup. I benchmark TeaVaR against several approaches: SMORE [27], FFC [30],
MaxMin (in particular, the algorithm used in B4 [19, 11]), and ECMP [12]. SMORE
minimizes the maximum link utilization without explicit guarantees on availability,
FFC maximizes the throughput while explicitly considering failures, MaxMin maxi-
mizes minimum bandwidth per user [11], and TeaVaR minimizes the CVaR for an
input probability. When link failures occur, traffic is redistributed across tunnels ac-
38
cording to the proportional assignment mechanism (see ยง3.2) without re-optimizing
weights. In our evaluations, I am concerned with both the granted bandwidth and
the probabilistic availability guarantee it comes with. In TeaVaR, the probability is
explicit (controlled by the ๐ฝ parameter in the formulation as shown in Eq. 3.3). In
FFC, the per-user bandwidth is granted with 100% availability for scenarios with up
to ๐-link failures. However, to fairly compare the ability of the algorithm to accom-
modate scaled-up demands, I allow all algorithms to send the entire demand (at the
expense of potential degradation in availability).
Availability vs. demand scale. I first seek to analyze the availability achieved
by various TE schemes as demand is scaled up. Given that current networks are
designed with traditional worst-case assumptions about failures, all topologies are
over-provisioned. Hence, I begin with the input demands, compute the availability
achieved when satisfying them for different schemes, and then scale the demands by
introducing a (uniform) demand scale-up factor ๐ โฅ 1 which multiplies each entry in
the demand matrix.
Availability is calculated by running a post-processing simulation in which I induce
failure scenarios according to their probability of occurrence, and attempt to send the
entirety of the demand through the network. For each scenario, I record the amount of
unsatisfied demand (loss) for each flow, as well as the probability associated with that
scenario. The sum of the probabilities for scenarios where demand is fully satisfied
reflects the availability in that experiment. For example, if a TE schemeโs bandwidth
allocation is unable to fully satisfy demand in 0.1% of scenarios, it has an availability
of 99.9%. I then scale the demand matrix and repeat the above analysis for at least
24 demand matrices per topology. In Figure 4-2, I summarize the results by depicting
the demand-scale vs. the corresponding availability.
The results show a consistent trend: TeaVaR achieves higher demand scale for
a given availability level. In particular, for any target availability level, TeaVaR
achieves up to twice the demand scale-up compared to other approaches. Notably, in
X-Net topology and with oblivious paths, TeaVaR is able to scale the demand by
a factor of 3.4, while MaxMin achieves a scale up of 2.6. The remaining approaches
39
70
75
80
85
90
95
100
99% 99.5% 99.9% 99.99%
Throughput (%)
ฮฒ
TEAVAR SMORE ECMP FFC-1 MaxMin
Figure 4-3: Averaged throughput guarantees for different ๐ฝ values on ATT, B4, andIBM topologies. TeaVaR is the only scheme that can explicitly optimize for a given๐ฝ. For all other schemes, the value on the x-axis is computed based on achievedthroughput.
cannot scale beyond 1.4x the original demand.
4.3 Achieved throughput
I next demonstrate the tradeoff between a target availability threshold and the achieved
throughput without scaling the demand. In the previous set of experiments, avail-
ability was measured as the probability mass of scenarios in which demand is fully
satisfied (โall-or-nothingโ requirement). In contrast, in this set of experiments, I mea-
sure the fraction of the total demand that can be guaranteed for a given availability
target (or the throughput). This fraction is optimized explicitly in TeaVaR for a
given value of ๐ฝ. For other TE schemes, I obtain the fraction of demand using a
similar post-processing method as before: for each failure scenario, I simulate the
outcome of sending the entire demand through the network, sort the scenarios in
increasing loss values and report the demand fraction at the ๐ฝ-percentile (i.e., the
throughput is greater or equal to that value for ๐ฝ percent of scenarios). The range of
availability values is chosen according to typical availability targets [18].
Figure 4-3 plots the average throughput for ATT, B4, and IBM topologies. For
each TE scheme, I report the results under the tunnel selection algorithm which has
40
TeaVaR FFC1 FFC2
Probability Avg๐ Min๐ Avg๐ Min๐ Avg๐ Min๐
90% 100% 100% 88.32% 4.62% 29.76% 0%95% 99.9% 99.9% 88.32% 4.62% 29.76% 0%99% 95.87% 95.87% 88.32% 4.62% 29.76% 0%
99.9% 92.53% 92.53% 88.32% 4.62% 29.76% 0%99.99% 82.78% 82.78% 0% 0% 29.76% 0%
Table 4.2: The mathematical bandwidth guarantees of TeaVaR and FFC averagedacross 3 topologies, and 10 demand matrices. Avg๐ represents the average band-width for all flows, guaranteed with probability ๐% of the time. Min๐ represents theminimum bandwidth of all flows.
performed the best (SMORE, TeaVaR and MaxMin with oblivious, and FFC with
link-disjoint paths). The results demonstrate that TeaVaR is able to achieve higher
throughput for each of the target availability values. This is because it can optimize
throughput for an explicit availability target within a probabilistic model of failures.
4.4 Mathematical guarantees with tunable ๐ฝ.
The previous two experiments used a post processing simulation is used to calculate
availability and throughput. This simulation allows sending all of the demand in the
network to compare fairly across all TE schemes instead of adhering to the mathe-
matical guarantees from each scheme. FFC and TeaVaR (explained in ยง3.2.5) both
give permitted bandwidth amounts for each flow (๐๐) to ensure their mathematical
guarantees hold. FFC guarantees that the amount sent will be successfully received
in all k-link failure scenarios. TeaVaR guarantees that the amount sent will be re-
ceived ๐ฝ% of the time. In Table 4.2, instead of sending all the demand, I assume
the permitted bandwidth amounts (๐๐) and report the guaranteed average and min-
imum across all flows. Results are averaged across multiple topologies and demand
matrices. While FFC๐ gives a single bandwidth amount that only holds up to the
probability mass of the k-failure scenarios, we see TeaVaR can achieve significant
bandwidth improvements by optimizing explicitly for a given probability. We also
41
0
0.5
1
1.5
2
90% 92% 94% 96% 98% 100%
CVaR
ฮฒ (%
Loss)
ฮฒ
TEAVAR_obliviousTEAVAR_edge_disjoint
TEAVAR_ksp3TEAVAR_ksp4
Figure 4-4: The effect of tunnel selection scheme on TeaVaRโs performance, quan-tified by the resulting ๐ถ๐ ๐๐ ๐ฝ (or loss) for different values of ๐ฝ. TeaVaR usingoblivious paths has better performance (lower loss).
see that TeaVaR uses max-min bandwidth allocation so that the average is also
the minimum allowed per flow. In contrast, FFC starves certain flows to achieve a
slightly higher average bandwidth. The paper on FFC [30] mentions an iterative ver-
sion of FFC that can avoid starvation, but the LP is not inherently built for fairness
or flexible guarantees like TeaVaR.
4.5 Impact of tunnel selection.
So far, TeaVaR has been simulated with either link-disjoint or oblivious tunnels [27]
schemes. I now analyze the effect of other tunnel selections on TeaVaRโs perfor-
mance. I demonstrate that while path selection is an important aspect of any TE
scheme, no specific choice is needed for TeaVaRโs success. In Figure 4-4, I plot the
obtained ๐ถ๐ ๐๐ ๐ฝ as a function of beta for various tunnel selection schemes: ๐-shortest
paths with 3 or 4 paths, FFCโs link disjoint paths, and SMOREโs oblivious rout-
ing. TeaVaR performs comparably well regardless of the tunnel selection scheme.
However, the figure shows that oblivious paths are still superior. For example, the
obtained ๐ถ๐ ๐๐ ๐ฝ value is at least 20% better for ๐ฝ = 0.99 than in any other tunnel
selection scheme. These results indicate that an oblivious-routing tunnel selection is
a good choice to complement TeaVaR. Oblivious routing is intended to avoid link
42
over-utilization through diverse and low-stretch path selection. Intuitively, these path
properties are useful for TeaVaR, providing the optimization framework with a set
of tunnels that can provide high availability.
4.6 Robustness to probability estimates
TeaVaR uses a probabilistic model of network failures. In particular, the optimiza-
tion framework explicitly accounts for the probabilities of failure events, such as link
failures. These probabilities are estimated by analyzing the historical time-series of
up/down status and are inherently prone to estimation errors, regardless of the tech-
nique used (I provide a basic technique here, but more sophisticated techniques, e.g.,
based on machine-learning, can be applied).
To examine the effects of such inaccuracies, I use the following methodology. I
assume there is a set of probabilities that is the ground-truth, and the actual per-
formance of any TE solution should be evaluated against these probabilities. In
particular, I evaluate two versions of TeaVaR: (i) TeaVaR using the ground-truth
probabilities; (ii) TeaVaR using a perturbed version of the ground-truth probabilities
(reflecting estimation errors). The generation of (ii) requires a magnitude of noise ๐.
The probabilities that are given to TeaVaR as input are generated via ๐๐ง = ๐๐ง+๐๐ง๐๐,
where ๐ is random noise, distributed uniformly on [โ1, 1].
For each noise level in Table 4.3, I report the percent error in average throughput
across all scenarios compared to the error when optimized using the ground-truth
Noise in probability estimations % error in throughput1% 1.43%5% 2.95%10% 3.07%15% 3.95%20% 6.73%
Table 4.3: The effect of inaccurate probability estimations on TeaVaRโs perfor-mance. The decrease in throughout is caused by running TeaVaR with the ground-truth probabilities {๐๐}.
43
98
98.5
99
99.5
100
B4
IBM
ATT
XNetScenario
Coverage (%)
(a) Scenario coverage
0 0.5
1 1.5
2 2.5
3 3.5
4
B4
IBM
ATT
XNet
Percent Error
(%)
(b) Cutoff error
0
20
40
60
80
100
120
20 40 60 80 100
Time (s)
Number of Edges
(c) Optimizer times
Figure 4-5: Impact of scenario pruning on accuracy and runtime.I use 10โ4 as thecutoff threshold for ATT and 10โ5 for all other topologies. (a) The cutoff thresholdscover more than 99.5% of all possible scenarios. (b) The error incurred by pruningscenarios is less than 5% in all cases. (c) The cutoffs applied lead to manageablerunning times.
probabilities. The noise has a relatively small effect on the solution quality. This
is because fine-grained distribution modeling pays off less than intuition might lead
us to believe. For example, when the perturbed probabilities are within 10% of the
ground truth, TeaVaRโs throughput is within 3% of the solution obtained with the
actual probabilities.
4.7 Sensitivity to scenario pruning
In ยง3.2 I described an efficient algorithm for pruning scenarios. I now elaborate on
this algorithm and evaluate its impact on performance and accuracy. The first step
is to see what percentage of the scenario space was being pruned by various cutoff
thresholds. The cutoff threshold is defined in ยง3.3. Figure 4-5(a) shows the probability
mass of all the scenarios remaining in the optimization after a given cutoff. Modest
cutoffs leave a large portion, over 95% of the total space. Figure 4-5(b) shows the
effect of pruning on accuracy. Specifically, I compare the case of the CVaR value
with pruning to the case where we consider 100% of the scenario space (โoptimalโ)
using a standard error formula, |๐ถ๐ ๐๐ ๐ฝ,cutoff โ๐ถ๐ ๐๐ ๐ฝ,optimal |๐ถ๐ ๐๐ ๐ฝ,optimal
. With a cutoff similar to that
used in the bulk of my experiments (10โ4 for ATT and 10โ5 for all other topologies),
we achieve high scenario coverage with less than 5% error. It is also important to
note that the speed benefits of the cutoff are substantial, as shown in Figure 4-5(c).
Using cutoff values reduces the computation time from minutes (in the near-optimal
44
case) to a few tens of seconds, a plausible compute overhead for typical TE periods of
5-15 minutes. Note that all time complexity benchmarks were performed on a fairly
standard processor (4-core, 2.60 GHz processor with 32 GB RAM).
45
46
Chapter 5
Simulating and visualizing results
Optical backbones are million dollar assets, with fiber comprising their most expensive
component. Companies like Google, Microsoft and Facebook purchase or lease fiber
to support wide-area connectivity between distant data center locations, and they
need to know how and when to purchase this fiber. The process of deciding which
fibers to buy and how to manage them is capacity planning. While TeaVaR is cast
as a TE scheme throughout this thesis, it can also be used for capacity planning.
In this chapter, I build out a simulation tool, called TE-VIS. The tool commu-
nicates with my implementation of TeaVaR, and simulates its usage in the data
plane. I explain how TE-VIS can not only be useful for debugging any TE scheme
but also for capacity planning. The simulation allows network operators to see how
their network is performing, where the bottlenecks are, and what kind of demand
loads it can handle. It is a user interface that allows a user to select various inputs to
a TE scheme and take the output of that scheme, in this case TeaVaR, and use it to
simulate traffic through any chosen network and any given failure scenario. TE-VIS
can be used as a first step to implementing TeaVaR in a Software-Defined Network
(SDN) or long-term capacity planning.
47
Figure 5-1: An image showing TE-VIS when two nodes are selected. The tunnelsbetween the two nodes along with their weights are displayed on the top right, andthe dashed line is the current selected path.
5.1 Developing an API for the optimization
For TE-VIS to be up and running, it must be able to communicate with the optimizer
of TeaVaR. To do this, I turn TeaVaR into an API with a single endpoint that
takes as input, a a network topology, demand matrix, path selection algorithm, ๐ฝ
value, and scenario cutoff. I use the Julia HTTP package to accept HTTP requests
with proper headers, and the JSON package to decode these inputs in the JSON
data payload. The optimizer then runs, encodes its results, and send back a JSON
response.
5.2 Visualizing the output
To visualize the output, I need an interactive, fast, flexible data visualization library.
I build out the front-end using React [1] and use D3 [9] to display the network as
a 2D force-graph. Using a D3 force graph, I make the network interactive, and add
labels, click, and drag events accordingly. Most importantly, I make the force-graph
reactive to changes in the state of the overall application, so the user can manually
change the failure scenario, toggle link utilization labels, and select various nodes
to view more information about them. In addition, the front-end has a form with
inputs for the the optimization and makes HTTP requests to get the TE output. The
48
Figure 5-2: This image shows the utilization on each link. Utilization is defined as theamount of traffic divided by that links total capacity. The labels in red show wherecongestion would occur and traffic would be dropped because of the link breakingcapacity.
output is then mapped onto the graph, and traffic is routed and re-routed according
to the application state using the routing mechanisms described earlier. This allows
a real-time look at utilization and node routing information. When a single node is
selected, incoming and outgoing traffic are displayed. When two nodes are selected,
the demand between them is displayed, and the paths and their weights can be cycled
through using the arrow keys. All paths from source to destination are highlighted,
and the current selected path is marked by a dashed line.
5.3 Applications to capacity planning
While TE-VIS was built to demonstrate and explain the results of TeaVaR, I think
it has promise as a general tool for capacity planning. From what I have seen, it
is often hard to compare various TE algorithms, and TE-VIS may give a realistic
visualization of how traffic would actually be routed and suggest the effect of failures
for any TE scheme. Figure 5-2 shows an image from TE-VIS with the amount of
traffic flowing on each link. We can see that one link in red is down because of
a failure. We can also see one link with its utilization number in red because it
is greater than 100%. This link would become congested and cause traffic loss in
this scenario. In this case, network operators could choose to augment this linkโs
49
capacity to avoid congestion there. With TE-VIS, it can easily be determined where
the bottleneck links are and how to avoid them with proper capacity planning. In
addition, users can manually increase the capacity on given links and see the response
in the network. They can also simulate high traffic hours or large data transfers and
view the immediate affects all across the network.
50
Chapter 6
Contributions
The main contributions of this thesis is its introduction of a novel TE paradigm,
TeaVaR, to explicitly account for the likelihood of different failure events with the
goal of minimizing a formal notion of risk to a level deemed acceptable by network op-
erators. I design and evaluate this scheme with an optimization framework that allows
operators to optimize bandwidth assignment subject to meeting a desired availabil-
ity bar (e.g., providing 99.9% availability). I address algorithmic challenges related
to the tractability of risk minimization in this context, as well as operational chal-
lenges. I explore various objective goals of network operators, and apply TeaVaR
to real-world data from the inter-datacenter backbone of a large service provider in
North America. I find TeaVaR can support up to twice as much traffic as todayโs
state-of-the-art TE schemes at the same level of availability. TeaVaR illustrates
the usefulness of adopting financial risk theoryโs notion of Conditional Value at Risk
to network resource allocation challenges. The approach may find other important
applications in the networking domain.
Another important contribution is the visualization tool that was originally built
for TeaVaR, as it can be extended to any other TE or capacity planning scheme.
This tool is extremely useful for visualizing complex network simulations and various
network events. TeaVaR takes an important step in the development of failure-
aware TE schemes by modeling the reality of failures directly in the algorithm, and it
provides the real world bandwidth guarantees that network operators seek to achieve.
51
52
Appendix A
Appendix
A.1 Calculating ๐ ๐๐ ๐ฝ
Here, I describe a post-processing procedure to calculate the ๐ ๐๐ ๐ฝ. Note that the
procedure is generic, and applicable to any setting with a discrete number of states
(recall that in the present case, each network state corresponds to a different com-
bination of links, switches, etc. that are up or are down). The procedure is the
following: fixing ๐ฅ, we sort the states in increasing order of their loss. With some
abuse of notations, we enumerate the states according to the sorted order; let the
corresponding state losses be โ1 โค โ2 โค ยท ยท ยท โค โ๐พ . Let ๐พ๐ฝ be the unique index such
that ฮฃ๐พ๐ฝ
๐=1๐๐ โฅ ๐ฝ > ฮฃ๐พ๐ฝโ1
๐=1 ๐๐, where ๐๐ is the probability of state ๐. Then the ๐ ๐๐ ๐ฝ
is given by ๐๐ฝ(๐ฅ) = โ๐พ๐ฝ. See Proposition 8 in [36] for a proof.
Computing ๐ ๐๐ ๐ฝ in the rare case when ๐๐ฝ(๐ฅ*) < ๐ผ* (see Theorem 1). We get
this inequality when ฮฃ๐พ๐ฝ
๐=1๐๐ = ๐ฝ. This is unlikely to occur because the probabilities
correspond to empirical measures of up and down time (hence, the numbers often
have several decimal positions more than ๐ฝ). But even if this case occurs, we have
๐๐ฝ(๐ฅ) = โ๐พ๐ฝas described above.
Determining the ๐ ๐๐ ๐ฝ for individual user SLAs. Unlike the current formula-
tion, the per-user SLA (allowed demand and the corresponding probabilistic guaran-
tee) is not explicitly obtained as output of the optimization framework. In such cases,
we apply the above procedure on each individual user as follows. The key observation
53
is that the tunnel allocation per user is given from the optimization. We also know
the demand ๐๐ . For each user ๐ , we first reduce the dimension of the state-space to
include events that correspond only to links, switches, etc., which belong to one or
more of its routs. We sort the states and obtain the ๐ ๐๐ ๐ฝ as described above; by
using this procedure, the network provider can actually give different probabilistic
guarantees for different users, e.g., by fixing a different ๐ฝ for each user.
A.2 Proof of theorem 2
Since the theoremโs guarantee is with probability greater than or equal to ๐ฝ, by
definition of ๐ ๐๐ ๐ฝ, we may restrict our attention to failure states whose loss is less
than or equal to ๐๐ฝ(๐*). Consider any such state; let ๐๐ โ ๐๐ be the subset of ๐ โs
tunnels that are available in that state. Since ๐๐ฝ(๐*) is an upper bound for each loss
scenario up to the ๐ฝ percentile, it follows from the definition of the loss function (3.2)
that
ฮฃ๐กโ๐๐๐*๐,๐ก โฅ
(๏ธ1โ ๐๐ฝ(๐*)
)๏ธ๐๐ .
Using this inequality together with the proportional assignment rule, we obtain that
each active tunnel ๐ โ ๐ ๐ carries flow ๐๐, satisfying
๐๐ =๐ค๐,๐ก
ฮฃ๐กโฒโ๐๐๐คโฒ
๐,๐ก
๐๐ (A.1)
=๐*๐,๐ก
ฮฃ๐กโฒโ๐๐๐*๐,๐กโฒ
(๏ธ1โ ๐๐ฝ(๐*)
)๏ธ๐๐ (A.2)
โค๐*๐,๐ก(๏ธ
1โ ๐๐ฝ(๐*)๐๐ )(๏ธ1โ ๐๐ฝ(๐*)
)๏ธ๐๐ = ๐*๐,๐ก. (A.3)
The theorem then follows immediately by recalling that each feasible solution satisfies
the capacity constraint (3.1).
54
Bibliography
[1] Todd Abel. ReactJS: Become a Professional in Web App Development. CreateS-pace Independent Publishing Platform, USA, 2016.
[2] Ian F. Akyildiz, Ahyoung Lee, Pu Wang, Min Luo, and Wu Chou. A roadmap fortraffic engineering in sdn-openflow networks. Comput. Netw., 71:1โ30, October2014.
[3] Mohammad Alizadeh, Tom Edsall, Sarang Dharmapurikar, RamananVaidyanathan, Kevin Chu, Andy Fingerhut, Vinh The Lam, Francis Matus, RongPan, Navindra Yadav, and George Varghese. Conga: Distributed congestion-aware load balancing for datacenters. SIGCOMM Comput. Commun. Rev.,44(4):503โ514, August 2014.
[4] Fredrik Andersson, Helmut Mausser, Dan Rosen, and Stanislav Uryasev. Creditrisk optimization with conditional value-at-risk criterion. Mathematical Program-ming, 89(2):273โ291, 2001.
[5] Ajay Kumar Bangla, Alireza Ghaffarkhah, Ben Preskill, Bikash Koley, ChristophAlbrecht, Emilie Danna, Joe Jiang, and Xiaoxue Zhao. Capacity planning for thegoogle backbone network. In ISMP 2015 (International Symposium on Mathe-matical Programming), 2015.
[6] Cynthia Barnhart, Niranjan Krishnan, and Pamela H. Vance. Multicommodityflow problems. In Encyclopedia of Optimization, pages 2354โ2362. Springer, 2009.
[7] Theophilus Benson, Ashok Anand, Aditya Akella, and Ming Zhang. MicroTE:Fine grained traffic engineering for data centers. In Proceedings of the Sev-enth COnference on emerging Networking EXperiments and Technologies, page 8.ACM, 2011.
[8] Jeff Bezanson, Stefan Karpinski, Viral B. Shah, and Alan Edelman. Julia: Afast dynamic language for technical computing. CoRR, abs/1209.5145, 2012.
[9] Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. D3 data-drivendocuments. IEEE Transactions on Visualization and Computer Graphics,17(12):2301โ2309, December 2011.
[10] Antonio J Conejo, Miguel Carriรณn, Juan M Morales, et al. Decision makingunder uncertainty in electricity markets, volume 1. Springer, 2010.
55
[11] E. Danna, S. Mandal, and A. Singh. A practical algorithm for balancing the max-min fairness and throughput objectives in traffic engineering. In 2012 ProceedingsIEEE INFOCOM, pages 846โ854, March 2012.
[12] B. Fortz, J. Rexford, and M. Thorup. Traffic engineering with traditional iprouting protocols. IEEE Communications Magazine, 40(10):118โ124, Oct 2002.
[13] Bernard Fortz and Mikkel Thorup. Internet traffic engineering by optimizingospf weights. In INFOCOM 2000. Nineteenth annual joint conference of theIEEE computer and communications societies. Proceedings. IEEE, volume 2,pages 519โ528. IEEE, 2000.
[14] Monia Ghobadi and Ratul Mahajan. Optical layer failures in a large backbone.In Proceedings of the 2016 ACM on Internet Measurement Conference, pages461โ467. ACM, 2016.
[15] Ramesh Govindan, Ina Minei, Mahesh Kallahalla, Bikash Koley, and Amin Vah-dat. Evolve or die: High-availability design principles drawn from googles net-work infrastructure. In Proceedings of the 2016 ACM SIGCOMM Conference,SIGCOMM โ16, pages 58โ72, New York, NY, USA, 2016. ACM.
[16] Zonghao Gu, Edward Rothberg, and Robert Bixby. Gurobi Optimizer ReferenceManual, Version 5.0. Gurobi Optimization Inc., Houston, USA, 2012.
[17] Chi-Yao Hong, Srikanth Kandula, Ratul Mahajan, Ming Zhang, Vijay Gill, Mo-han Nanduri, and Roger Wattenhofer. Achieving high utilization with software-driven WAN. In ACM SIGCOMM 2013 Conference, SIGCOMMโ13, Hong Kong,China, August 12-16, 2013, pages 15โ26, 2013.
[18] Chi-Yao Hong, Subhasree Mandal, Mohammad Al-Fares, Min Zhu, Richard Al-imi, Kondapa Naidu B., Chandan Bhagat, Sourabh Jain, Jay Kaimal, ShiyuLiang, Kirill Mendelev, Steve Padgett, Faro Rabe, Saikat Ray, Malveeka Tewari,Matt Tierney, Monika Zahn, Jonathan Zolla, Joon Ong, and Amin Vahdat. B4and after: Managing hierarchy, partitioning, and asymmetry for availability andscale in googleโs software-defined wan. SIGCOMM โ18, pages 74โ87, 2018.
[19] Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Ar-jun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, Jon Zolla,Urs Hรถlzle, Stephen Stuart, and Amin Vahdat. B4: Experience with a globally-deployed software defined wan. SIGCOMM Comput. Commun. Rev., 43(4):3โ14,August 2013.
[20] Virajith Jalaparti, Ivan Bliznets, Srikanth Kandula, Brendan Lucier, and IshaiMenache. Dynamic pricing and traffic engineering for timely inter-datacentertransfers. In Proceedings of the 2016 conference on ACM SIGCOMM 2016 Con-ference, pages 73โ86. ACM, 2016.
56
[21] Wenjie Jiang, Rui Zhang-Shen, Jennifer Rexford, and Mung Chiang. Cooperativecontent distribution and traffic engineering in an isp network. In ACM SIGMET-RICS Performance Evaluation Review, volume 37, pages 239โ250. ACM, 2009.
[22] P. Jorion. Value at Risk: The New Benchmark for Managing Financial Risk.MacGraw-Hill international editions: Finance series. McGraw-Hill, 2001.
[23] Srikanth Kandula, Dina Katabi, Bruce Davie, and Anna Charny. Walking thetightrope: Responsive yet stable traffic engineering. In ACM SIGCOMM Com-puter Communication Review, volume 35, pages 253โ264. ACM, 2005.
[24] Srikanth Kandula, Ishai Menache, Roy Schwartz, and Spandana Raj Babbula.Calendaring for wide area networks. In Fabiรกn E. Bustamante, Y. Charlie Hu,Arvind Krishnamurthy, and Sylvia Ratnasamy, editors, ACM SIGCOMM 2014Conference, SIGCOMMโ14, Chicago, IL, USA, August 17-22, 2014, pages 515โ526. ACM, 2014.
[25] F. A. Kuipers. An overview of algorithms for network survivability. CN,2012:24:24โ24:24, January 2012.
[26] Alok Kumar, Sushant Jain, Uday Naik, Nikhil Kasinadhuni, Enrique Cauich Zer-meno, C. Stephen Gunn, Jing Ai, Bjรถrn Carlin, Mihai Amarandei-Stavila, Math-ieu Robin, Aspi Siganporia, Stephen Stuart, and Amin Vahdat. Bwe: Flexible,hierarchical bandwidth allocation for wan distributed computing. In Sigcommโ15, 2015.
[27] Praveen Kumar, Yang Yuan, Chris Yu, Nate Foster, Robert Kleinberg, PetrLapukhov, Chiun Lin Lim, and Robert Soulรฉ. Semi-oblivious traffic engineering:The road not taken. In 15th USENIX Symposium on Networked Systems Designand Implementation (NSDI 18), pages 157โ170, Renton, WA, 2018. USENIXAssociation.
[28] Youngseok Lee, Yongho Seok, Yanghee Choi, and Changhoon Kim. A constrainedmultipath traffic engineering scheme for MPLS networks. In ICC, pages 2431โ2436. IEEE, 2002.
[29] George Leopold. Building Express Backbone: Facebookโs new long-haul network.http://code.facebook.com/posts/1782709872057497/, 2017.
[30] Hongqiang Harry Liu, Srikanth Kandula, Ratul Mahajan, Ming Zhang, andDavid Gelernter. Traffic engineering with forward fault correction. In ACMSIGCOMM 2014 Conference, SIGCOMMโ14, Chicago, IL, USA, August 17-22,2014, pages 527โ538, 2014.
[31] Houra Mahmoudzadeh. Robust Optimization Methods for Breast Cancer Radia-tion Therapy. PhD thesis, University of Toronto, 11 2015.
57
[32] A. Markopoulou, G. Iannaccone, S. Bhattacharyya, C. N. Chuah, Y. Ganjali,and C. Diot. Characterization of failures in an operational ip backbone network.IEEE/ACM Transactions on Networking, 16(4):749โ762, Aug 2008.
[33] Jeffrey C Mogul, Rebecca Isaacs, and Brent Welch. Thinking about availabilityin large service infrastructures. In Proceedings of the 16th Workshop on HotTopics in Operating Systems, pages 12โ17. ACM, 2017.
[34] Dritan Nace and Michal Piรณro. Max-min fairness and its applications to routingand load-balancing in communication networks: A tutorial. IEEE Communica-tions Surveys and Tutorials, 10(1-4):5โ17, 2008.
[35] R Tyrrell Rockafellar and Stanislav Uryasev. Optimization of conditional value-at-risk. Journal of risk, 2:21โ42, 2000.
[36] R Tyrrell Rockafellar and Stanislav Uryasev. Conditional value-at-risk for generalloss distributions. Journal of banking & finance, 26(7):1443โ1471, 2002.
[37] Farhad Shahrokhi and David W. Matula. The maximum concurrent flow prob-lem. J. ACM, 37:318โ334, 1990.
[38] Martin Suchara, Dahai Xu, Robert Doverspike, David Johnson, and JenniferRexford. Network architecture for joint failure recovery and traffic engineer-ing. Proceedings of the ACM SIGMETRICS Joint International Conference onMeasurement and Modeling of Computer Systems, 2011.
[39] Hong Zhang, Kai Chen, Wei Bai, Dongsu Han, Chen Tian, Hao Wang, HaibingGuan, and Ming Zhang. Guaranteeing deadlines for inter-data center transfers.IEEE/ACM Transactions on Networking (TON), 25(1):579โ595, 2017.
58
Recommended