Link Failure Monitoring Using Network Coding Hamed Firooz Sumit Roy, Linda Bai firooz,sroy,[email protected]

Link Failure Monitoring Using Network Coding

Hamed FiroozSumit Roy, Linda Bai

firooz,sroy,[email protected]

Fundamentals of Networking Lab(FunLab)

Outline Network Tomography

Introduction (Network Monitoring) Approaches:

Deterministic vs. Stochastic Active vs Passive

Challenges: Overhead, Identifiability Network Coding

Applications to network monitoring: new method Optimization : speed/complexity tradeoffs OPNET Implementation


Network Tomography

Networks: set of nodes, links modeled as graph G(V,E)

Network monitoring Involves collection of network performance

statistics (link delay, link loss or failure status)

Important for QoS guarantees (media streaming, interactive video applications)

Challenges Choice of appropriate measurement

technique and algorithmics

1

2

34

56

7

8

Node or network

A Logical Network

G(V,E)


Measurement Methods Node-oriented: These

methods are based on cooperation among network nodes, e.g. ping or traceroute Using Ping, round trip delay to

every node can be measured. Uses Internet control message

protocol (ICMP) packets Many routers do NOT

respond to these packets Many service providers do not

own the entire network

R

R

R1lD

2lD

1

2


Measurement Methods

Edge-oriented: Access is available to nodes at the edge only (and not to any in the interior) Does not require exchanging

special control messages between interior nodes

Inverse problem: estimate link level status from end-2-end (path level) measurements

S

SS

S

Network(?)Network(?)


Measurement Methods

Active (sending probe packets)

- Adds overhead to normal data traffic by

introducing new control packets Passive (insitu traffic analysis)

- No overhead; temporal and spatial dependence might bias measurement

Our method: edge-oriented, active network tomography Given a network, and a limited number of

end hosts, when can we infer failure status of the links?

Network?


End1

End2 End3

router1

link1

link2

link3

11032

10131

01121

321

EndEnd

EndEnd

EndEnd

linklinklink

Routing matrix A

End-to-End Probing

• Probes are inserted into a data stream, and end-to-end properties on that route measured.• Probes are exchanged between end nodes using routing matrix of the graph


End-to-End Probes

Routing matrix relates link attribute to route attribute

For some parameters like delay or path loss, this relation is linear under some assumptions

End1

End2 End3

R

1

23

3

2

1

110

101

011

32

31

21

l

l

l

EndEnd

EndEnd

EndEnd

D

D

D

D

DD


Deterministic

Link attributes (e.g. delay) are considered unknown, constant

Goal: estimate constants Link attributes are typically time varying

method is suitable for periods of local ‘stationarity’


Stochastic

Link attribute specified by a suitable probability distribution e.g. link delay follows a Gaussian distribution

Estimation problem: unknown model parameters

based on path observation in the presence of additive noise


Deterministic vs. Stochastic Methods

Stochastic Bayesian - requires a prior distribution

incorrect choice leads to biases in the estimates More computationally intensive

Deterministic Lower complexity but suffers from generic non-

identifiability


Link Failure Model

congestedis1

okis0

i

il l

lx

i

End1 End2R1 R2

1 2 3

..1

okis,,ofall0 32121 wo

llly endend

Define an indicator function for status of each link


Binary Deterministic Model

32121 lllendend xorxorxy

y = Ax

A: N-by-M binary routing matrix

x: M-by-1 binary vector, the status of each link

y: N-by-1 binary vector, the status of each path (measurements)

End1 End2R1 R2

1 2 3


Failure Monitoring

Network G(V,E) with set of paths P x, y are binary vectors A path is congested if at least one of its links

is congested

}1,0{,

110

101

011

32

31

21

1

3

2

1

3

2

1

321

l

l

l

l

x

x

x

x

EndEnd

EndEnd

EndEnd

y

yy

lll

32

31

21

)(

)()(

3

2

1

ll

ll

ll

xORx

xORxxORx

y

yy

|||| }1,0{,}1,0{ PE yx

End1

End2 End3

Router

1

2 3


Identifiability y = Ax Problem: Estimate x from y with

A (N-by-M) : binary routing matrix x (M-by-1) : binary link failure status y (N-by-1) : end-to-end measurements

Identifiability: a network is identifiable if y = Ax has a unique solution Usually, M ( # of links in network) >> N (# of

measurements), so network is generically NOT identifiable.

6 links, 3 End-to-End routes N=6, M=3


Identifiability: Binary Model Solution: limit (maximum) number of failed links

inside the network Suppose at most k links can fail simultaneously

Defn: k-Identifiability Network is k-identifiable if

from end-to-end observation it is possible to uniquely identify up to k congested links

k01|E| x

2121020121 ,k,ks.t., AxAxxxxxxx Only one link can be congested

1016 x


Example of 1-identifiability

1016 x

-- 1 2 3 4 5 6

0 1 1 0 0 0 0

0 1 0 1 1 0 1

0 0 0 0 0 1 1

21y

31y

32y

1

2 3

45

6

111000

100101

001011

A


Example: k=2 identifiability

2016 x

0

1

1

32

31

21

y

y

y

y

1

2 3

45

6

0

1

1

32

31

21

y

y

y

y

Ambiguity

111000

100101

001011

A


1-Identifiability

A network with an intermediate degree two node is not 1-identifiable If path End1End2 is congested, it is

impossible to determine which link among 1 and 2 is congested .

Necessary but not sufficient!

`

`

l1

l2

End1

End2

11 211 EndEndl yx

11 212 EndEndl yx


k=1 Identifiability

1-identifiability Theorem: End-to-End probe based measurements can detect a unique congested link in a network if and only if there are no two identical columns in the network routing matrix

10000000

01000000

00011000

00100100

00000010

00000001

P1 P3

P1

P3


k- identifiability

k-identifiability Theorem: End-to-End probe based measurements can detect a unique congested link in a network only if there are no k+1 dependent columns in the network routing matrix

10000000

01000000

00011000

00100100

00000010

00000001


Example: k=2 identifiability

2016 x

0

1

1

32

31

21

y

y

y

y

1

2 3

45

6

0

1

1

32

31

21

y

y

y

y

Ambiguity

111000

100101

001011

A


Shortest Path Routing Revisited

Packets are sent on shortest path between two end nodes

- sub-graphs = tree starting from a boundary (source) node Node 4 has degree two in all graphs

But node 4 has degree four in the original network


Revisiting Shortest Path Routing What if we could change routing

matrix ?

Example: in place of shortest path routing, route packets through longer paths, e.g. n124n2

Now network is 1-identifiable ! Intrinsic limitation for end-to-end

measurement methods based on shortest path routes probes transmitted along such paths

contain only minimum information


Solution

Look to exchange probes between boundary nodes via other (non-shortest) paths?

Changing the routing tables violates tomography assumption

Use Network Coding; exploit broadcast nature of network coding, a transmitted probe will traverse almost every path between two boundary nodes


Network Coding: Short Review Present: routers just forward incoming packets, i.e. copy

the packets on an input link onto the output links Proposed: What if each node in a network performs

some computation on received data prior to forwarding?

y1

y2

y1

y2

y1

y2

y3

f1(y1,y2,y3)

f2(y1,y2,y3)


How does NC work? (1)

sender sreceiver t2

receiver t1

A

B

C D

“Butterfly” network: All links have the same capacity 1 b/s s wants to send data bits a, b to both t1 and t2

Bottleneck is CD


How does NC work?(2)

Node C XORs received messages on each of its links

sender sreceiver t2

receiver t1

A

B

XOR

Da

b a+b

a

b


How does NC work?(3)

t1 and t2 know both a and b Now s can send data at rate 2 b/s/receiver

sender sreceiver t2

receiver t1

A

B

XOR

Da

b a+ba+ba+b

a

b


Linear Network Coding

Network Coding is a coding at layer three The coding is conducted over the finite field Fu, u=2q

each coded symbol can be represented by q-bits within an IP layer frame

Signal Y(j) on an outgoing link j of node v, is a linear combination of signals Y(i) on incoming link i of v: We assume there is no process generated at node v

})(:{

)()(vldll lYjY


Received Symbols

Pi : i-th route from source to destination Source sends α over Pi

βi depends on topology G hence βi(G)

Coef. NCPath )(

),(2

i

q

i

Plli

iPl

l

G

FGy

S D

γ1 γ2

γ3

γ4 γ5

α)(121 Gy


Received Symbols: Linear Model

ek one of source outgoing links Pek : collection of all paths between source and

destination starts at ek

Source sends αk over ek. By superposition destination receives

||

1, )(

ke

k

kei i

P

ieik

PP Pllk Gy

S D

γ1 γ2

γ3

γ4 γ5

e1

1eP

)()(11 ,2,11531211 eey

αα1



Source sends out symbols αk over ek using superposition once more

In vector format: y=αtβ(G) β(G) is total network coding vector

K

k

P

ieik

ke

kGy

1

||

1, )(

S D

γ1 γ2

γ3

γ4 γ5

e1

1eP

αα1

α2



Source sends symbols in M succ. time slots:

11 )( NNMM GAy

t

P

eN

P

eNe

P

eNeeN

Ke

KK

ee

G

,,,1,,2,11

2

221

1

1111)(


Link Failure Model

If a link is severely congested, packets are significantly delayed and assumed lost at the destination

We model the network with link in congestion state by its edge deleted subgraph denoted by G(V,E)

S D

γ1

γ3

γ4 γ5


Link Failure Model Total network coding vecor of G(V;E), β(G) is

different from β(G)

if the congested link doesn’t belong to i-th path from source to destination, Pi, it will not affect packets going through those paths It is zero otherwise

..0

)()()( ,

,wo

dPlifGG

ieei

leikk

k

S D

γ1 γ2

γ3

γ4 γ5

e1

e2

211 )( G

542 )( G

0)(11 lG

)()( 22 1GGl

1


Link Failure Model

Training sequence is A y : vector of symbols observed at the

destination in M time slots with link congested

Potential for identifying: received symbols change uniquely in response to link congestion

11 )( NlNMlM GAy

1 2

1 1

1 1

lM M

l lM M

y y

y y


Example

11, 1 1 1e

S D

11

2

3 2

e1

e2

1eP

2eP

12, 1 2 2 3e

22, 3 2 1e

1

( ) 3

1

G

1 1 2

3 3 3A

-- e1 e2 1

2 3

1st time slot 0 2 2 3 1 1

2nd time slot 2 3 1 0 1 3


Theorem 1: Sufficient Conditions

If Rank(A)= deg(S), and for all Pek set of paths between source and

destination starting at ek

then

jj

P

jejj

ke

i

00||

1,

EllGAGA

ElGAGA

ll

l

21,)()(

)()(

21

(more next slide)


Theorem 1

Condition means For a set of paths having ek in common, Pek , NC

coefficient of the paths are independent !

jj

P

jejj

ke

i

00||

1,

S D

γ1 γ2

γ3

γ4 γ5

e1

e2

1eP

2eP

t independen531,2

21,1

1

1

e

e

54,1 2 e

t

P

eN

P

eNe

P

eNee

Ke

KK

ee

,,,1,,2,1

2

221

1

1111

Independent Independent


Example

11, 1 1 1e

S D

11

2

3 2

e1

e2

1eP

2eP

12, 1 2 2 3e

22, 3 2 1e

1

( ) 3

1

G

1 1 2

3 3 3A

( ) 2 deg( )Rank A S

-- e1 e2 1

2 3


2nd time slot 2 3 1 0 1 3

Independent


Complexity/Speed First condition of Theorem 1:

In previous example M=2=deg(S) Number of time slots: at least the number of

outgoing links of source Is it possible to decrease number of time

slots? faster monitoring Possible by increasing number of bits in LNC

coeff. more complexity

)deg( implies)deg()Rank( SMSA NM


Example

q=3 A=[1 1 4]

-- e1 e2 1

2 3


S D

11

2

3 2

e1

e2


Theorem 2: Complexity/Speed tradeoff

Ni=|Pi| q bits per symbol are used in network coding M number of (desired) time slots Let Z={1,2,…,K} K degree of source ZM: collection of all partitions of Z with size M

K=3, 2 Z={1,2,3} ZM={ {{1,2},{3}} , {{1,3},{2}} , {{2,3},{3}} }

M

ijiiMM HHZHHHHZ

121 },|},...,,{{

S

K links


Theorem 2: Complexity/speed tradeoff

Network is 1-identifiable if

Rank(A)=M

iMi Hj

jiZMiH

Nq maxmin},...,1,{


Theorem 3: Random LNC

Random linear network coding is a distributed approach achieving capacity asymptotically

Intermediate node choose their NC coefficients uniformly from the elements of Fu (u=2q)

Exponential increase with q (number of bits) and M (number of time slots)

Quadratic decrease with size of network

1( is1-identifiable) 1 | | (| | 1)( )

2M

qP G E E


Multi-source Multi-destination

So far, considered only Single source Single destination

Easily extendable to Multi-source Multi-destination


Simulation Simulation environment

OPNET 14.5 MATLAB 7.1 (finite field operations)

Evaluation University of Washington’s Electrical Engineering

network Thirteen subnets 3 backbone routers Full Duplex Ethernet links


Simulation Set-Up Implementation of Network Coding (NC) within OPNET We employ network coding at transport layer (instead

of IP layer) Easier to implement

Routers model is modified to distinguish between non-NC/NC packets through the use of a flag bit within the UDP header NC packets are sent for separate processing non-NC packets are processed normally

We assign a q-bit field called LNC field within the TCP/UDP header, for linear network coding.

UDP packetLNC field1


RECEIVE/SEND interface

Inherently network coding operates on unidirectional links

Each interface within a router mode is designated as a SEND or RECEIVE interface only for the network coded packets operating regularly with non-network coded

packets Finite field operation is done in MATLAB

Using MATLAB API within OPNET


RECEIVE/SEND


Evaluation


UW EE Network


UW EE Network-lookup table



Network Tomography: A Stochastic Model [1]

Passage of probes can be modeled as two stochastic process: {X(i)} and {Z(i)} for each node k

Z(i) time delay process of link k

X(i) called bookkeeping process: cumulative probe from root to k

[1] V. Arya, N. Duffield, D. Veitch “ Temporal Delay Tomography”, IEEE Infocom 2008


Network Tomography: Stochastic Method

Discretize delay D={0,b,2b,…,mb,∞} mb is delay threshold X(i)=X’(i)+Z(i)

k-1

k

X(i)

Z(i)

’

X’(i)

..

if

if

])(Pr[

1

0

])(|)(Pr[

wo

vd

vd

vdiZ

viXdiX

k

ll

Documents

Link Failure Monitoring Using Network Coding Hamed Firooz Sumit Roy, Linda Bai firooz,sroy,[email protected]