1

Interconnection Networks

• Direct

• Indirect

• Shared Memory

• Distributed Memory (Message passing)

2

Topology• Diameter: Longest path length between two

processorsd = max {dmin(vi, vj) | for all vi, vj V}

• Node connectivitymin. # of nodes to be removed for the network to

be disconnected

node connectivity r There are r disjoint paths between every pair of nodes.

3

Topology• Fault diameter

Suppose the node connectivity = r, diameter of the network with at most r-1 faulty nodes

• Bisection widthThe # of edges to be removed to separate the

graph into two equal parts.

• Cost1) # of nodes & # of edges

2) Bisection width.

4

• cw (Channel width): # of wires connecting two nodes.

• cr (Channel rate): # of bits/sec/wire.

• Channel bandwidth = cw cr

• Bisection bandwidth

= bisection width channel bandwidth

• Cost Bisection bandwidth

bisection bandwidth is constant

5

• Hypercube: N = 22n

bisection width = 22n-1

channel width = wh

bisection bandwidth = 22n-1 wh

• 2-D Torus: N = 22n

bisection width = 2 2n

channel width = wd

bisection bandwidth = 2n+1 wd

2n+1 wd = 22n-1 wh

wd / wh = 22n-1 / 2n+1 = 2n-2 = 2n/4 = N /4

• 3-D Torus: w3d / wh = (N)1/3/4

6

Network Topology

• Graph Model:Processors Nodes

Wires joining processors Link

7

Linear Array

1

• # of processors = n

• # of links = n-1

• Connection (i, i+1) for i=1,2, … , n-1

• Diameter(Longest Path Length)= n-1

• Communication delayworst case: diameter

Average case: 1/n2 dij

32 n. . . . .

8

Hypercube• Hamming Distance

DH(x, y) = # of positions in which x & y differ

x & y are binary vectors

DH(1100, 0111) = 3

• Nodes are labeled as n-bit binary

• Two nodes, x & y, are adjacent if DH(x, y) =1

001

101 111

011

110

010000100

9

Product of graphs• G1 = (V1, E1) & G2 = (V2, E2)

G1 G2 = G = (V, E)= (V1 V2 , E)

Two nodes in G, ((v1, v2), (v3, v4)) are adjacent

1) if v1= v3 and (v2, v4) E2 or

2) if v2= v4 and (v1, v3) E1

Q2 Qn-1= Qn or Qk Qn-k = Qk

001

101 111

011

110

010000100

10

Routing: A BA=(an-1, an-2, … , a1, a0)

B =(bn-1, bn-2, … , b1, b0)

an-1, an-2, … , a2, a1, a0

an-1, an-2, … , a2, a1, b0

an-1, an-2, … , a2, b1, b0

an-1, an-2, … , b2, b1, b0

••• an-1, bn-2, … , b2, b1, b0

bn-1, bn-2, … , b2, b1, b0

# of steps required

= DH(A, B)

Diameter of the network

= n = log2N

Average comm. delay

= 1/N (( )+2( ) … n( ))

= n2n-1/N = n2n-1/2n

= n/2

n1

n2

nn

11

Connectivity

Let DH(x, y) = d

n : node disjoint paths between x & y

d : paths have length d

n-d : paths have length d+2

Example: 110000 111111, d(110000,111111)=4

12

11000

010000 100000 110001 110010 110100 111000

010001 100001 110011 110110 111100 111001

010011 100011 110111 111110 111101 111011

010111 100111

011111 101111

111111

Fault Diameter = n+1

13

De Bruijn Network

• # of nodes, N = 2n

• Node A=(an-1, an-2, …, a0 ) is adjacent to

1. an-2, an-3, … , a1, a0, 0

2. an-2, an-3, … , a1, a0, 1

3. 0 an-1, an-2, … , a1

4. 1 an-1, an-2, … , a1

14

De Bruijn NetworkN=23 = 8

2n - 4 nodes have degree 4

2 nodes have degree 3

2 nodes have degree 2

# of links 42n / 2 = 2n+1

000

001

100

010

011

101

110

111

15

Routing: A B A=(an-1, an-2, … , a1, a0)

B =(bn-1, bn-2, … , b1, b0)

an-1, an-2, … , a2, a1, a0

an-2, an-3, … , a1, a0, bn-1

an-3, an-4, … , a0, bn-1, bn-2

an-4, an-5, … , bn-1, bn-2, bn-3

••• bn-1, bn-2, … , b2, b1, b0

log2N

• Example

A = 1010

B = 1101

1010

0101

1011

0110

1101

16

an-1, an-2, … , a1, a0

an-2, … , a1, a0, 0

an-3, … , a0, 0 , 0

… 0, 0, 0, … , 0, 0

b0, 0, 0, … , 0, 0

b1, b0, 0, … , 0, 0…

bn-2, bn-3, … , b0, 0

an-2, … , a1, a0, 1

an-3, … , a0, 1 , 1

… 1, 1, 1, … , 1, 1

b0, 1, 1, … , 1, 1

b1, b0, 1, … , 1, 1 …

bn-2, bn-3, … , b0, 1

bn-1, bn-2, … , b1, b0

2-disjoint paths

17

k-ary n-cube

• Multidimensional torus

00 01 02 03

10 11 12 13

20 21 22 23

30 31 32 33

18

k-ary n-cube• Given a node (an-1, an-2, … , a1, a0) adjacent

to 2n nodes given by(an-1, an-2, … , a1, a0±1)

(an-1, an-2, … , a1 ±1, a0):

(an-1 ±1, an-2, … , a1, a0)

• k=8, n=3 node (3 2 4) is adjacent to (3, 2, 5), (3, 2, 3)(3, 3, 4), (3, 1, 4)(4, 2, 4), (2, 2, 4)

• N = # of nodes in k-ary n-cube = kn

19

00 01 02 03

10 11 12 13

20 21 22 23

30 31 32 33

001

101 111

011

110

010000100

k=4, n=2k=2, n=3

Hypercube

20

0• k=8

• Lee Distance

DL ((a3, a2, a1, a0 ), (b3, b2, b1, b0 ))

= min (ai - bi, bi - ai ) mod k

DL (1 2 3, 3 2 1) = min (1-3, 3-1) + min (2-2, 2-2) + min (3-1, 1-3) = 2 + 0 + 2 = 4

4

6 2

1

35

7

21

Routing: A BA=(an-1, an-2, … , a1, a0)

B =(bn-1, bn-2, … , b1, b0)

an-1, an-2, … , a2, a1, a0 ±1

an-1, an-2, … , a2, a1, a0 ±2:

an-1, an-2, … , a2, a1, b0

an-1, an-2, … , b2, a1 ±1, b0

:

an-1, an-2, … , a2, b1, b0

:

bn-1, bn-2, … , b2, b1, b0

• Example (k=5)

(4,4,2,1) (2,1,4,4) (4,4,2,1)

(4,4,2,0) (4,4,2,4) (4,4,3,4) (4,4,4,4,) (4,0,4,4)

(4,1,4,4)(3,1,4,4)(2,1,4,4)

# of steps = DL (A, B)

22

Torus of size kn-1, kn-2, … , k1, k0

• Mixed radix number system

• Processor (an-1, an-2, … , a1, a0 )

0 ai < ki-1 i = 0, 1, 2, … n-1

Decimal value

= an-1 (kn-2 kn-3, … , k0) + an-2 (kn-3 kn-4, … , k0) + …+ a0

• Two nodes, A=(an-1, an-2, … , a1, a0) &

B =(bn-1, bn-2, … , b1, b0)

are adjacent if DL (A, B) = 1

23

Torus of size kn-1, kn-2, … , k1, k0 • Example: (854)

(3,2,1) = 3(54) + 24 + 1 = 60 + 8 +1 = 69 4 69

Decimal to mixed radix 5 17 … 1 69 = (3,2,1) 3 … 2

• Example: (888)(3,2,4) = 382 + 28 + 4 8 212

= 364 + 16 +4 = 212 8 26 … 4 212 = (3,2,4) 3 … 2

• Radix k-number(an-1, an-2, … , a1, a0)= an-1 kn-1 + an-2 kn-2 + … + a0

24

Embedding Cycles• Hypercube- Gray codes

0 00 000 00001 01 001 0001

11 011 001110 010 0100

110 0110111 0111101 0101100 0100

110011011111 …

25

Mapping (Binary to Gray)

000 000

001 001

010 011

011 010

100 110

101 111

110 101

111 100

f(xn-1xn-2 … x0)

= (gn-1gn-2 … g0)

where

gn-1 = xn-1

gi = xi xi+1

i= n-2, n-3, … ,0

10110 11101

26

k-ary n-cube

• f: Radix Gray

f(xn-1xn-2 … x0) = (gn-1gn-2 … g0)

gn-1 = xn-1

gi = xi - xi+1 mod k

for i= n-2, n-3, … ,0

27

3-ary 2-cube

Radix Gray0 00 001 01 012 02 023 10 124 11 105 12 116 20 217 21 22 k=58 22 20 2431 2243

00 01 02

10 11 12

20 21 22

28

De Bruijn Network

23 - nodesj+3 + sj+1 + sj= 0 sj+3 = sj+1 + sj mod 2

Initial conditions s0 =1, s1 = 0 and s2 = 1

s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 1 0 1 1 1 0 0 1 0 1

101011111110100000001010101

000

001

100

010

011

101

110

111

29

How to choose this different equation?

• Take an n-r order difference equation whose characteristic equation is a primitive polynomial of degree r.Primitive Polynomial

x2+x+1 x7+x3+1

x3+x+1 x8+x4+ x3+x2+1

x4+x+1 x9+x4+1

x5+x2+1 x10+x3+1

x6+x2+1 sj+5 = sj+2 + sj

30

Disjoint Cycle

• sj+3 = sj+1 + sj+1

Initial conditions s0 =1, s1 = 0 and s2 = 1s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 1 0 1 0 0 0 1 1 0 1

101010100000001011110