Upload
erin-young
View
215
Download
1
Embed Size (px)
Citation preview
1
Outline of Contents① Issues, Criteria and Topics.
② Motivations behind This Line of Research
③ Survey of Typical Networks
④ My Research Work and Contributions
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
2
① Issues / Criteria / Topics
Topology and Analysis Routing and Communication Mapping and Simulation Algorithm and Computation VLSI Design and Construction
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
3
① Issues / Criteria / Topics
Topology and Analysis degree,diameter,average distance, bisection
bandwidth,connectivity,symmetry,recursiveness, scalability,Hamiltonian path
Routing and Communication Mapping and Simulation Algorithm and Computation VLSI Design and Construction
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
4
① Issues / Criteria / Topics
Topology and Analysis Routing and Communication
Pattern: routing,broadcasting,multicasting,gossip Evaluation:Easy routing,deadlock-free,delay,traffic
density, Control Strategy: centralized/distributed,
deterministic/adaptive,minimal/non-minimal, Switching: circuit/packet, wormhole, virtual cut-through
Mapping and Simulation Algorithm and Computation VLSI Design and Construction
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
5
Topology and Analysis Routing and Communication Mapping and Simulation
To compare the computing power G H Smallest possible dilation and congestion
Algorithm and Computation VLSI Design and Construction
① Issues / Criteria / Topics
embedding
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
6
① Issues / Criteria / Topics
1 2 3 4 5 6 7 8
1
8
2
7
3
6
4
5
1 2 3 4 5 6 7 8
12
3
45
6
7
8
Dilation=2, congestion=2
Dilation=7, congestion=2
How to embed a ring into a line:
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
7
① Issues / Criteria / Topics
Topology and Analysis Routing and Communication Mapping and Simulation Algorithm and Computation
PRAM model is unpractical. Basic algorithms:
sorting,searching,permutation,matrix multiplication,bit reversal, graph algorithm,iteration method,symbolic computing.
VLSI Design and Construction
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
8
① Issues / Criteria / Topics
Topology and Analysis Routing and Communication Mapping and Simulation Algorithm and Computation VLSI Design and Construction
Wafer-scale integration Layout design: area and wire length,wire
area,crossing number,node cost and modularity Applicable to the board design
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
9
② Motivations
A branch of combinatorics Structures for building multiprocessor
and multicomputer More than structural:
ASIC:implementing special parallel algorithm on VLSI/Wafer scale.
P2P overlay topologies Data Center Networking SoC or NoC for Many-core
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
10
② Big Names F. Thomson Leighton (Theory) http://theory.lcs.mit.edu/~ftl/ Lionel M. Ni (wormhole) http://www.cps.msu.edu/~ni/ Arnold L. Rosenberg (Butterfly) http://www.cs.umass.edu/~rsnbrg/ Burkhard Monien (Embedding) http://www.uni-paderborn.de/fachbereich/AG/monien/ Laxmi N. Bhuyan (Multi-stage) http://www.cs.tamu.edu/faculty/bhuyan/ Kenneth E. Batcher (Bitonic) http://nimitz.mcs.kent.edu/~batcher/index.html Ivan Stojmenovic (Honeycomb) http://www.csi.uottawa.ca/~ivan/ Ke Qiu (Star and Pancake) http://dragon.acadiau.ca/~kqiu/home.html Wei Zhao (Routing) http://www.cs.tamu.edu/faculty/zhao/ S. Lennart Johnsson (Hypercube) http://www.cs.uh.edu/~johnsson/ Satoshi Fujita(gossip) http://www.se.hiroshima-u.ac.jp/~fujita/ William J. Dally (k-ary n-cube) http://www.ai.mit.edu/people/billd/ S. Yalamanchili (Engineering ct) http://users.ece.gatech.edu/~sudha/ C.E. Leiserson (Parallel Algorithms) http://supertech.lcs.mit.edu/~cel/ Kai Hwang (Benchmarking) http://ceng.usc.edu/~kaihwang/
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
11
② A Special Journal on Interconnection Networks
D Frank Hsu (Fordham University) Bruce M Maggs (Carnegie Mellon )Jean-Claude Bermond (CNRS/INRIA/UNSA)Tse-Yun Feng (Penn State University)Yoji Kajitani (Tokyo Institute of Tech)F Tom Leighton (MIT)Guo-Jie Li (Chinese Academy of Sciences)Burkhard Monien (University of Paderborn)Howard Jay Siegel (Purdue University)Tom Stern (Columbia University)
Among the editorial board:
Website: http://journals.worldscinet.com/join
12
③ Survey of Network Topology
The following discussion of the properties of interconnection networks is based on a collection of nodes that communicate via links. In an actual system the nodes can be either processors, memories, or switches. Two nodes are neighbors if there is a link connecting them.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
13
③ Network Parameters
Channel width: Number of wires that are used per channel (i.e. the number of bits
that can be transmitted simultaneously on one channel).
Channel direction: The direction in which the messages can be transmitted. Unidirectional : can send messages in just one direction Bi-directional : support two-way communication over the same
channel.
Bisectional width (BW): It is defined as the number of channels that are cut when the
network is divided into two equal parts.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
14
③ Network Parameters
Node degree (): Number of channels connecting a node to its neighbors. In practice the degree of a topology has an effect on cost, since the
more links a node has the more logic it takes to implement the connections.
Network diameter (D): Maximum distance between any two nodes in the network.
Cost Effectiveness: x D, Problem of Dense graph: Given the fixed degree and diameter, how
to design a graph which can contain as many nodes as possible? Average distance:
= Vji n
jid
,2
),(
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
15
③ Network Parameters
Number of links (l): The total number of channels in the network.
Symmetry: A network is said to be symmetric if it looks the same
from every node. (E.g. Hypercube, crossbar) Recursiveness
A large-size network consists of two or many small-size networks of the same kind, such as tree and hypercube. This property renders the network suitable for solving divide-and-conquer algorithm
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
16
③ Crossbar switch
P0 –> M1P1 –> M3P2 –> M2P3 –> M0
Crossbar switch
N*N switches (N processors N memory modules). The switch configures itself dynamically to connect a processor to a memory
module. No contention -- Supports N! permutations. Costly, hard to scale, wastes switches for most patterns.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
17
③ Crossbar switch
Easy for broadcasting and also for any permutation. As long as each processor wants to communicate with a different memory there will be no contention -- Supports N!
permutations. If two or more processors need to access the same
memory, however, one will be blocked until the switch reconfigures itself.
A crossbar has a short diameter - information needs to pass through only one switching element on a path from one edge to another.
Poor scalability -- If there are N processors and N memories, there are N2 interior switches. Adding another processor or memory means adding another N interior nodes.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
18
③ Multistage Network
P1
P2
P3
PP-1
P1
P2
P3
PP-1
Stag
e 1
Stag
e 2
Stag
e 3
Stag
e k
Systems built with these topologies have processors on one edge of the network, memories or processors on another edge, and a series of switching elements at the interior nodes.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
19
③ Multistage Network : Omega Network
p processors and log2 p stages Each stage consists of a perfect shuffle
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
20
③ Multistage Network : Butterfly
ButterFly Network
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
21
③ Multistage Network
Built from small (e.g., 2X2 crossbar) switch nodes, with a regular interconnection pattern.
In order to send information from one edge to another, the interior switches are configured to form a path that connects nodes on the edges.
The information then goes from the sending node, through one or more switches, and out to the receiving node.
The size and number of interior nodes contributes to the path length for each communication, and there is often a ``setup time'' involved when a message arrives at an interior node and the switch decides how to configure itself in order to pass the message through.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
22
③ Multistage Network: Problems
Generalized MINs: for in Inputs and jn outputs, use i×j switches at n stages.
Bens networks: with 2logn-1 stages, it becomes a nonblocking network, allowing all permutations.
Fault tolerance: form all switches at a stage as a ring.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
23
③ Line, Ring and Fully Connected
Linear Array: the simplest topology. Fully Connected: Direct connection between every pair of processors, Highest cost, Similar to crossbar in some properties. Ring: Each node in the ring is connected to only two other nodes.Chordal Ring: A compromise between Ring and Fully Connected Network.
Fully connected: Degree = N-1, Diameter = 1, BW = (N/2)2, Links = N*(N-1)/2, Symmetric. Ring: Diameter = N/2, Degree=2, BW = 2, Symmetric
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
24
③ Mesh and Torus
Mesh: Deg = 2,3,4, Diameter = 2*Sqrt(N), Bisect= Sqrt(N), Easy to build, scalable .k-ary n-cube: Generalization of mesh-like Networks
Mesh Topology 3D Torus2D Torus
2D torus: Meshes with ``wraparound'' connections, e.g. the node at the top of the grid has an ``up'' link that connects to the node at the bottom of the grid (also left to right).
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
25
③ Mesh and Torus: Problems
Diameter of Asymmetric Networks: linear array, mesh, tree, shuffle exchange.
Torus and Midmiew: how to use the wraparound links to obtain the optimal diameter?
Layout of torus: Reduce the physical wire length.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
26
③ Hypercube
Can embed Hamiltonian cycle, mesh, tree, etc. Low latency, high bandwidth, but costly (high number of links). Hard to build (layout the chip and wires). Hard to scale up (As degree increases, number of I/O ports increases). Solution: CCC
For dimension D, Degree = D, Diameter = D, Bisect = 2(d-1) Nodes = 2D, Links = D*2(D-1)
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
27
③ Hypercube : assign node IDs
The nodes are numbered so that two nodes are adjacent if and only if the binary representations of their IDs differ by one bit. For example, nodes 011 and 010 are immediate neighbors.
011
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
28
③ Hypercube : Properties
Hamiltonian Cycle 2n-node hypercube contains I×J mesh, where I×J = 2n
Contains 2 binary tree with 2n-1 - 1 nodes Optimal in fault-tolerance Variants of hypercube:folded hypercube, hierarchical
hypercube,incomplete hypercube,CCC, shuffle-change. Y. Saad and M.H. Schultz, Topological Properties of
Hypercube, IEEE TC, July 1988. Generalized Hypercube, IEEE TC, April 1984.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
29
③ CCC: Solution to Hypercube :
F.P. Preparata, et al, The Cube-Connected Cycles: A Versatile Network for Parallel Computation, CACM, May 1981,G. Chen, et al, Tight Layouts of CCC, IEEE TPDS, Feb. 2000.G. Chen, et at, Layout of CCC without Long Wires, Computer Journal, in 2005.
000 001
100
110 111
011010
101
(000,0)
(000,2) (000,
1)
1
2
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
30
③ Tree and Star
Tree and Star
Tree: Degree = 1 (leaves), 2 (root), 3 (interior nodes), Diameter = 2logN, Bisect = 1. Tree/Star bottleneck: Expect at root
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
31
③ Tree
The tree has nodes of degree 1 (leaves), 2 (root), 3 (interior nodes). So it is not symmetric.
Short diameter: For depth k, the number of nodes is 2k - 1, and the diameter is
2k (or ~ 2 log N, at the same order as Hypercube). For example, a processor with 262,144 nodes would have
diameter 512 in a mesh but only 36 in a tree.
Bisection bandwidth is 1. It suffers from a serious bottleneck
-- Solution : Fat Tree
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
32
③ Fat Tree
Fat-tree layout
Fat-Tree
Expand link bandwidth at each higher level
BB
BB/8
B/4
B/2
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
33
③ Fat Tree
Leiserson's fat-trees [Lei85] – built in CM-5. The Fat Tree network provides uniform bandwidth between
any two end-points on a net. It does this by doubling the number of "up" paths as one goes "up" the tree from a processor in a leaf to the root node.
Given a fat tree of this type with h levels of switches, Number of processor nodes = 2h
Number of switch nodes = 2h- 1 Greatest distance between processor nodes = 2h
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
34
③ Packet Routing in Fat Tree
Note that a message going from PE2 to PE5 may choose any one of four paths from the lower left router to the one at the root. This means that all four PE's attached to the lower left router have a path available to them to reach another node.
4 port
8 port
PE2 PE5
four paths
PE1 PE3 PE10PE9PE8PE7PE6 PE16PE14PE13PE12PE11PE4 PE15
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
35
③ Comparison -- (N node, n=log 2 (N))
Degree Diameter Bisection No. of Links(d) (D) (B) (L)
1D mesh 2 N-1 1 N-1
2D mesh 4 2(N1/2 - 1) N1/2 2N- N1/2
star N-1 2 2 N-1
Ring 2 N / 2 2 N
2D torus 4 2(N1/2 / 2) N1/2 2N
Hypercube n n N/2 n N/2
Complete connected N-1 1 (N/2)2 N(N-1)/2
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
36
③ Example MPP Networks
Name Number Topology Bits Clock Link Bisect. YearnCube/ten 1-1024 10-cube 1 10 MHz 1.2 640 1987iPSC/2 16-128 7-cube 1 16 MHz 2 345 1988MP-1216 32-512 2D Mesh 1 25 MHz 3 1,300 1989Delta 540 2D Mesh 16 40 MHz 40 640 1991CM-5 32-2048 fat tree 4 40 MHz 20 10,240 1991CS-2 32-1024 fat tree 8 70 MHz 50 50,000 1992Paragon 4-1024 2D mesh 16 100 MHz 200 6,400 1992T3D 16-1024 3D Torus 16 150 MHz 300 19,200 1993
No standard MPP topology!
MBytes/second
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
37
③ Scalable Topology
Scalability Refers to the increase in the complexity of
communication (time) as more nodes are added. In a highly scalable topology more nodes can be added
without severely increasing the amount of logic required to implement the topology and without increasing the diameter.
Example : Doubling the number of nodes in a hypercube increases the degree by only 1 link per node, and likewise increases the diameter by only 1 path. An opposite example is linear array.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
38
③ Hypercube Problems
Embedding the links: For a 64K node (d=16) hypercube machine, there would be
512K (16x64K/2) links. With current technology, it is difficult to scale a hypercube
beyond about 4K nodes, with about 24K links.
Operation cost: As the number of dimensions increases, the nodes must do
more work to keep up with the incident message traffic. In fact, because most processors can handle only one I/O
transaction at a time, many hypercube algorithms operate on the principle of serializing processing by dimension. i.e. all pairs of nodes in one dimension communicate, then the next dimension, etc. See Ascend/Descend Algorithms
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
39
③ Symmetric Topology
Symmetric Rings, fully connected networks, and hypercubes
are all node symmetric. Trees and stars are not. A tree has three different
types of nodes, namely a root node, interior nodes, and leaf nodes, each with a different degree. A star has a distinguished node in the center which is connected to every other node.
When a topology is node asymmetric a distinguished node can become a communications bottleneck.
Importance of Symmetry: Node symmetry renders identical software at every node;edge symmetry avoids hot traffic spot in the network.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
40
Routing algorithms—XY Routing
P Sx,Sy source processor
P Dx,Dy destination processor
Shortest path – “manhattan distance”, abs(Sx-Dx) + abs(Sy-Dy)
Algorithm: 1.Reduce distance along X
dimension until 0 2.Reduce distance along Y
dimension until at P Dy,Dx
P 0,0 :source
P 3,2 :destination
P 3,2
P 0,0
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
41
③ Routing algorithms—XY Routing
Begin receive(M);If D=I accept(M)Elseif D • x >I • x
sendright(M) Elseif D • x <I • x sendleft(M)Elseif D • y >I • y
senddown(M)Elseif D • y >I • y sendup(M)Endif;
end
In a mesh, S I D More Information
M[S,D] M[S,D]
S(0,0)
D(3,2)
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
42
E-Cube Routing
E-Cube RoutingPs source processor Pd destination processor
Shortest path -- hamming distance between Ps and Pd
Routing taken in the following manner: 1. Exclusive-or the source and
destination processor numbers 2. Going from the Least
Significant Bit (lsb) to Most Significant Bit (MSB). Each position a “1” exists in the result of the exclusive, an edge is taken.
P000 source P110 destination
110
43
④ My Work and contributions Novel Interconnection Networks:
Shuffle-Ring, Shuffle-Cube, Wall Mesh.
Layout Design: MIDIMEW, CCC.
Routing in special Networks:
Shuffle-Exchange, Unidirectional Networks
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
44
④ New Topology: Shuffle-Ring
Definition: anan-1…a1a0 is connected to
shuffle an-1an-2…a0an
ring anan-1…a1 ( a0 ±1 )
Properties:
Simplicity
Constant degree
Keep all hypercube properties
Compact layout
•G. Chen et al, Shuffle-Ring: Overcoming the Increasing Degree of Hypercube, Proc. 2nd Int. Symp. on High-Performance Computer Architecture(HPCA-2), California, Feb. 1996, 130-138.
•G. Chen el al, Shuffle-Ring: A New Constant-degree Network, International Journal of Foundations of Computer Science, March 1998, 77-98
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
45
④ New Topology: Shuffle-Cube
Properties:
Constant degree
Keep all hypercube properties
Compact layout
•G. Chen, et al, CTSN: A New Fault-Tolerant Network, Proceedings of 13th International Conference on Parallel and Distributed Computing Systems(PDCS'2000), Las Vegas, Nevada, August 2000, 517-522.
0 1
2 3
4 5
6 7
c d
e f
8 9
a b
Definition: anan-1…a1a0 is connected to
shuffle an-1an-2…a0an
Cube anan-1…ai … a0 (i<k)
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
46
④ New Topology: Wall Mesh
Definition: ( x, y ) is connected to
( x ±1, y ) horizontally
and ( x, y ±1 ) vertically.
Properties:
Constant degree=3 or 4,
Equivalent to mesh in Computing Power
Logarithmic diameter
Easy layout
•G. Chen,et al, The Wall Mesh, Computer Architecture'97: Selected Papers of the 2nd Australasian Conference, Springer, 1997, 217-230.
•陈贵海等, 墙式网孔, 计算机学报, 2000年 4 月, 374-381 页。
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
47
④ New Topology: Isomorphism
•G. Chen et al, Comments on ``A New Family of Cayley Graph Interconnection Networks of Constant Degree Four'', IEEE Transactions on Parallel and Distributed Systems, December 1997, 1299-1300.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
48
④ Layout Design: MIDIMEW
•G. Chen, et al, Laying Out Midimew Networks with Constant Dilation, Lecture Notes in Computer Science(854), September 1994, 773-784.
•G. Chen et al, Optimal Layouts of Midimew Networks, IEEE Transactions on Parallel and Distributed Systems, September 1996, 954-961.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
49
④ Layout Design: Cube-Connected Cycles
G. Chen, et al, A Compact Layout of Cube-Connected Cycles, Proc. of 4th Int. Conf. on High Performance Computing, India, Dec. 1997, 422-427. G. Chen, et al, Tight Layouts of CCC, IEEE Transactions on Parallel and Distributed Systems, Feb. 2000, 182-191.G. Chen, et at, Layout of CCC without Long Wires, Computer Journal, Nov. 2001, Vol. 44, No. 5.
000 001
100
110 111
011010
101
(000,0)
(000,2) (000,
1)
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
50
④ Routing: Shuffle-Exchange Networks
•G. Chen, et al, Shortest-Path Routing in Shuffle-Exchange Networks, Lectures in Operations Research, August 1998, 142-153. •陈贵海等,洗牌交换网中的最短路由算法,计算机学报, 2001 年 1 月。•G. Chen, et al, An Algorithm for Optimal Routing in Shuffle-Exchange Networks, to appear in IEEE Transactions on Computers.
Definition: anan-1…a1a0 is connected to
shuffle an-1an-2…a0an
Exchange an-1an-2…a1a0
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
51
④ Routing: Unidirectional Networks
G. Chen, at al, A Distance-Vector Routing Protocol for Networks with Unidirectional Links, Computer Communications, Feb. 2000, 418-424. •G. Chen, et al, An Improved Routing Protocol for Networks with Unidirectional Links, ICPP, Spain, Sept. 2001.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
Questions?
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
53
Assignment 11) Prove any graph/network has a longest path?
2) Choose one of the following
2.1) Write the routing algorithm for complete binary tree?
(Suppose nodes are numbered from top to down, from left to right, and beginning from root.)
2.2) Prove that hypercube has optimal fault tolerance.
Send your solutions to TA via Email before Nov.17, 2012.
Sh
an
gh
ai Jia
o T
on
g U
niv
ersity
20
11
54
Reading materials 1) Y. Saad and M. H. Schultz, Topological Properties of Hypbercube, IEEE
Transactions on Computers , Vol. 24, No. 5
2) L.N. Bhuyan and D.P. Agrawal, Generalized Hypercube and Hyperbus Structures for a Computer Network, IEEE Transactions on Computers, Vol. 33, No. 5, 1984
3) C. E. Lerserson, Fat-tree: Universal network for Hardware-Efficient Supercomputing, IEEE Transactions on Computers, pp. 892-901, 1994