Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
Lecture 6:Deadlocks - IITushar Krishna
Assistant ProfessorSchool of Electrical and Computer EngineeringGeorgia Institute of Technology
ECE 8823 A / CS 8803 - ICNInterconnection NetworksSpring 2017http://tusharkrishna.ece.gatech.edu/teaching/icn_s17/
Recap: Resource Dependence
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
2
A B
u v w x
C D
Hold (i..e, wait for release)Wait to acquire
Agents
Resources
Resource A is dependent on resource B if it is possible for A to be held-byan agent X and it is also possible for X to wait-for B
u v w x
Resource Dependence Graph
To avoid deadlocks, resource dependence graph should be acyclic
Recap: Network Deadlock§ Agent = packet or flit, Resource = buffer
§ Packet A holds buffer u (in 1) and wants buffer v (in 2)§ Packet B holds buffer v (in 2) and wants buffer w (in 3)§ Packet C holds buffer w (in 3) and wants buffer x (in 0)§ Packet D holds buffer x (in 0) and wants buffer u (in 1)
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
3
0 1
23
A
BC
Du
vw
x
We will use the following convention: channel == buffer at end of channel
Channel 01 blocked => Buffer at 1 not free
Channel 01 depends on Channel 12 => Buffer at 1 depends on Buffer at 2
Deadlock Avoidance¡Eliminate cycles in Resource Dependency Graph¡ Resource Ordering¡ Enforce a partial/total order on the resources, and insist that
an agent acquire the resources in ascending order¡ Deadlock avoided since a cycle must contain at least one
agent holding a higher numbered resource waiting for a lower-numbered resource which is not allowed by the ordering allocation
¡ Implementation¡ Restrict certain routes so that a higher numbered resource
cannot wait for a lower numbered resource¡ Partition the buffers at each node such that they belong to
different resource classes. A packet only any route can only acquire buffers in ascending order of resource class
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
4
Example: Channel Dependency Graph for Mesh
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
5
F
A B
E D
C
Acyclic CDG
Disable certain edges
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
Cyclic CDG
FEED
DC CBBE EF FA
AFBA
EBDECDBCAB
This is the West-first turn model!
Acyclic CDG for a Ring
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
6
F
A B
E D
C
FE
ED DC
CB
BAAF
CD
DE EF
FA
ABBC
Option 3
Problem?
Route from E to C disabled(E to D) and (D to C) allowed
Route from B to D disabled
Acceptable CDGE to C no longer minimal
CDG
Acyclic CDG for a Large Ring
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
7
H
C D
G F
E
I
B
J
A
G, H, I have to take non-minimal paths to reach E!
D, C, B have to take non-minimal paths to reach F
Problem?
Suppose two channels
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
8
F
A B
E D
C
FE0
ED0 DC0
CB0
BA0AF0
CD0
DE0 EF0
FA0
AB0BC0
ED1 DC1
FE1
AF1 BA1
CB1
DE1 EF1
CD1
AF1 BA1
FA1
Dateline
DatelineDateline
Need not be physical channels
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
9
FE0
ED0 DC0
CB0
BA0AF0
CD0
DE0 EF0
FA0
AB0BC0
ED1 DC1
FE1
AF1 BA1
CB1
DE1 EF1
CD1
AF1 BA1
FA1
Dateline
DatelineDateline
Need at least 2 classes of buffers - called“Virtual Channels”Start in VC in Class0After Dateline, jump to VC in Class1
F
A B
E D
C
Deadlock Avoidance¡Eliminate cycles in Resource Dependency Graph¡ Resource Ordering¡ Enforce a partial/total order on the resources, and insist that
an agent acquire the resources in ascending order¡ Deadlock avoided since a cycle must contain at least one
agent holding a higher numbered resource waiting for a lower-numbered resource which is not allowed by the ordering allocation
¡ Implementation¡ Restrict certain routes so that a higher numbered resource
cannot wait for a lower numbered resource¡ Partition the buffers at each node such that they belong to
different resource classes. A packet only any route can only acquire buffers in ascending order of resource class
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
10
Using VCs for Deadlock Avoidance¡Ring¡Use VC from class 0 before dateline¡Use VC from class 1 after dateline
¡Fully-Oblivious (e.g., O1turn)¡Use VC 0 for XY, VC 1 for YX
¡Fully-Adaptive Routing (no turns restricted)¡Use VC from class 0 before turning¡Use VC from class 1 after turning
¡Valiant’s Routing Algorithm¡DOR over VC in class 0 from source till intermediate¡DOR over VC in class 1 from intermediate to destination
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
11
VC utilization
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
12
F
A B
E D
C
Problem?Ring with Dateline
Mesh with O1Turn
VC from Class 1 never used!
Packet on Ring never crosses datelinePacket on Mesh does not make any turns
Class 0
Class 1
Solution: Overlapping Resource Classes
Class 0Class 1
As long at least one buffer per class, can guarantee deadlock freedom
Deadlock Avoidance¡So far, we said deadlock is avoided if cycles
eliminated in Channel Dependence Graph¡ Remove cycles via turn restriction¡ Convert cyclic CDG into a spiral using VCs¡ Called extended CDG
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
13
FE0
ED0 DC0
CB0
BA0AF0
CD0
DE0 EF0
FA0
AB0BC0
ED1 DC1
FE1
AF1 BA1
CB1
DE1 EF1
CD1
AF1 BA1
FA1
Deadlock Avoidance¡So far, we said deadlock is avoided if cycles
eliminated in Channel Dependence Graph¡ Remove cycles via turn restriction¡ Convert cyclic CDG into a spiral using VCs¡ Called extended CDG
¡However, it is possible for a (extended) CDG to have cycles and still be deadlock-free (Duato*, 1993)¡ As long as the cycle connects to some sub-graph within
the (extended) CDG that is acyclic¡ Known as the escape path or escape VC
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
14
*José Duato. A new theory of deadlock-free adaptive routing in wormhole networks.IEEE Transactions on Parallel and Distributed Systems, 4(12):1320–1331, December 1993.
CDG for Escape VCs
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
15
FE0
ED0 DC0
CB0
BA0AF0
ED1 DC1
FE1
AF1 BA1
CB1
F
A B
E D
C
Escape VC
Acyclic Escape VC
Why escape VCs work¡ Intuitively, at least one packet in the cycle has an
option to take an acyclic route¡ Packets should not wait on any specific channel¡ If allocation is fair, escape VCs guaranteed to show up
eventually
¡Use of escape channels by a message is not unidirectional¡ If a message enters the escape network it can move
back to the adaptive network, and vice versa, if minimal* routes¡ *for non-minimal routes, message has to continue on
escape VC once it gets in, without going back to the adaptive VCs
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
16
Example¡Consider a 2D Mesh with 8 VCs and minimal
routing¡VC 1-7 can use any arbitrary minimal routing¡ Cyclic CDG
¡VC 0 (escape VC) is restricted to DOR (provides escape path)¡ Acyclic CDG
¡As long as a packet can allocate all VCs fairly, there will always be an escape path available in case the network deadlocks
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
17
Example
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
18
Escape VC
West-first
Regular VCs
Deadlock-free escape path
Rules for getting in/out of escape VCs¡The escape VC should
always makes forward progress!¡A flit that is going NW or
SW should never enter a router from the S or N port in escape VC, else SàW or NàW turn is inevitable¡How to guarantee this?¡ When selecting VC at
previous router¡ Lab 3!
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
19
VC 1 (escape VC)West-first
VC 0 (regular VC)
X?
Ejection not shown
Deadlock Avoidance¡Routing¡ Turn Model in a Mesh¡ Acyclic CDG
¡Buffer Assignment¡ Acquire VCs in ascending order¡ Acyclic extended CDG
¡ Escape VCs¡ Cyclic CDG + Acyclic sub-graph
¡Flow Control¡ Control traffic flow such that deadlock does not occur¡ Cyclic CDG + injection control to avoid dependence cycle
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
20
Bubble Flow Control
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
21
F
A B
E D
C
Ring Traversal Rule: traverse if one bubble free
V. Puente et al. The adaptive bubble router. Journal of Parallel and Distributed Computing, 2001.
Bubble Flow Control
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
22
Ring Traversal Rule: traverse if one bubble free
F
A B
E D
CV. Puente et al. The adaptive bubble router. Journal of Parallel and Distributed Computing, 2001.
Should it inject?BFC Injection Rule:only inject if 2 bubbles free.
Bubble Flow Control
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
23
Ring Traversal Rule: traverse if one bubble free
F
A B
E D
C
BFC Injection Rule:only inject if 2 bubbles free.
V. Puente et al. The adaptive bubble router. Journal of Parallel and Distributed Computing, 2001.
Problem?
Bubble Flow Control
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
24
Ring Traversal Rule: traverse if one bubble free
BFC Injection Rule:only inject if 2 bubbles free.
V. Puente et al. The adaptive bubble router. Journal of Parallel and Distributed Computing, 2001.
F
A B
E D
C
Not allowed to inject!Even though no deadlock
Critical Bubble Flow Control
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
25
Ring Traversal Rule: traverse if one bubble free
CBFC Injection Rule:only inject if not critical bubble.
F
A B
E D
C
Allowed to inject!
L. Chen et al., “Critical Bubble Scheme: An Efficient Implementationof Globally Aware Network Flow Control,” IPDPS 2011
Critical Bubble
Critical Bubble Flow Control
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
26
Ring Traversal Rule: traverse if one bubble free
CBFC Injection Rule:only inject if not critical bubble.
F
A B
E D
CL. Chen et al., “Critical Bubble Scheme: An Efficient Implementationof Globally Aware Network Flow Control,” IPDPS 2011
Critical Bubble
How does critical bubble move?If flit moves into critical bubble, its own buffer becomes new critical bubble
Dealing with Deadlocks¡Avoidance¡Guarantee that the network will never deadlock¡Almost all modern networks use deadlock avoidance
¡Recovery¡Detect deadlock and correct
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
27
Deadlock Recovery¡When might this make sense?¡ Cannot accept performance degradation required to avoid
deadlocks¡ Average-case performance more important
¡ Two phases¡ Detection: ¡ E.g., timeouts attached with each resource¡ Can lead to false positives
¡ Recovery:¡ Regressive – remove packets/connections that are deadlocked¡ E.g., drop packets after timeout
¡ Progressive – recover without removing packets/connections¡ E.g. shared escape buffer to drain deadlocked packets
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
28
K. V. Anjan and T. M. Pinkiston, “An efficient fully adaptive deadlock recovery scheme: DISHA”, ISCA 1995
Another kind of deadlock: Protocol Deadlock
January 30, 2017ICN | Spring 2017 | L06: Deadlocks-II © Tushar Krishna, School of ECE, Georgia Tech
29
Cache 0
Directoryreqreqreqreq
req
req
req
resp
resp
req
req
out
out
in
in
req
Cache / Directory can process a request only if there is space in its output queue to send a response
Need separate Virtual Channels* for requests and responses(called Virtual Networks)
Responses should always be drained (“consumption assumption”)
Deadlock, even though network is deadlock-free