Upload
nikita
View
25
Download
0
Embed Size (px)
DESCRIPTION
Fault-containment in Weakly Stabilizing Systems. Anurag Dasgupta Sukumar Ghosh Xin Xiao University of Iowa. Preview. - PowerPoint PPT Presentation
Citation preview
Fault-containment in Weakly Stabilizing Systems
Anurag Dasgupta Sukumar GhoshXin Xiao
University of Iowa
Preview
• Weak stabilization (Gouda 2001) guarantees reachability and closure of the legal configuration. Once “stable”, if there is a minor perturbation, apparently no recovery guarantee exists, let alone “efficient recovery”.
• We take a weakly stabilizing leader election algorithm, and add fault-containment to it.
Our contributions
• An exercise in adding fault-containment to a weakly stabilizing leader election algorithm on a line topology. Processes are anonymous.
• Containment time = O(1) from all single failures
• Lim m∞ (contamination number) is O(1) (precisely 4), where m is a tuning parameter
(Contamination number = max. no. of non-faulty processes that change their states during recovery)
The big picture
Model and NotationsConsider n processes in a line topologyN(i) = neighbors of process iVariable P(i) = {N(i) U ⊥} (parent of i)Macro C(i) = {q ∈ N(i): P(q) = i} (children of i)Predicate Leader(i) ≡ (P(i)=⊥)
Legal configuration:• For exactly one process i: P(i) = ⊥ j ≠ i: P(j) = k P(k) ≠ j
Node i
P(i)
C(i)
Leader
Model and Notations
Shared memory model and central schedulerWeak fairness of the schedulerGuarded action by a process: g AComputation is a sequence of (global) statesand state transitions
Node i
P(i)
C(i)
Leader
Stabilization
A stable (or legal) configuration satisfies a predicateLC defined in terms of the primary variables p that are observable by the application. However, fault-containment often needs the use of secondary variables (a.k.a auxiliary or state variables) s. Thus,
Local state of process i = (pi, si)Global state of the system = (p, s), where p = the set of all pi, and s = the set of all si
(p, s) LC p LCp and s LCs
Definitions
Containment time is the maximum time needed to establish LCp from a 1-faulty configuration
Containment in space means the primary variables of O(1) processes changing their state during recovery from any 1-faulty configuration
Fault-gap is the time to reach LC (both LCp and LCS) from any 1-faulty configuration
LCp restored LCs restored
Fault gap
Weakly stabilizing leader election
We start from the weakly stabilizing leader electionalgorithm by Devismes, Tixeuil,Yamashita [ICDCS 2007], and then modify it to add fault-containment. Here is the DTY algorithm for an array of processes.
DTY algorithm: Program for any process in the array
Guarded actions:
R1 :: not leader ∧ N(i) = C(i)→ be a leader
R2 :: not leader∧ N(i) \ {C(i) U P(i)} ≠ → switch parent
R3 :: leader∧ N(i) ≠ C(i) → parent := k : k C(i)
Effect of a single failure
With a randomized scheduler, the weakly stabilizing system
will recover to a legal configuration with probability 1.
However, If a single failure occurs, the recovery time can be
as large as n (Using situations similar to Gambler’s ruin). For
fault-containment, we need something better.
We bias a randomized scheduler to achieve our goal. The
technique is borrowed [Dasgupta, Ghosh, Xiao: SSS 2007].
Here we show that the technique is indeed powerful enough
to solve a larger class of problems.
Biasing a random scheduler
For fault-containment, each process i uses a secondary variable
x(i). A node i updates its primary variable P(i). when thefollowing conditions hold:
• The guard involving the primary variables is true • The randomized scheduler chooses i• x(i) ≥ x(k), where k N(i)
Biasing a random scheduler
ij k
x(i)=10 x(k)=7x(j)=8
ij k
x(i)=13 x(k)=7x(j)=8
(Let m = 5)
ij k
x(i)=10 x(k)=7x(j)=8
ij k
x(i)=10 x(k)=8x(j)=8
After the action, x(i) is incremented as x(i) := max q ∈N(i) x(q) + m, m ∈ Z+
(call it update x(i), here m is a tuning parameter). When x(i) < x(k) but conditions 1-2 hold, the primary variable P(i) remains unchanged -- onlyx(i) is incremented by 1
UPDATE x(i) INCREMENT x(i)
* * * *
The Algorithm
Algorithm 1 (containment) : program for process i
Guarded actions:
R1 :: (P(i) ≠ ⊥) ∧ (N(i) = C(i)) → P(i) := ⊥
R2 :: (P(i) = ⊥) ∧ (∃k ∈ N(i) \ C(i)) → P(i) := k
R3a :: (P(i) = j) ∧ (∃k ∈ N(i) : P(k) ≠ i or ⊥)∧ x(i) ≥ x(k) → P(i) := k; update x(i)
R3b :: (P(i) = j) ∧ (∃k ∈ N(i) : P(k) ≠ i or⊥) ∧ x(i) < x(k) → increment x(i)
R4a :: (P(i) = j) ∧ (∃k ∈ N(i) : P(k) = ⊥) ∧ x(i) ≥ x(k) → P(i) := k
R4b :: (P(i) = j) ∧ (∃k ∈ N(i) : P(k) = ⊥) ∧ x(i) < x(k) → increment x(i)
R5 :: (P(i) = j) ∧ (P(j) = ⊥) ∧ (∃k ∈ N(i) : P (k) ≠ i or⊥} → P(i) := k
Analysis of containment
Consider six cases
1. Fault at the leader
2. Fault at distance-1 from the leader
3. Fault at distance-2 from the leader
4. Fault at distance-3 from the leader
5. Fault at distance-4 from the leader
6. Fault at distance-5 or greater from the leader
Case 1: fault at leader node
0 1 72 6543
0 1 72 6543
0 1 72 653 4
R1 applied by node 5
R1 applied by node 4: node 4 is the new leader
**
R1 :: (P(i) ≠ ⊥) ∧ (N(i) = C(i)) → P(i) := ⊥
Case 2: fault at distance-1 from the leader node
0 1 72 6543
0 1 72 6543
R1: node 3
0 1 72 6543
R2: node 5
* *
**
*
R2 :: (P(i) = ⊥) ∧ (∃k ∈ N(i) \ C(i)) → P(i) := k
Case 5: fault at distance-4 from the leader node
0 1 72 6543
* *0 1 72 6543
**
R4a(2): x(2)>x(1)
0 1 72 6543 R5 (4)
* *
*R2(5)
*
R3a(3): x(3)>x(2)
0 1 72 6543
0 1 72 6543 stable
Non-faulty processes up to distance 4 from the faulty node being affected
R4a :: (P(i) = j) ∧ (∃k ∈ N(i) : P(k) = ⊥) ∧ x(i) ≥ x(k) → P(i) := k
Case 6: fault at distance ≥ 5 from the leader node
0 1 72 6543
* *0 1 72 6543
**
R4a(2): x(2)>x(1)
0 1 72 6543 R3a (3); R5 (2)
* *
*R2 (1)
*
R3a(3): x(3)>x(2), x(4)
0 1 72 6543
With a high m, it is difficult for 4 to change its parent, but 3 can easily do it
Recovery complete0 1 72 6543
Current leader
Fault-containment in space
Theorem 1. As m∞ , the effect of a single failure is restricted
within distance-4 from the faulty process i.e., algorithm is spatially
fault-containing.
Proof idea. Uses the exhaustive case-by-case analysis. The worst
case occurs when a node at distance-4 from the leader node fails
as shown earlier.
Fault-containment in time
Theorem 2. The expected number of steps needed to contain
a single fault is independent of n. Hence algorithm containment is
fault-containing in time.
Proof idea. Case by case analysis. When a node beyond distance-4
from the leader fails, its impact on the time complexity remains
unchanged.
Fault-containment in time
0 1 72 6543
* *
Recovery completed in a single move regardless of whether node 3 or 4 executes a move.
Case 1: leader fails
Case 2: A node i at distance -1 from the leader fails.
• P(i) becomes :⊥ recovery completed in one step
• P(i) switches to a new parent: recovery time = 2 +∑∞n=1 n/2n = 4
Fault-containment in time
Summary of expected containment times
Fault at leader - 1
Fault at dist-1 1 4
Fault at dist-2 2 151/108
Fault at dist-3 131/54 115/36
Fault at dist-4 10/9 29/27
Fault at dist ≥ 4 33/32 115/36
P(i) ⊥ P(i) switches
Thus, the expected containment time is O(1)
Another proof of convergence
Theorem 3. The proposed algorithm recovers from all single faultsto a legal configuration in O(1) time.
Proof (Using martingale convergence theorem)
A martingale is a sequence of random variables X1, X2, X3, … s.t. ∀n•E(|Xn|) < ∞, and•E(Xn+1|X1 … Xn) = Xn (for super-martingale use ≤ for =, and
for sub-martingale, use ≥ for =)
We use the following corollary of Martingale convergence theorem:
Corollary. If Xn ≥ 0 is a super-martingale then as n → ∞, Xn converges to X with probability 1, and E(X) ≤ E(X0).
Proof of convergence (continued)
Let Xi be the number of processes with enabled guards in step i.After 0 or 1 failure, X can be 0, 2, or 3 (exhaustive enumeration).
When Xi = 0, Xi+1 = 0 (already stable) When Xi = 2, E(Xi+1)= 1/2 x 1 + 1/2 x 2 = 1 ≤ 2When Xi = 3, E(Xi+1)= 1/3 x 0 + 1/3 x 2 + 1/3 x 4 = 2 ≤ 3
Thus X1, X2, X3, … is a super-martingale. Using the Corollary, as n → ∞, E(Xn) ≤ E(X0). Since X is non-negative by definition, Xn
converges to 0 with probability 1, and the system stabilizes.
Proof idea of weak stabilization
DTY algorithm Our algorithm
R1
R2
R3
R1
R2
R3
R4
R5Weakly stabilizing
Executes the same action (P(i) :=k) as in DTY, but the guards are biased differently
Weakly stabilizing
Stabilization from multiple failures
Theorem 3. When m → ∞, the expected recovery time from multiple
failures is O(1) if the faults occur at distance 9 or more apart.
Proof sketch. Since the contamination number is 4, no non-faultyprocess is influenced by both failures.
4 4
FaultFault
Conclusion
1. With increasing m, the containment in space is tighter, but
stabilization from arbitrary initial configurations slows down.
2. LCs = true, so the systems is ready to deal with the next single
failure as soon as LCp holds. This reduces the fault-gap and
increases system availability.
• The unbounded secondary variable x can be bounded using the
technique discussed in [Dasgupta, Ghosh, Xiao SSS 2007] paper.
• It is possible to extend this algorithm to a tree topology (but
we did not do it here)