Ch11Distributed Agreement
Outline
Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus
Outline
Distributed Agreement Adversaries
Byzantine Agreement
Impossibility of Consensus
Randomized Distributed Agreement
Exponential Time Shared Memory Consensus
Distributed Agreement Adversaries When proving correctness or analyzing an algorithm, it is convenient to assume that: the inputs, the failure times, failure behaviors
any system variables are under the control of an adversary who, intuitively, makes as much difficulty for the algorithm as possible
Worst case analysis: worst case choices of the adversary
Outline
Distributed Agreement Adversaries
Byzantine Agreement
Impossibility of Consensus
Randomized Distributed Agreement
Exponential Time Shared Memory Consensus
Distributed Agreement The Agreement or the Consensus Problem: Assume P = {p1,…,pM} is the set of all the processors in the system Some processors in P are faulty Let F be the set of all faulty processors in P Every processor p in P has a value p.Val
The requirement: devise a distributed algorithm that lets each processor p computes a value p.A such that when the execution of this distributed algorithm terminates, the following two conditions hold: 1. (agreement value) For every pair of processors p and q, q.A = p.A
2. The agreement value is a function of the initial values {p.Val} of non-faulty processors
Outline
Why Distributed Agreement is an interesting problem? Processor p is the leader
Processor p has the right to enter the critical section
Processor p has failed
Distributed Agreement Byzantine Agreement: Assumptions A failed processor can send arbitrary messages
A non-failed processor always responds to a message within T seconds
When a processor receives a message, it can reliably determine the sender of that message
Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem:
“ENEMY”the Sultan’s army
Byz_A1Byz_A3
Byz_A2
G2
G1G3
Some byzantine generals are “corrupted”
“Non-corrupted” generals knew that they will be victorious only if they attack simultaneously
Loyal generals must find a consensusto attack or to retreat
Byz_A4
G4
Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem (basic idea) Each general has to make a decision based on the opinions it gets from the other generals
All loyal generals must make the same decision
If all loyal generals get the same set of opinions for making the decision, then all loyal generals can achieve a consensus using the same procedure to decide How can we ensure that all loyal generals get the same set of opinions ?
Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem (basic idea) To ensure that each loyal general gets the same set of values, it is sufficient that each loyal general uses the same value Vj for every other general Gj in order to decide
The Byzantine Generals Problem is then reduced to agreement by generals on the value sent by a particular general: a commandinggeneral
Formally, we must have: 1. If the sender ps is loyal and sends the value Vs, the loyal generals will decide that the value sent is Vs
2. If the sender ps is treacherous, the loyal general will agree on the same value
This problem is known as the interactive consistency problem
Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem (continued) Assuming that each general can reliably broadcast its opinion, the loyal generals can reach an agreement!
How and under which conditions?
Distributed Agreement The Byzantine Generals Problem (continued) Question 1 assuming that there is a reliable protocol for broadcast, is it possible to reach an agreement with three generals with one disloyal ?The answer is NO!
Why?
L1
C
L2 L1
C
L2
disloyal
disloyal
retreatattackretreat
attack
attackattackretreat
attack
Distributed Agreement The Byzantine Generals Problem (continued) Question 2 assuming that there is a reliable protocol for broadcast, is it possible to reach an agreement with four generals with one disloyal ?
The answer is Yes!
Why? Two cases are in order: The commanding general is disloyal The commanding general is loyal
Distributed Agreement The Byzantine Generals Problem (continued) Justification of the answer to Question 2
C L1 L2 L3
attack
retreat
By the end of the second round,
L1 has 2 attack and 1 retreat
L2 has 2 attack and 1 retreat
L3 has 2 attack and 1 retreat
Disloyal commandingGeneral:C
Each Lieutenant obeysthe majority
Round 1
Round 2
Distributed Agreement The Byzantine Generals Problem (continued) Justification of the answer to Question 2
C L1 L2 L3
attack
retreat
By the end of the second round,
L1 has 2 attack and 1 retreat
L2 has 2 attack and 1 retreat
L3 has 3 attack and 1 retreat
Disloyal Lieutenant: L3
Each Lieutenant obeysthe majority:Each Loyal General decides“attack”
Round 1
Round 2
Distributed Agreement The Byzantine Generals Problem (continued) Theorem: Assuming a synchronous system with M processors ,of which up to t can be faulty, the loyal generals can reach a consensus only if M 3t+1
The algorithm to solve the Byzantine Generals Problem is parameterized by k the maximum number of disloyal generals
This algorithm is BG(k)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
The algorithm works in rounds of messages exchange.
C
L1 L2 LM-1
L(C)=the set of Lieutenants for C; size of L(C) = M-1
Round 1
If no message is sent to a Lieutenantthat Lieutenant takes “retreat” as the default value
L(C)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
p1
L1 L2 LM-2
L(p1:C)=the set of Lieutenants for p1
with respect to C’s opinion; size of L(p1:C) = M-2
Round 2 Every processor p1 in L(C) acts as the commanding General
p1 sends M-2 messagesp1 receives M-2 messages
p1.v(2) := majority(V) where V = {p1.v(1)} {p1.Rq(2) : q in L(p1:C)}
L(p1)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
p2
L1 L2 LM-3
L(p2:p1)=the set of Lieutenants for p2
with respect to p1’s opinion; size of L(p2:p1) = M-3
Round 3 Every processor p2 in L(p1), for each p1 in L(C), acts as the commanding General
p2 sends M-3 messagesp2 receives M-3 messages
p2.vr(3,p1) := majority(V) where V = {p2.vr(2,p1)} {p2.Rq(3,p1) : q in L(p2:p1)}
L(p2)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
p3
L1 L2 LM-3
L(p3:p2)=the set of Lieutenants for p3
with respect to p2’s opinion on ...; size of L(p3) = M-4
Round 4 Every processor p3 in L(p2), for each p2 in L(p1), p1 in L(C), acts as the commanding General
p3 sends M-4 messagesp3 receives M-4 messages
p3.vr(4,p2) := majority(V) where V = {p3.vr(3,p2)} {p2.Rq(4,p2) : q in L(p3:p2)}L(p3)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
pi-1
L1 L2 LM-i
L(pi-1:pi-2)=the set of Lieutenants for pi-1
with respect to pi-2’s opinion on ...; size of L(pi-1:pi-2) = M-i
Round iEvery processor pi-1 in L(pi-2), for each pi-2 in L(pi-3), …, p2 in L(p1), p1 in L(C), acts as the commanding General
pi-1 sends M-i messagespi-1 receives M-i messages
pi-1.vr(i,pi-2) := majority(V) where V = {pi-1.vr(i-1,pi-2)} {pi-1.Rq(i,pi-2) : q in L(pi-1:pi-2)}
L(pi-1)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
pk
L1 L2 LM-k-1
L(pk:pk-1)=the set of Lieutenants for pk
with respect to pk-1’s opinion on ...; size of L(pk) = M-k-1
Round k+1, BG(0) Every processor pk in L(pk-1), for each pk-1 in L(pk-2), …, p2 in L(p1), p1 in L(C), acts as the commanding General
Pk sends M-k-1 messagespk receives M-k-1 messages
pk.vr(k+1,pk-1) := majority(V) where V = {pk.vr(k,pk-1)} {pk.Rq(k+1,pk-1) : q in L(pk)}
L(pk)For each pk-1, pk decides pk.vr(k+1,pk-1)
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea:
pk
L1 L2 LM-k-1
Round k+1
L(pk) pk decides pk.vr(k+1,pk-1)
Processor pk decides using (1+M-k-1)opinions
So, if M=3k+1, then we have thatpk decides using 2k+1 opinions
Since at most k processors can befaulty, it follows that all non faulty processors make the same decision
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k):
BG(0):1. The commanding sends its value to all the other n-1 processors2. Each processor uses the value it receives from the commanding or uses the default value
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k):
BG(k), k>0:1. The commanding sends its value to all the other n-1 processors
2. Let vp be the value the processor p receives from the commanding general, or the default value if no value is received. Processor p acts as the commanding in BG(k-1) to send the value vp to each of the other M-2 processors.
3. For each processor p, let vq be the value received from processor q (q p). Processor p uses the majority({v} {vq: q in L(C)}) where v is the value processor p received from the commanding general
Distributed Agreement The Byzantine Generals Problem (continued) The BG(k): number of messages sent
Following the presentation gave above, one can see that the number of messages sent is proportional to (M-1)(M-2)(M-3)…(M-k-1)Since k can be (M-1)/3, it follows thatthe number of messages is O(Mk)
Outline
Distributed Agreement Adversaries
Byzantine Agreement
Impossibility of Consensus
Randomized Distributed Agreement
Exponential Time Shared Memory Consensus
Distributed Agreement The Byzantine Generals Problem (continued) Impossibility result
If the system is asynchronous (no bound on the relative speeds of processors or the communication delays), then it can be shown (Fisher, Lynch, Paterson 1985) that agreement is impossible if even one processor can fail, and even if the failure isa crash failure
Outline
Distributed Agreement Adversaries
Byzantine Agreement
Impossibility of Consensus
Randomized Distributed Agreement
Exponential Time Shared Memory Consensus
Distributed Agreement The Randomized Distributed Agreement
Randomization: processors can flip coinAssumptions: The system consists of N processors, of which up to t can be faulty
Processors communicate by using shared registers
The shared registers are non-faulty
The accesses to the shared register are sequentially consistent
Atomic reads and writes of the contents of the registers
The system is asynchronous
Outline
Distributed Agreement Adversaries
Byzantine Agreement
Impossibility of Consensus
Randomized Distributed Agreement
Exponential Time Shared Memory Consensus
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: The Naïve algorithm: Assume : the system is synchronous each processor p has a initial value Vp to prefer The idea (algorithm for processor p): while I have not yet decided do 1. Read the set {Vi} of values of every other processors 2. If for all i, Vi = Vp then decide Vp else Vp := coin_flip() end
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: Transformation of the Naïve synchronous algorithm into an asynchronous algorithm Idea 1: “simulate” the synchronous algorithm: add a round variable at each processor
The naïve algorithm becomes
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: Idea 1 continue (algorithm for processor p): while I have not yet decided do 1. Read the set {Vi} of values of every other processors 2. If for all i, Vi = Vp and p.round = I.round then decide Vp else Vp := coin_flip(); p.round := pround+1 endProblem : some processors can fail; a fail processor may not increment its round variable when it executed
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: Idea 2: At any moment, the set of processors can be regarded as consisting of FP: the set of the largest round value; LP: the other processors
If ( p,q in FP: Vp = Vq ) and eventually ( s in LP: Vs = Vq , q in FP ) then one can decides on Vq, q in FP
How can we achieve the eventually part ?
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: Idea 2(continued):How can we achieve the eventually part ?Intuitively, the idea is to make the slower processors prefer to accept the value of faster processors.
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: The algorithm :Variable usedV[1..M] shared array of records, one per each processor V[i].value : the preferred decision of processor i; V[i].round : execution round of processor i;
Local_V[1..M] local copy of V[1..M]
Leaders : the processors that have the largest round values in round Local_V
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: The algorithm :Functions usedleader_set(Local_V) : returns the set of leaders
Flip() : randomly returns either 0 or 1.
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: The algorithm :InitiallyV[i].value := NIL;V[i].round :=0;
/* not necessary */Local_V[i].value := NIL;Local_V[i].round := 0;leaders := empty
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus:The algorithm : SM_Consensus(self, preference)(V[self].value, V[self].round) := (preference,1)while I have not made a decision do read V into Local_V; leaders := leader_set(Local_V); if (self leaders) and ( i : Local_V[i].value Local_V[self] : Local_V[i].round < Local_V[self].round -1) then decide(V[self].value) elseif (i,j in leaders: Local_V[i].value = Local_V[j].value) then (V[self].value, V[self].round) := (V[i].value, V[i].round) for a i in leaders elseif V[self].value NIL then (V[self].value, V[self].round):=(NIL, V[self].round) else (V[self].value, V[self].round) := (Flip(), round+1)
Distributed Agreement The Randomized Distributed Agreement
The Exponential Time Shared Memory Consensus: The probability that all leaders choose the same value: O(2-N)
The expected number of rounds: O(2N)
Recommended