View
214
Download
0
Category
Preview:
Citation preview
1
The Case for Byzantine Fault Detection
2
Challenge: Byzantine faults Distributed systems are subject to
a variety of failures and attacks Hacker break-in Freeloading Censorship Data corruption Software/hardware failure
Byzantine failure model: Faulty nodes may exhibit arbitrary behavior
Dependable systems must be protected against Byzantine faults
3
Existing approach: Fault tolerance
Byzantine fault tolerance (BFT) can mask a limited number of Byzantine faults
Example: Castro and Liskov [OSDI'99]
Client
Serverreplicas
4
Alternative approach: Fault detection Nodes monitor each other for faulty behavior When a fault occurs, the correct nodes
identify the faulty node(s) distribute evidence of the fault
Nodes can isolate the faulty node + initiate recovery
Byzantine Fault Detection
5
Byzantine Fault Detection
Alternative approach: Fault detection Nodes monitor each other for faulty behavior When a fault occurs, the correct nodes
identify the faulty node(s) distribute evidence of the fault
Nodes can isolate the faulty node + initiate recovery
D C
B
A
ESet X=5
D C
A
E
D C
B
A
EOK
X=?X=7 E: X=5
7! B
6
Level3
Best approach depends on the application
Best-effort service Goal: Find faulty components Wide-area delays, limited
bandwidth, many nodes
Air traffic control Inter-domain routing
Failures may be fatal! Goal: Mask fault
symptoms Delays negligible,
bandwidth plentiful, few nodes
Machine roomAT&T
Sprint
Typical application for Fault DetectionTypical application for Fault Tolerance
7
Detection can provide accountability In an accountable system:
Actions are undeniable State is tamper-evident Correctness can be certified
Good nodes can provide evidence that they are good
Bad nodes cannot hide evidence of misbehavior
Proven concept in society Banking, administration ...
Desirable for distributed systems [Yumerefendi05] Example: Building trust in federated systems
8
What about performance?
If up to f nodes can be faulty, we need f+1 replicas to guarantee detection (fault tolerance: 3f+1)
More throughput using the same resources Works even when >33% of the nodes can become
faulty
Detection can defer overhead to periods of low load
System can deliver high peak throughput
Detection does not require consensus Potentially less expensive than BFT
9
Outline
Introduction BFD abstraction PeerReview algorithm Conclusion
10
How is BFD used?
Each correct node has state machine + detector Detector can inspect all messages at its local node When detector observes a fault on another node,
it informs its local application, and it provides evidence of the fault to other detectors
?
Application
State machine Detector
Network
Node Xis
faulty!
No assumptionsabout faulty nodes
11
Only observable faults can be detected
Two classes of observable faults: Detectable faultiness: Node breaks the protocol Detectable ignorance: Node refuses to respond
As long as the faulty node continues to follow the protocol, BFD cannot detect this!
Set X=5
OKGet X
5
A B C
Correct
Set X=5
OKGet X
A B CSet X=5
OKGet X
7
A B C
Detectably ignorantDetectably faulty
12
BFD can give strong guarantees Three types of detector output
Trusted, suspected, exposed
Strong completeness "No false negatives"
Strong accuracy "No false positives"
Precise definitions are in the paper
Trusted
Suspected Exposed
13
Outline
Introduction BFD abstraction PeerReview algorithm Conclusion
14
Assumptions
1. Protocol can be modeled as a deterministic state machine
2. Each node has a strong identity, as well as a public/private keypair for signing messages
3. The faulty nodes cannot prevent two correct nodes from communicating break the cryptographic keys
15
Secure logging
All messages are signed and acknowledged Each node keeps a log of all local inputs and outputs Nodes must commit to the contents of their log
Log is tamper-evident [Maniatis02]
Rcv(A, "Set X=5")Send(A, "Okay")Rcv(C, "Get X")Send(C, "5")
Snd(B, "Set X=5")Rcv(B, "Okay")
Snd(B, "Get X")Rcv(B, "5")
B's log
A
B
C
16
Detecting ignorance
If a node refuses to acknowledge a message Send message as evidence to other nodes Correct nodes will challenge the ignorant node to prove
that its log contains a 'Rcv' entry for that message A correct node can always respond
Rcv(A, "Set X=5")Send(A, "Okay")Recv(C, "Get X")
A
B
C
17
Detecting faultiness
Nodes can audit each other's log at any time Auditors replay input in the log, compare output If a divergence is detected
Send log as evidence to other nodes Other nodes can repeat the same procedure to check
whether the node is really faulty (no he-said-she-said!)
Rcv(A, "Set X=5")Send(A, "Okay")Rcv(B, "Get X")Send(B, "7")
A
B
C
B'
Rcv(A, "Set X=5")Send(A, "Okay")Rcv(B, "Get X")Send(B, "5")
State machine B is expected to run
Rcv(A, "Set X=5")Send(A, "Okay")Rcv(B, "Get X")Send(B, "7")
Snap-shots
18
Summary
New approach: Byzantine Fault Detection Alternative to fault tolerance Provides accountability
Fault Detection can give strong guarantees Eventual strong accuracy and completeness
Early results indicate Fault Detection is practical Example: PeerReview algorithm
Recommended