Embed Size (px)
Citation preview
Separating Agreement from Executionfor Byzantine Fault-Tolerant Services
Rethinking Replicated State Machines
Jian Yin, Jean-Philippe Martin, Arun Venkataramani,Lorenzo Alvisi and Mike Dahlin
[email protected], {jpmartin,arun,lorenzo,dahlin}@cs.utexas.edu
Laboratory for Advanced Systems Research (LASR)The University of Texas at Austin
JPM, The University of Texas at Austin p.1/17
Problem: Tolerating Byzantine Faults
wrong replyno reply
unauthorized reply
request
• Current solution: replicated state machine3f + 1 versions of serviceHurts confidentiality
• Our solution: rethinking replicated state machineCheaper: 2f + 1 versions of serviceHelps confidentiality
JPM, The University of Texas at Austin p.2/17
Outline
• Introduction• Separating Agreement from Execution• Enables
Fewer service replicaConfidentiality
• Prototype• Conclusion
JPM, The University of Texas at Austin p.3/17
Current Solution
Client 1V
Client 2V
3f+1 replicasf=1
• ClientSend request and repeatsPick majority reply
• Correct replica must return same replyStart from same stateAll replicas process the same requestsin the same order (replica coordination)
• HowReplicated state machine protocol
JPM, The University of Texas at Austin p.4/17
Separating Agreement from Execution
ClientV
Agreementcluster3g+1
Executioncluster2f+1
• Split problem into independent concernsAgreement: All agree on sequence of requestsExecution: Requests executed in order
• Note different requirementsAgreement: 3g + 1 servers, g faultsExecution: 2f + 1 servers, f faults
JPM, The University of Texas at Austin p.5/17
Implementation
Agreementcluster
Executioncluster
Agreementcertificate
Replycertificate
request
ClientV
Replycertificate
1. Assign unique sequence number to request2. !request, sequence number"A: unique, certified
3. Execute in sequence order4. !reply, sequence number"E: unique, certified
JPM, The University of Texas at Austin p.6/17
Cluster Implementation is Simple
Agreementcluster
Executioncluster
Agreementcertificate
Replycertificate
request
ClientV
Replycertificate
• Simple protocolAgreement using traditional protocolSend instead of executing
• Tricks in retransmissionExecution cluster internal retransmissionConfidential intercluster retransmission
JPM, The University of Texas at Austin p.7/17
Separation makes Replication Cheaper
ClientV
Agreementcluster3g+1
Executioncluster2f+1
• Execution clusterFewer service replicasExpensive because different
• Agreement clusterSimple nodes, reusable
• Can mergeJPM, The University of Texas at Austin p.8/17
Separation makes Replication Cheaper
ClientV
Combinedclusters3f+1
• Execution clusterFewer service replicasExpensive because different
• Agreement clusterSimple nodes, reusable
• Can mergeJPM, The University of Texas at Austin p.8/17
Confidentiality: The Problem
Hostile ClientV
traditionalreplicated state machine
f=1
Confidentialinformation
• Replication hurts confidentiality• Privacy Firewall restores it
JPM, The University of Texas at Austin p.9/17
Separation Enables Confidentiality
ClientV
Agreementcluster3g+1
Executioncluster2f+1
• Separation enables confidentialityAgreement nodes as filters
• Key 1: Restrict communication• Key 2: Separate choice from secrets
Choice in reply contentsChoice in who signs the replycertificateChoice in retransmission
• One choice remains: speed
JPM, The University of Texas at Austin p.10/17
The Privacy Firewall
h+1
V
h+1
• Nodes check reply certificate• Replicated for h Byzantine failures• Restrict communication• Only valid replies
h + 1 rows# one is correct• Always reply
h + 1 columns# one is correct• Minimal: (h + 1)2 servers
JPM, The University of Texas at Austin p.11/17
The Privacy Firewall
V
h+1
h+1
• Nodes check reply and order• Replicated for h Byzantine failures• Restrict communication• Only valid replies
h + 1 rows# one is correct• Always reply
h + 1 columns# one is correct• Minimal: (h + 1)2 servers
JPM, The University of Texas at Austin p.11/17
The Privacy Firewall
V
h+1
h+1
• Nodes check reply and order• Replicated for h Byzantine failures• Restrict communication• Only valid replies
h + 1 rows# one is correct• Always reply
h + 1 columns# one is correct• Minimal: (h + 1)2 servers
JPM, The University of Texas at Austin p.11/17
The Privacy Firewall
V
h+1
h+1
• Nodes check reply and order• Replicated for h Byzantine failures• Restrict communication• Only valid replies
h + 1 rows# one is correct• Always reply
h + 1 columns# one is correct• Minimal: (h + 1)2 servers
JPM, The University of Texas at Austin p.11/17
Privacy Firewall Guarantees
ClientV
Agreementcluster
Executioncluster
PrivacyFirewall
Client
• Output set confidentialOutput of correct cut is a valid output for acorrect node through unreliable link
• Only correct replies get throughReplies that correct nodes send
JPM, The University of Texas at Austin p.12/17
Timing Attacks Remain
ClientV
ClientV
answer="yes" answer="no"
• One choice remains: execution speed• Faulty execution server can influence whenmajority forms
• Information-theoretic confidentiality impossiblewithout synchrony
JPM, The University of Texas at Austin p.13/17
Prototype
• Built prototype from BASE [Rodrigues01]• Implements BFT confidential network file system• 10 machines: 1 client, 4 ag+PF, 2 PF, 3 exec.
Tolerate 1 fault in each of agreement, PF, exec.128MB RAM, 100Mbps switch
• Limitations of prototypeNo uninterruptible power supplySame codeCommunication not restricted
JPM, The University of Texas at Austin p.14/17
Latency Micro-Benchmarks
BASE Separate Confidential
Micro-benchmark
0
5
10
15
20
Late
ncy
(ms)
• Micro-benchmark latencyRemoved some BASE optimizationsOnly implemented one of six optimizations
JPM, The University of Texas at Austin p.15/17
Good Performance
NFS BASE Separate
MAB 500
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Run
time
(h)
• Separation and PF perform well in benchmarks+16% for confidentiality
JPM, The University of Texas at Austin p.16/17
Conclusion
• Take home message:
Separate agreement from execution!
• BenefitsFewer service replicasPrivacy FirewallEasy
JPM, The University of Texas at Austin p.17/17
