Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
DistributedSystemsSecurity
Topics
• Byzan7nefaultresistance
• BitCoin
• CourseWrapUp
FaultTolerance
• Wehavesofarassumed“fail-stop”failures(e.g.,powerfailuresorsystemcrashes)
• Inotherwords,iftheserverisup,itfollowstheprotocol
• Hardenough:
• difficulttodis7nguishbetweencrashvs.networkdown
• difficulttodealwithnetworkpar77on
LargerClassofFailures
• Canonehandlealargerclassoffailures?
• Buggyserversthatcomputeincorrectlyratherthanstopping
• Serversthatdonotfollowtheprotocol
• ServersthathavebeenmodifiedbyanaQacker
• ReferredtoasByzan7nefaults
Model
• Provideareplicatedstatemachineabstrac7on
• Assume2f+1of3f+1nodesarenon-faulty
• Inotherwords,oneneeds3f+1replicastohandleffaults
• Asynchronoussystem,unreliablechannels
• Usecryptography(bothpublic-keyandsecret-keycrypto)
GeneralIdea
• Primary-backupplusquorumsystem
• Execu7onsaresequencesofviews
• Clientssendsignedcommandstoprimaryofcurrentview
• Primaryassignssequencenumbertoclient’scommand
• Primarywritessequencenumbertothe“register”implementedbythequorumsystemdefinedbyalltheservers
AQacker’sPowers
• Worstcase:asingleaQackercontrolstheffaultyreplicas
• Suppliesthecodethatfaultyreplicasrun
• Knowsthecodethenon-faultyreplicasarerunning
• Knowsthefaultyreplicas’cryptokeys
• Canreadnetworkmessages
• CantemporarilyforcemessagestobedelayedviaDoS
Whatfaultscannothappen?
• Nomorethanfoutof3f+1replicascanbefaulty
• Noclientfailure--clientscanneverdoanythingbad(orrathersuchbehaviorcanbedetectedusingstandardtechniques)
• Noguessingofcryptokeysorbreakingofcryptography
• Ques7on:inaPaxosRSMsebng,whatcouldtheaQackersorbyzan7nenodesdo?
Whatcouldgowrong?
• Primarycouldbefaulty!
• Couldignorecommands;assignsamesequencenumbertodifferentrequests;skipsequencenumbers;etc.
• Backupscouldbefaulty!
• Couldincorrectlystorecommandsforwardedbyacorrectprimary
• Faultyreplicascouldincorrectlyrespondtotheclient!
ExampleUseScenario
• Arvind:
echoA>grade
echoB>grade
tellPaul"thegradefileisready"
• Paul:
catgrade
Design1
• client,nservers
• clientsendsrequesttoallofthem
• waitsforallntoreply
• onlyproceedsifallnagree
• whatiswrongwiththisdesign?
Design2
• letushavereplicasvote
• 2f+1servers,assumenomorethanfarefaulty
• clientwaitsforf+1matchingreplies
• ifonlyfarefaulty,andnetworkworkseventually,mustgetthem!
• whatiswrongwithdesign2?
IssueswithDesign2
• f+1matchingrepliesmightbefbadnodes&1good
• somaybeonlyonegoodnodegottheopera7on!
• nextopera7onalsowaitsforf+1
• mightnotincludethatonegoodnodethatsawop1
• example:S1S2S3(S1isbad)
• everyonehearsandrepliestowrite("A")
• S1andS2replytowrite("B"),butS3missesit
• clientcan'twaitforS3sinceitmaybetheonefaultyserver
• S1andS3replytoread(),butS2missesit;read()yields"A"
• result:clienttrickedintoaccep7ngout-of-datestate
Design3
• 3f+1servers,ofwhichatmostfarefaulty
• clientwaitsfor2f+1matchingreplies
• fbadnodesplusamajorityofthegoodnodes
• soallsetsof2f+1overlapinatleastonegoodnode
• doesdesign3haveeverythingweneed?
RefinedApproach
• letushaveaprimarytopickorderforconcurrentclientrequests
• useaquorumof2f+1outof3f+1nodes
• haveamechanismtodealwithfaultyprimary
• replicassendresultsdirecttoclient
• replicasexchangeinfoaboutopssentbyprimary
• clientsno7fyreplicasofeachopera7on,aswellasprimary;ifnoprogress,forcechangeofprimary
PBFT:Overview
• Normalopera7on:howtheprotocolworksintheabsenceoffailures;hopefully,thecommoncase
• Viewchanges:howtodeposeafaultyprimaryandelectanewone
• Garbagecollec7on:howtoreclaimthestorageusedtokeepvariouscer7ficates
• Recovery:howtomakeafaultyreplicabehavecorrectlyagain
NormalOpera7on
• Threephases:
• Pre-prepare:assignssequencenumbertorequest
• Prepare:ensuresfault-tolerantconsistentorderingofrequestswithinviews
• Commit:ensuresfault-tolerantconsistentorderingofrequestsacrossviews
• Eachreplicamaintainsthefollowingstate:
• Servicestate
• Messagelogwithallmessagessent/received
• Integerrepresen7ngthecurrentviewnumber
Clientissuesrequest
• o:statemachineopera7on
• t:7mestamp
• c:clientid
Pre-prepare
• v:view
• n:sequencenumber
• d:digestofm
• m:client’srequest
Pre-prepare
Pre-prepare
Prepare
Prepare
PrepareCer7ficate
• P-cer7ficatesensuretotalorderwithinviews
• ReplicaproducesP-cer7ficate(m,v,n)iffitslogholds:
• Therequestm
• APRE-PREPAREforminviewvwithsequencenumbern
• 2fPREPAREfromdifferentbackupsthatmatchthepre-prepare
• AP-cer7ficate(m,v,n)meansthataquorumagreeswithassigningsequencenumberntominviewv
• Notwonon-faultyreplicaswithP-cer7ficate(m1,v,n)andP-cer7ficate(m2,v,n)
P-cer7ficatesarenotenough
• AP-cer7ficateprovesthatamajorityofcorrectreplicashasagreedonasequencenumberforaclient’srequest
• Yetthatordercouldbemodifiedbyanewleaderelectedinaviewchange
Commit
CommitCer7ficate
• C-cer7ficatesensuretotalorderacrossviews
• can’tmissP-cer7ficateduringaviewchange
• AreplicahasaC-cer7ficate(m,v,n)if:
• ithadaP-cer7ficate(m,v,n)
• logcontains2f+1matchingCOMMITfromdifferentreplicas(includingitself)
• ReplicaexecutesarequestaoeritgetsaC-cer7ficateforit,andhasclearedallrequestswithsmallersequencenumbers
Reply
BackupsDisplacePrimary
• Adisgruntledbackupmu7nies:
• stopsaccep7ngmessages(butforVIEW-CHANGE&NEW-VIEW)
• mul7casts<VIEW-CHANGE,v+1,P>
• PcontainsallP-Cer7ficatesknowntoreplicai
• Abackupjoinsmu7nyaoerseeingf+1dis7nctVIEW-CHANGEmessages
• Mu7nysucceedsifnewprimarycollectsanew-viewcer+ficateV,indica7ngsupportfrom2f+1dis7nctreplicas(includingitself)
ViewChange:NewPrimary
• The“primaryelect”p’(replicav+1modN)extractsfromthenew-viewcer7ficateV:
• thehighestsequencenumberhofanymessageforwhichVcontainsaP-cer7ficate
• twosetsOandN:
• ifthereisaP-cer7ficateforn,minV,n≤h
• O=O∪<PRE-PREPARE,v+1,n,m>
• Otherwise,ifn≤hbutnoP-cer7ficate:
• N=N∪<PRE-PREPARE,v+1,n,null>
• p’mul7casts<NEW-VIEW,v+1,V,O,N>
ViewChange:Backup
• BackupacceptsNEW-VIEWmessageforv+1if
• itissignedproperly
• itcontainsinVavalidVIEW-CHANGEmessagesforv+1
• itcanverifylocallythatOiscorrect(repea7ngtheprimary’scomputa7on)
• AddsallentriesinOtoitslog(sodidp’)
• Mul7castsaPREPAREforeachmessageinO
• AddsallPREPAREtologandentersnewview
GarbageCollec7on
• Forsafety,acorrectreplicakeepsinlogmessagesaboutrequestoun7lit
• ohasbeenexecutedbyamajorityofcorrectreplicas,and
• thisfactcanprovenduringaviewchange
• TruncatelogwithStableCer7ficate
• Eachreplicaiperiodically(aoerprocessingkrequests)checkpointsstateandmul7casts<CHECKPOINT,n,d,i>
• 2f+1CHECKPOINTmessagesareaproofofthecheckpoint’scorrectness
BFTDiscussion
• IsPBFTprac7cal?
• Doesitaddresstheconcernsthatenterpriseuserswouldliketobeaddressed?
Topics
• Byzan7nefaultresistance
• BitCoin
Bitcoin
• adigitalcurrency
• apublicledgertopreventdouble-spending
• nocentralizedtrustormechanism<--thisishard!
Whydigitalcurrency?
• mightmakeonlinepaymentseasier
• creditcardshaveworkedwellbutaren'tperfect
• insecure->fraud->fees,restric7ons,reversals
• recordofallyourpurchases
Whatishardtechnically?
• forgery
• doublespending
• theo
What’shardsocially/economically?
• whydoBitcoinshavevalue?
• howtopayforinfrastructure?
• monetarypolicy(inten7onalinfla7on)
• laws(taxes,laundering,drugs,terrorists)
Idea
• Signedsequenceoftransac7ons
• thereareabunchofcoins,eachownedbysomeone
• everycoinhasasequenceoftransac7onrecords
• oneforeach7methiscoinwastransferredaspayment
• acoin'slatesttransac7onindicateswhoownsitnow
Transac7onRecord
• pub(user1):publickeyofnewowner
• hash(prev):hashofthiscoin'sprevioustransac7onrecord
• sig(user2):signatureovertransac7onbypreviousowner'sprivatekey
• BitCoinhasmorecomplexity:amount(frac7onal),mul7plein/out,...
Transac7onExample
1. Yownsacoin,previouslygiventoitbyX:
• T7:pub(Y),hash(T6),sig(X)
2. YbuysahamburgerfromZandpayswiththiscoin
• ZsendspublickeytoY
• Ycreatesanewtransac7onandsignsit
• T8:pub(Z),hash(T7),sig(Y)
3. Ysendstransac7onrecordtoZ
4. Zverifies:T8'ssig()correspondstoT7'spub()
5. ZgiveshamburgertoY
DoubleSpending
• Ycreatestwotransac7onsforsamecoin:Y->Z,Y->Q
• bothwithhash(T7)
• Yshowsdifferenttransac7onstoZandQ
• bothtransac7onslookgood,includingsignaturesandhash
• nowbothZandQwillgivehamburgerstoY
Defense
• publishlogofalltransac7onstoeveryone,insameorder
• soQknowsaboutY->Z,andwillrejectY->Q
• a"publicledger"
• ensureYcan'tun-publishatransac7on
StrawmanSolu7on
• Assumeap2pnetwork
• Peersfloodnewtransac7onsover“overlay”
• Transac7onisacceptableonlyifmajorityofpeersthinkitisvalid
• Whataretheissueswiththisscheme?
BitCoinBlockChain
• theblockchaincontainstransac7onsonallcoins
• manypeers,eachwithacompletecopyofthechain
• proposedtransac7onsfloodedtoallpeers
• newblocksfloodedtoallpeers
• eachblock:hash(prevblock),setoftransac7ons,nonce,currentwallclock7mestamp
• newblockevery10minutescontainingnewxac7ons
• payeedoesn'tverifyun7lxac7onisintheblockchain
“Mining”Blocks
• requirement:hash(block)hasNleadingzeros
• eachpeertriesnoncevaluesun7lthisworksout
• tryingonenonceisfast,butmostnonceswon'twork
• miningablocknotaspecificfixedamountofwork
• onenodecantakemonthstocreateoneblock
• butthousandsofpeersareworkingonit
• suchthatexpected7metofirsttofindisabout10minutes
• thewinnerfloodsthenewblocktoallpeers
• thereisanincen7vetomineablock—12.5bc
Timing
• start:allpeersknow7llB5
• andareworkingonB6(tryingdifferentnonces)
• YsendsY->Ztransac7ontopeers,whichfloodit
• peersbufferthetransac7onun7lB6iscomputed
• peersthatheardY->Zincludeitinnextblock
• soeventuallyblockchainis:B5,B6,B7,whereB7includesY->Z
DoubleSpending
• whatifYsendsoutY->ZandY->Qatthesame7me?
• nocorrectpeerwillacceptboth
• ablockwillhaveonebutnotboth
• buttherecouldbeafork:B6<-BZandB6<-BQ
ForkedChain
• eachpeerbelieveswhicheverofBZ/BQitsawfirst
• triestocreateasuccessor
• ifmanymoresawBZthanBQ,morewillmineforBZ
• soBZsuccessorlikelytobecreatedfirst
• evenotherwiseonewillbeextendedfirstgivensignificantvarianceinminingsuccess7me
• peersalwaysswitchtominingthelongestfork,reinforcingagreement
DoubleSpendingDefense
• waitforenoughblockstobeminted
• ifafewblockshavebeenminted,unlikelythatadifferentforkwillwin
• ifsellingahigh-valueitem,thenwaitforafewblocksbeforeshipping
• couldaQackerstartaforkfromanoldblock?
• yes,butforkmustbelongerforotherstobelieve
• yes--butforkmustbelongerinorderforpeerstoacceptit
• iftheaQackerhas1000sofCPUs--morethanallthehonestbitcoinpeers--thentheaQackercancreatethelongestfork
• systemworksonlyifnoen7tycontrolsamajorityofnodes
BitCoinSummary
• Keyidea:blockchain
• Publicledgerisagreatidea
• Decentraliza7onmightbegood
• MiningisacleverwaytoavoidsybilaQacks
• WillBitCoinscalewell?
ClassSummary
• Implemen7ngdistributedsystems:systemandprotocoldesign
• Corealgorithms:clocks,snapshots,transac7ons,2PC,Paxos
• Realsystems:VM-FT,DSM,GFS,BigTable,MegaStore,Spanner,Chord,Dynamo
• Abstrac7onsforbigdataanaly7cs
• Buildingsecuresystemsfromuntrustedcomponents
Trends
• Transac7onsovergeo-distributed,replicateddata
• COPS(Princeton),Tapir(UW),RIFL/RamCloud/Rao(Stanford)
• Accelera7ngdistributedsystemsusinghardwaresupport
• Catapult(Microsoo),Annapurna(Amazon),Cavium,Mellanox
• Bigdataanaly7csforDNNs
• MXNet/TVM(UW),Torch,Theano,Dawn(Stanford),Rise(Berkeley)