OVERVIEWLBFS   MOTIVATION   INTRODUCTION

  CHALLENGES   ADVANTAGESOFLBFS   HOWLBFSWORKS?   RELATEDWORK

  DESIGN   SECURITYISSUES   IMPLEMENTATION

  SERVERIMPLEMENTATION   CLIENTIMPLEMENTATION

  EVALUATION   SHARK

MOTIVATION

  UsersrarelyconsiderrunningNFSoversloworwideareanetworks.

  Ifbandwidthislow,performanceisunacceptable.

  Datatransferssaturatebo@lenecklinksandcauseunacceptabledelays.

  InteracAveapplicaAonsareslowinrespondingtouserinput.

  RemoteloginisfrustraAng

  SoluAon? Run InteracAve programs locally and manipulate remote files

throughthefilesystem. NetworkFilesystemshouldconsumelessbandwidth.  LBFS.

INTRODUCTION

 LBFSisusedforsloworWideareaNetworks.

 ExploitssimilariAesbetweenFilesorversionsofthesamefile.

 ItusesConvenAonalcomparisonandCaching.

 In LBF, interacAve programs and accessing remote datathroughfilesystemrunlocally.

 LBFS requires 90% less bandwidth than TradiAonalNetworkFileSystem.

 Challenges? Advantages

 ProvidesTradiFonalFSSchemaFcs LocalCache ExploitsCrossFilesimilariFes

 VariableSizeChunks Indexeschunksbyhashvalues

 HowLBFSWork? ProvidesClosetoOpenConsistency

RELATEDWORK

  AFSusesservercallbackstoreducenetworktraffic.

  LeasesarecallbackswithexpiraAondate.

  CODA supports slow networks and even disconnected operaAonsthroughopAmisAcreplicaAon.

  CODAsavesbandwidthasitavoidstransferringfilestotheserver.

  BayouandOceanStoreinvesAgateconflictresoluAonforopAmisAcupdates.

  SpringandWetherall: Uselargeclientandservercaches

  RsyncexploitssimilariAesbetweendirectorytrees.

DESIGN

 LBFSuseslargepersistentfilecacheatclient. ItassumesClienthasenoughcache.

  ItExploitssimilariAesbetweenfilesandfileversions. DividesFilesintoChunks. Onlytransmitsdatachunkscontainingnewdata.

 Tosavechunktransfer,LBFSreliesontheSHA‐1Hash.

 LBFSUses“gzip”compression.

 CentralchallengeinDesignis: Keepingtheindexareasonablesize Dealingwithshi[ingoffsets.

PROBLEMSWITHFIXEDSIZEDBLOCKS

 Single byte inserAon shi[s all the blockboundaries.

OriginalFile

AUerInserFng

 PossiblesoluFons: Indexfilesbythehashesofalloverlapping8KBblocksatalloffsets.

 Rsync:Consideronly twofilesataAme.Existenceofafileisfoundusingfilename.

x

LBFSSoluFonforBlockSize

 LBFS Onlylooksfornon‐overlappingchunksinfiles

 AvoidssensiFvitytoshiUingfileoffsetsbySeZngchunkboundariesbasedonfilecontents.

 Todivideafileintochunks,LBFS Examinesevery(overlapping)48‐byteregionofthefile.

 LBFSUses Rabin’s fingerprints to select boundary regionscalledbreakpoints.

 Fingerprintsareefficienttocomputeonaslidingwindowinafile.

RabinFingerPrints

 Polynomial representaAonofdata in48‐byte regionmoduloanirreduciblepolynomial. FingerPrint=f(x)modp(x)

 ProbabilityofCollision=max(|r|,|s|)/2w‐1

 Boundaryregionshavethe13leastsignificantbitsoftheir fingerprint equal to an arbitrary predefinedvalue.

 Methodisreasonablyfast.

ChunkBoundariesAUeraSeriesofEdits?

  Figureshowsthefiledividedintovariablelengthchunkswithbreakpointsdeterminedbyhashofeach48bitregion.

  EffectofinserAngsometextintothefileatchunkC4.  TransferonlyC8.

  EffectofinserAngadatainC5thatcontainsabreakpoint  Spliengthechunksintotwonewchunks.(C9andC10)  TransferonlytwonewchunksC9andC10.

  Oneofthebreakpointiseliminated.C2+C3‐>C11  TransferonlyC11

PATHOLOGICALCASES VariablesizechunkscanleadtoPathologicalbehavior.

  Ifevery48bytesofafilehappenedtobeabreakpoint.

 VerylargechunkswouldbetoodifficulttosendinasingleRPC.

 ArbitrarysizeRPCmessageswouldbesomewhatinconvenient.

 Chunksizesmustbebetween2Kand64K

 ArAficiallyinsertchunkboundariesiffileisfullofrepeatedsequences.

CHUNKDATABASE Thechunkdatabase Indexeschunksbyfirst64bitsofSHA‐1

hash.

 Thedatabasemapskeysto(file,offset,count)triples.

 LBFSneverreliesonthecorrectnessofthechunkdatabase.

 Howtokeepthisdatabaseuptodate? Mustupdateitwheneverfileisupdated

 CansAllhaveproblemswithlocalupdatesatserversite Crashescancorruptdatabasecontents.

FILECONSISTENCY TheLBFSclientcurrentlyperformswholefilecaching.

 LBFSusesathree‐Aeredschemetodetermineifafileisuptodate. OPENAFILE:

 IFLeaseNotExpired IFLeaseExpired

 ClientgetsaleasefirstAmeafileisopenedforread.

 ClientRenewstheexpiredleasebyrequesAngfilea@ributes.

  It’sthejoboftheClienttocheckifthecachedcopyissAllcurrent.

FILEREADS LBFSUseaddiAonalcallsnotinNFS‐>GETHASHforreads

FILEWRITES LBFSServerupdatesfilesatomicallyatcloseAme.

 UsesTemporaryFiles. 4RPC’sareusedinupdateprotocol:

*1.MKTMPFILE*2.CONDWRITE*3.TMPWRITE*4.COMMITTMP.

SECURITY LBFSusesthesecurityinfrastructurefromSFS. AllServershavepublickeys. AccessControl.

IMPLEMENTATION

 mkdbuAlity  IfFileSize<8KB TrashDirectory

EVALUATION–REPEATEDDATAINFILES

 Bandwidth consumpAon and network uAlizaAon aremeasuredunderseveralcommonworkloads.

 LBFSiscomparedwith: CIFS, NFSversion3and AFS.

EVALUATION(Cont)–BANDWIDTHUTILIZATION

•  Used3WORKLOADS.

•  (MSWord1.4MBfile,gcc‐>Compiledemacs20.7,ed‐>perl)

EVALUATION(3)–APPLICATIONPERFORMANCE

OVERVIEW

SHARK  MOTIVATION   INTRODUCTION  CHALLENGES  ADVANTAGESOFSHARK  HOWSHARKWORKS?

  DESIGN   IMPLEMENTATION   EVALUATION  CONCLUSTION  DISCUSSION(QUESTIONARIES)

MOTIVATION

CurrentSystems

1.ReplicaFngExecuFonEnvironment 2.MulFpleClientCopyingSameFiles

SERVER

Program1

libraries

P2P3

P4

Data

CLIENT

P1Launch

Data

P1

C2

P1

C3

P1

C6

P3

C4

P1

C5

Replicate their execuFon environment on each machine beforelaunchingdistributedapplicaFon‐>WASTERESOURCES+DEBUG

SERVER1

Data1 Data2

Data3

Data3

Data2

Data1

SERVER3

Data1

Data2

Data3

SERVER2

Client1

C3

C2C4

RequestData3

UpdateDB

UpdateDB

Data3,1 Data2,1

UpdateDB

Data3

ReplicaFngdataandservingsamecopyofdatatomulFpleclients‐>INCONSISTENCIES

INTRODUCTION

 Sharkisadistributedfilesystem

 Designedforlarge‐scale,wide‐areadeployment.

 Scalable.

 ProvidesanovelcooperaAve‐cachingmechanism

 Reducestheloadonanoriginfileserver.

CHALLENGES

 Scalability:Whatifalargeprogram(sayinMB’s)isbeingexecutedfromafileserverbymanyclients?

 Becauseofbandwidth,servermightdeliverunacceptableperformance.

 As the model is similar to P2P file systems, administraAon,accountability,andconsistencyneedtobeConsidered.

HOWSHARKWORKS?

 Shark clients findnearby copies of data by using distributedindex.

 Clientavoids transferring thefile/chunksof thefile fromtheserver,ifthesamedatacanbefetchedfromnearby,client.

 Shark is compaAble with exisAng backup and restoreprocedures.

 By shared read, Shark greatly reduces server load andimprovesclientlatency.

DESIGN–PROTOCOL(1/2)

1.  Sharkserverdividesthefile intochunksbyusingRabinfingerprintalgorithm.

2.  Shark interacts with the local host using an exisAngNFSv3andrunsinuserspace.

3.  WhenFirstClientreadsaparAcularfile  Gets file andRegisters as replica proxy for the chunks of the

fileinthedistributedindex.

4.  Nowwhen2ndclientwantstoaccessthesamefile:  it discovers the proxies of the file chunks by querying the

distributedindex. establishes a secure channel to (mulAple such) proxy(s) , and

downloadthefilechunksinparallel.

DESIGNPROTOCOL(2/2)

5.A[erfetching,theclientthenregistersitselfasareplicaproxyforthesechunks.

6.ServerexposestwoApi’s. Put: Client executes put to declare that it has something.

 Get: clientexecutesget toget the listof clientswhohavesomething.

SECUREDATASHARING7. Data is encrypted by the sender and can be decrypted

onlybytheclientwithappropriatereadpermissions.

8.Clientscannotdownload largeamountsofdatawithoutproperreadauthorizaAon.

SECUREDATASHARING

 How? Cross‐file‐systemsharing

 Sharkuses tokengeneratedbythefileserverasasharedsecretbetweenclientandproxy.

 Clientcanverifytheintegrityofthereceiveddata.

 For a sender client (proxy client) to authorize requesterclient,requesterclientwillprovidetheknowledgeoftoken

 Once authorized, receiver client will establish the readpermissionofthefile.

FILECONSISTENCY

 Sharkusestwonetworkfilesystemtechniques: Leases

 AFS‐stylewhole‐filecaching

 WhenclientmakesareadRPCtothefileserver,itgetsareadleaseontheparAcularfile.

  InSharkdefaultleaseduraAon=5mins.

 ForFileCommonaliAes:ItusesLBFS.

COOPERATIVECACHING

Client(C1) Server C2 C3 C4

FileNotCached/LeaseExpired

GETTOK (fh, offset, count)

1. TF = tok (F) = HMACr(F) 2. Split data into chunks 3. Compute tokens for chunks

1. file attributes 2. file token 3. (chunktok1, offset1, size1) (chunktok2, offset2, size2) (chunktok3, offset2, size2)

Determinesif(LocalCache==Latest)

if(LocalCacheisnotlatest=>Create‘K’Threads

t1

Multicast Requesting for Chunk Fi

Chunk F1

Chunk F3

t2t3

t1t2t3

Request Chunk F2 t3

F2

F3

Chunk F2

Issues Series of Read calls to the Kernel NFS server.

Caches the tokens for future reference.

.

Fetch‘K’ChunksinParallel

PUT() -> Announuces as a Proxy for Fi

DISTRIBUTEDINDEXING

 Sharkusesglobaldistributedindexforallsharkclients.

 Systemmaps opaque keys onto nodes by hashing the valueontoakeyID.

 AssigningID’stonodesallowslookupontheO(log(n))

 SharkstoresonlysmallinformaAonaboutwhichclientsstoreswhatdata.

 SharkusesCoralasitsdistributedindex.

 CoralProvidesDistributedSloppyHashTable(DSHT).

 Coralcacheskey/valuepairsatnodeswhoseIDsareclose.

IMPLEMENTATION

  Sharkconsistsof3MainComponents: Serversidedaemon

 Clientsidedaemon

 Coraldaemon

  Implemented in C++ andarebuiltusingSFStoolkit.

  Clientsidedaemon

  BiggestComponentofshark.

  HandlesUserRequests  Transparently

incorporateswholefilecaching.

  Codeis~12,000Lines

EVALUATION(1/2)

 SharkisevaluatedagainstNFSv3andSFS.

 Readtestsareperformedboth

 withinthecontrolledEmulabLANenvironmentand

 InthewideareaonthePlanetLabv3.0test‐bed.

 Theserverrequired0.9seconds tocomputechunksfora10MBrandomfile,and3.6secondsfora40MBrandomfile.

EVALUATION‐Microbenchmarks

  Forthelocal‐areamicro‐benchmarks,localmachinesatNYUareusedasaSharkclient.

  Inthismicro‐benchmark,Shark’schunkingmechanismreducesredundantdatatransfersbyexploiAngdatacommonaliAes.

QUESTIONS

THANKYOU