Efficient Fork-Linearizable Access to Efficient Fork-Linearizable Access to Untrusted Shared MemoryUntrusted Shared Memory

Presented by: Alex Shraer (Technion)Presented by: Alex Shraer (Technion)

IBM Zurich Research Laboratory

Christian CachinChristian Cachin

IBM ZurichIBM Zurich

Abhi ShelatAbhi Shelat

University of VirginiaUniversity of Virginia

Joint work with:Joint work with:

Data in a storage systemData in a storage systemUsers store data on remote storageUsers store data on remote storage

Data integrity:Data integrity: Single user – hashingSingle user – hashing

Merkle hash trees for large data volume Merkle hash trees for large data volume Multi user – digital signaturesMulti user – digital signatures

Public key infrastructure is neededPublic key infrastructure is needed

Data consistency?Data consistency? What if the server is faulty?What if the server is faulty?

Server

ModelModelAsynchronous system

Clients C1,…, Cn correct communicate only with the server via reliable links have public/private key pair (stored data is signed) each client executes read/write operations sequentially

Server S emulates n SWMR registers - client Ci writes only to register i CANNOT BE TRUSTED - perhaps faulty

Consistency Model?Consistency Model?

Attempt 1: linearizable shared memory Attempt 1: linearizable shared memory Requires a read to return the latest written valueRequires a read to return the latest written value ImpossibleImpossible

σ is linearizable if exists a sequential permutation that preserves 1. the real-time order of σ and 2. the sequential specification

Attempt 2: Sequential consistencyAttempt 2: Sequential consistency

Read does not have to return latest valueRead does not have to return latest value

For every process, the order in which its For every process, the order in which its operations take effect must be preserved operations take effect must be preserved Example:Example:

time

C1:

C2:

write (1, v) write (1, u)

read (1) → uread (1) → v

Sequential consistency – Cont.Sequential consistency – Cont.

Still impossible to implement !Still impossible to implement !

Proof:Proof:

write (1, u) read (2) → ┴

write (2, v) read (1) → ┴

time

C1:

C2:

Previously defined: Previously defined: (1) [Mazières, Shasha PODC 2002] (2) [Oprea, Reiter DISC 2006](1) [Mazières, Shasha PODC 2002] (2) [Oprea, Reiter DISC 2006]

The definition we use - similar to (2): The definition we use - similar to (2):

A seq. of events A seq. of events is is fork-linearizablefork-linearizable if for each client C if for each client Ci i

there exists a subsequence there exists a subsequence ii of of consisting only of consisting only of

completed operations and a sequential permutation completed operations and a sequential permutation ii of of ii

1.1. All operations of client CAll operations of client C ii are in are in ii

2.2. i i preserves real-time order of preserves real-time order of ii

3.3. II satisfies the sequential specificationsatisfies the sequential specification

4.4. If an operation If an operation op op is in is in II and and kk then the sequence of operations then the sequence of operations

that precede that precede opop in both groups is the same in both groups is the same

Fork-LinearizabilityFork-Linearizability

Every client sees

linearizable memory

R1

W1 R2

W2R3

C2

C3

C1

By telling one lie to CBy telling one lie to C11 & C & C22 and another to C and another to C33, ,

the server “forks” their viewsthe server “forks” their views

C1 & C2

Fork Linearizable Byzantine emulation (simplified)Fork Linearizable Byzantine emulation (simplified)

- every execution is fork-linearizable - every execution is fork-linearizable

- if the server is correct - if the server is correct

every operation completesevery operation completes

every execution is every execution is linearizablelinearizable

Global Fork-Linearizability: a simpler and equivalent Global Fork-Linearizability: a simpler and equivalent definitiondefinition

New on Fork-LinearizabilityNew on Fork-Linearizability

Some MotivationSome MotivationGuarantees a linearizable view for each client, and Guarantees a linearizable view for each client, and linearizable executions when the server is correctlinearizable executions when the server is correct

The server can hide operations from clients, but nothing worse!The server can hide operations from clients, but nothing worse! If the server forks the views of CIf the server forks the views of C11 and C and C22, their views are forked ever , their views are forked ever

after (no join), i.e., they do not see each other’s further updates after (no join), i.e., they do not see each other’s further updates otherwise the run is not fork-linearizable, which can be detected by the otherwise the run is not fork-linearizable, which can be detected by the

clients (unlike in linearizability or sequential consistency)clients (unlike in linearizability or sequential consistency)

fork-linearizability is not a weak conditionfork-linearizability is not a weak condition Linearizability is stronger Linearizability is stronger

(New) Sequential consistency is not !(New) Sequential consistency is not !

proof – in previous slides

Linearizab

le

ForkLinearizable

Seq.Consistent

Fork-Linearizability is the strongest known consistency that can be enforced

with a possibly faulty server

Emulating fork-linearizable Emulating fork-linearizable memory requires waitingmemory requires waiting

Theorem: Every protocol has executions with a correct server

where Ci must wait for Cj

Formal proof in the paper. The idea: by contradiction, assume that no waiting is necessary

r’(1) must return v since w’(1, v) might have completed

The server can cause this run to be indistinguishable from Run 1 v cannot be returned r’(1) cannot return neither u nor v in Run 1 – it must wait…

w(1, u) w’(1, v)

r(1) → u r’(1) → ?

C1

C2

w(1, u) w’(1, v)

r(1) → u r’(1) → v

C1

C2

Run 1: Correct server

Run 2: Faulty server

ProtocolsProtocolsTrivial method: Sign the complete history

Server sends history with all signatures Client verifies all operations and signatures Client adds its operation and signs new history Message size proportional to system age

[Mazières, Shasha PODC 2002] : Use n “version vectors” A blocking protocol and a concurrent protocol Communication complexity Ω(n2) Message size ~400MB for 10’000 users

Our results: A blocking protocol and a concurrent protocol Communication complexity O(n) Message size ~40KB for 10’000 users

Lock-Step with Correct Server (simplified)Lock-Step with Correct Server (simplified)

C1 <SUBMIT, WRITE, 1>000 0

00 <REPLY, >0

00

100

100< COMMIT, , (val,1) >1

00

Correct Server

<SUBMIT, READ, 1>000 1

00<REPLY, , >(val,1) 1

00

110

C2

110

100

100

110

100

<COMMIT, >

110

000

000

Lock-Step with Faulty Server (simplified)Lock-Step with Faulty Server (simplified)

000<REPLY, > , (┴,0) 0

00

010

010

100

<COMMIT, >

010

<SUBMIT, WRITE, 1>000 0

00<REPLY, >0

00

100

100

Faulty Server

<SUBMIT, READ, 1>000

100

100

000

000

C1

C2

100< COMMIT, , (val,1) >

What happened?What happened?Example 1 Example 2

write1(1, val)

start

read2 (1) → val

write1(1, val)

start

read2 (1) → ┴

000

100

110

000

100

010

The ≥ relation: V ≥ V’ if for all j, V[j] ≥ V’[j] B reads stale data of A B signs a version structure which cannot be ordered with what A signedProof idea – based on “No-Join” property: no operation signs a version structure V s.t. V ≥ VA and V ≥ VB

Subsequences i i can be constructed from the branch of Ci in the fork-tree

Increasing concurrency

Any protocol will sometimes block…

Concurrent protocol – details in the paper Allow operations to complete concurrently A read must wait for a previously scheduled concurrent write to the

same register to complete (at most one such operation)

Message size: O(n)

Summary of ResultsSummary of ResultsOn the notion of Fork-Linearizability

Global Fork-Linearizability Fork-Linearizable Byzantine emulations

Comparing Fork-Linearizability with seq. consistencyComparing Fork-Linearizability with seq. consistency

Communication efficient protocolsCommunication efficient protocols Lock-StepLock-Step ConcurrentConcurrent

A proof that any Fork-Linearizable Byzantine emulation A proof that any Fork-Linearizable Byzantine emulation must sometime blockmust sometime block

As in As in [MS02] and in our concurrent protocols and in our concurrent protocols

Questions?Questions?