TRUST Spring Conference, April 2-3, 2008 Write Markers for Probabilistic Quorum Systems Michael Merideth, Carnegie Mellon University Michael Reiter, University

TRUST Spring Conference, April 2-3, 2008

Write Markers for Probabilistic Quorum Systems

Michael Merideth, Carnegie Mellon University

Michael Reiter, University of North Carolina

4/3/08 Michael Merideth 2

Replication via Quorum Systems

Replicated data– Server becomes n replicas

Server

Clients


Replicas Replicated data

– Server becomes n replicas Clients issue read and write

operations– Involve quorums (subsets)

of replicas

High availability– Yet, no writes lost,

forged, or corrupted Clients

Replication via Quorum Systems


Types of Servers (in Examples)

bowling ball ice cream fish any value

non-faulty faulty


Types of Clients (in Examples)

non-faulty faulty


Write Operation

Client wants to write “ice cream” to system


Write Operation

Client submits write to write quorum


Write Operation Complete

Positive responses from quorum means write complete




Read Operation

Client queries read quorum for values


Read Operation

Determines read value based on votes (responses) from entire quorum

(Chooses “ice cream”)


Write Markers Concept

Write marker: additional data (written with value) that identifies write quorum– Verified by clients during read

Improves properties of probabilistic quorum systems– Tolerate more faults and use smaller quorums


Outline

Strict, Byzantine quorum systems Probabilistic, Byzantine quorum systems Benefits of write markers Idea for implementation


Byzantine Quorum System[malkhi & reiter 98]

Byzantine (arbitrary) faults– Faulty nodes may lie– Faulty clients and servers may

collude b faulty servers

– Identity of faulty nodes unknown by non-faulty nodes


Write Operation

Write quorum may contain faulty servers




Read Operation

Faulty servers may fabricate value


Stale Values

Stale (logically older) values are detectible


Conflicting Values

Faulty servers may also fabricate conflicting (logically concurrent) values– E.g., same timestamp

Here “fish” conflicts with “ice cream”– But ice cream has more votes


More Conflicting Values

Non-faulty servers may also return conflicting values

For example, in single-round write protocols– Such protocols are desirable for

efficiency– Client may (perhaps unknowingly)

submit a write that is conflicting


Conflicting Write

Same as normal write


Conflicting Write Incomplete

Accepted by non-faulty servers that have not accepted (conflicting) value

Write does not complete


Which Value is Correct?

“Ice cream” was complete– … therefore is correct

“Fish” was incomplete– … therefore should be ignored

But ice cream and fish get equal votes

Client uncertain?


Conflicting Values: Problematic

Must outvote conflicting replicas Thus, many potentially conflicting

replicas implies ability to tolerate (relatively) few faults

?


Impact of Conflicting Replicas

Quorum Conflict Faults Protocols

Opaque< n/5 (least)

e.g., Q/U

Masking < n/4e.g., Fleet,

PASIS

Dissemin-ation

< n/3 (most)

e.g., BFT, HQ

?


Choice of Quorums Important

Choices of read quorum and both write quorums led to problem– Other choices lead to correct

answer

?


Choice of Quorums Important

Choices of read quorum and both write quorums led to problem– Other choices lead to correct

answer


Idea: Select Quorums at Random

In fact, correct answer in expectation (in this example)– If quorums chosen uniformly at

random(an access strategy)


Probabilistic Quorum Systems[malkhi, reiter, wool, wright 01]

Weakening intersection property to hold only with high probability – Provides better availability – Tolerates more faults

Bounds error probability– Probability that quorums chosen

according to access strategy yield incorrect (or uncertain) result


Probabilistic Opaque Quorum Systems[merideth & reiter 07]

Generalize access strategy– Quorums chosen from access sets– Access sets are chosen according to

access strategy

Tolerate Byzantine clients for all probabilistic quorum systems– Enforce access strategy


Probabilistic Quorum Systems

Reduce number of conflicting values in expectation– Therefore, tolerate more faults (with some bounded probability of error)

Conflicting

Faults

Strict Prob.

Opaque< n/5

(fewest)

Masking < n/4

Dissemination < n/3

< n/3.15

< n/2.62

< n (most)


Reduce conflicting replicas further?

Yes (for probabilistic masking and opaque quorum systems)– Write markers


Write Markers

Recall,– Write operations write values– Read operations poll replicas for values

Write marker– Additional data (written with value) that identifies

the write quorum (or access set) that was used– Client accepts vote (during read) only if replica was

part of write quorum (or access set)


Write Operations with Write Markers

Create write marker for quorum




Conflicting Write with Write Markers

Same as normal write


Conflicting Write Incomplete

Accepted by non-faulty servers that have not accepted (conflicting) value



“Ice cream” was complete– … therefore is correct

“Fish” was incomplete– … therefore should be ignored



Faulty client can only vote for “triangle”

Faulty client cannot vote for “star”


Benefit of Write Markers

Faulty servers cannot vote for conflicting value unless they are part of write

Due to probabilistic access strategy, faulty server not always part of write

Thus, fewer conflicting servers to outvote in expectation


Benefits of Write Markers

Conflicting

Faults

Strict Prob.Write-

markers

Opaque< n/5 (fewest)

Masking < n/4

Dissemination < n/3

< n/3.15

< n/2.62

< n (most)

< n/2.62

< n/2

< n(most)

Tolerate more faults


Benefits of Write Markers

Tolerate more faults Use smaller quorums

– See paper


Example with Benign Clients

For writes: clients choose access sets uniformly at random – Then encode and, e.g., digitally sign their choices (i.e.,

create a write marker) For reads: clients verify write marker


Write Markers with Byzantine Clients

Faulty clients:– Cannot be trusted to follow access strategy– May intentionally choose quorums that maximize

conflicting values

Constrain clients [merideth&reiter 07]– Even faulty clients follow access strategy– Avoids additional communication on critical path– Choice is verified by servers as (pseudo) random

Treat choice as write marker– Modify protocol so that clients also verify choice


Protocol Intuition

Servers provide pseudorandom sequence of access sets per client– Threshold signature from servers

…



For each operation, client locally chooses next access set in sequence; servers verify choice

Protocol Intuition

…


Protocol Intuition

…


For each operation, client locally chooses next access set in sequence; servers verify choice


Misuse by Faulty Client

What if faulty client:– Skips ahead to “better” access

set?– Waits to perform operation until

advantageous?

In either case, access set no longer random

…


Defending Against Misuse

Exponential increase in cost to use later access sets– Client puzzle (requires solution)

Correct value propagates in background [c.f. malkhi et al. 03]

Sequence becomes invalid as system progresses– Must obtain new sequence

…


Write Markers Mechanism

Use client puzzle – Servers already verify solution

Have clients verify as well – Treat solution and access set

as write marker– Return during read operations

Provides mechanism for write markers

…


Conclusion

Write markers provide benefits for probabilistic quorum systems– Reduce number of faulty servers that can vote for

conflicting value in expectation– Increase number of faults that can be tolerated

Opaque: up to n/2.62 (probabilistic: n/3.15; strict: n/5) Masking: up to n/2 (probabilistic: n/2.62; strict: n/4)

– Allow for smaller quorums in some cases

For more information:– Write Markers for Probabilistic Quorum Systems.

Michael G. Merideth and Michael K. Reiter. CMU Technical Report: CMU-ISR-08-110


Questions?






Documents

TRUST Spring Conference, April 2-3, 2008 Write Markers for Probabilistic Quorum Systems Michael Merideth, Carnegie Mellon University Michael Reiter, University