The Push/Pull Model of Transactions - Computer Science · The Push/Pull Model of Transactions ... Out-of-order PUSHing with redo-logs Application: Optimism vs. Pessimism

The Push/Pull Model of Transactions

Eric KoskinenYale University, New Haven

United States

Matthew ParkinsonMicrosoft Research, Cambridge

United Kingdom

Appeared in PLDI 2015

CertiKOS Weekly Lunch 16 October 2015

Concurrent HashMap

Concurrent Queue

Concurrent List

Thread 1

Thread 3

Thread 2

Thread 4

put get enq deq add rm

Shared Memory

rd wr

Concurrent HashMap

Concurrent Queue

Concurrent List

Thread 1

Thread 3

Thread 2

Thread 4


Shared Memory

rd wr

Adding atomic to a language . . .

Version 4.7

Serializable. Opaque. …

Concurrent HashMap

Concurrent Queue

Concurrent List

Thread 1

Thread 3

Thread 2

Thread 4


Shared Memory

rd wr


Concurrent HashMap

Concurrent Queue

Concurrent List

Thread 1

Thread 3

Thread 2

Thread 4


Shared Memory

rd wr

many issues arise in the implementation.

Serializable. Opaque. …? ? ?


Pessimism!Thread

atomic { x := 3; y := 9; }

x changed?

y changed?

Thread

atomic { x := 3; y := 9; }all ok?

Optimism!

Serializability!

Interleaved execution equivalent to some serial execution.

Opacity!

Interleaved execution equivalent to some

serial execution and

cannot observe intermediate state.

Pessimism!

Optimism!

Dependent Txns

ADT Ops (Boosting)

Thread

atomic { x := 3; y := 9; }

Thread

atomic { stk.push(4); ht.get('a'); }

Serializability!

Opacity!Pessimism!

Optimism!Memory Ops

ADT Ops (Boosting)

Serializability!

Opacity!Pessimism!

Optimism!Memory Ops

Hardware TMSoftware TM

Hybrid TM

Dependent Txns

ADT Ops (Boosting)

Serializability!

Opacity!Pessimism!

Optimism!Memory Ops


Hybrid TM Open Nested

Dependent Txns

Irrev

ocab

ility

Consistency

Eager Writes

Lazy Reads

ADT Ops (Boosting)

Serializability!

Opacity!Pessimism!

Optimism!Memory Ops


Hybrid TM Open Nested

Dependent Txns

Irrev

ocab

ility

Consistency

Eager Writes

Lazy Reads

Serializable. Opaque. …? ? ?

The PUSH/PULL Model

Push⟨ht.get(5),_⟩

Pull⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩

⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩

The PUSH/PULL Model




Serializable. …Opaque.


Linearizable Commute

The PUSH/PULL Model




Thread

atomic { if(b) { opA(…); } else … }

Thread

atomic { while(…){ if(x>MAX) opB(…); … }

Thread

atomic { if(b) { opA(…); } else … }

Thread

atomic { while(…){ if(x>MAX) opB(…); … }Linearizable

OperationLinearizable Operation

Thread Thread

{tx c1, σ1} {tx c2, σ2}

1. Abstract away the language semantics

Thread Thread

{tx c1, σ1} {tx c2, σ2}


example language:


Thread Thread

2. Abstract away the state (with logs)

{tx c1, σ1, L1} {tx c2, σ2, L2}G

⟨op1,_⟩⟨op2,_⟩


Thread Thread

2. Abstract away the state (with logs)

{tx c1, σ1, L1} {tx c2, σ2, L2}G

Closed under log prefix

Parameterized by predicate allowed

⟨op1,_⟩⟨op2,_⟩

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}G

Closed under log prefix

Parameterized by predicate allowed

Log Equality

⟨op1,_⟩⟨op2,_⟩

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}G

Log Equality

⟨op1,_⟩⟨op2,_⟩

7 Simple Rules:

Apply Push Pull

Unpull Unpush Unapply

Commit

Apply Push Pull Unpull Unpush Unapply Commit

Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}Two Logs

atomic { op1; op2; … }


G

⟨ht.map(3,x),σ,σ1,c⟩⟨ht.map(7,2),_,gUC⟩


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}


atomic { op3; op4; … }A

pp

ly

G

1

⟨ht.map(3,x),σ,σ1,c⟩⟨q.enq(‘a’),σ1,σ'1,c1⟩

⟨ht.map(7,2),_,gUC⟩

1


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



pp

ly

G

Criterion (i):Criterion (ii):Criterion (iii):

L1 allows ⟨m,σ1,σ'1,id⟩

fresh(id)

APPLY

1

⟨ht.map(3,x),σ,σ1,c⟩⟨q.enq(‘a’),σ1,σ'1,c1⟩


1

{tx c1, σ1, L1}, G ➝

{tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G

append

Possible next op

Valid semantics of log


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}


atomic { op3; op4; … }U

na

pp

ly

G

UNAPPLY

⟨ht.map(7,2),_,gUC⟩⟨ht.map(3,x),σ,σ1,c⟩⟨q.enq(‘a’),σ1,σ'1,c1⟩

{tx c'1, σ'1, L1·[op,σ1,σ'1,c1]}, G ➝

{tx c1, σ1, L1}, G


Thread Thread




⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩

G



Thread Thread




⟨ht.map(3,x),_⟩Push

⟨q.enq(‘a’),_⟩⟨ht.map(3,x),_,gUC⟩

G



Thread Thread




Push

G

PUSH

Criterion (i):Criterion (ii):Criterion (iii):

op ◀ ⎣`L⎦unpushed

G allows op

Act as if op happens next

No conflict w/ other uncmted

Left-mover over logs⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩


{tx c, σ, `L·[op]·L'}, G ➝

{tx c, σ, `L·[op]·L'}, G·[op,gUC]

⎣G⎦gUC ∖ ⎣L1⎦pushed ◀ op

Uncommitted

Application: Out-of-order PUSHing with redo-logsApplication: Optimism vs. Pessimism


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



⟨ht.map(3,x),_⟩Pull

G

PULL

L2 allows opCriterion (i):Criterion (ii):

op ◀ ⎣L2⎦pushed ∪ ⎣L2⎦unpushedCriterion (iii):

op ∉ L2Didn’t pull already

Local log allows op

Can act as if op happened earlier

⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩


{tx c2, σ2, L2}, G1·[op,g]·G2 ➝

{tx c2, σ2, L2·[op]}, G1·[op,g]·G2

Application: Opacity [GK’08] and dependent transactions [RRHW’09]

⟨q.enq(‘a’),_,gUC⟩Push


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



pp

ly ⟨ht.get(5),_⟩

G

⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩



Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



Push⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩

Pull

G

⟨ht.get(5),_⟩⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩

⟨q.enq(‘a’),_⟩ ⟨q.enq(‘a’),_,gUC⟩⟨ht.map(3,x),_,gUC⟩⟨ht.map(7,2),_,gUC⟩


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩

G



Pull⟨q.enq(‘a’),_⟩


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}




G



⟨q.enq(‘a’),_⟩⟨ht.get(9),_⟩

Co

mm

it


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}




G



⟨q.enq(‘a’),_⟩⟨ht.get(9),_⟩

Co

mm

it

COMMIT

cmt(G, L1, G')⎣L1⎦pulled ⊆ ⎣G⎦gC

Criterion (ii):Criterion (iii):Criterion (iv):

L1 ⊆ GCriterion (i): fin(c1)

Pushed all my stuff

Pulled ops are committed

Swap my flags from gUC to gC

{tx c1, σ1, L1}, G ➝

{tx c1, σ1, L1}, G’


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩⟨ht.get(5),_⟩



⟨q.enq(‘a’),_⟩⟨

Un

ap

ply

G


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}



⟨ht.get(5),_,gUC⟩⟨ht.get(5),_⟩⟨ht.get(5),_⟩



⟨q.enq(‘a’),_⟩U

np

ull

G

UNPULL

(Local log is still allowed, even with element removed.)


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}






⟨q.enq(‘a’),_⟩

G

Unpush

UNPUSH

(Everything I subsequently pushed could have been pushed earlier.)

Application: Inverses or “compensating actions”


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}




⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩


⟨q.enq(‘a’),_⟩

G

Un

ap

ply

q.deq()


Thread Thread

{tx c1, σ1, L1} {tx c2, σ2, L2}




⟨ht.map(3,x),_⟩⟨ht.map(3,x),_⟩⟨q.enq(‘a’),_⟩


⟨q.enq(‘a’),_⟩

G

Un

ap

ply

q.deq()

Theorem. The Push/Pull model is serializable.Simulation with uninterleaved machine

Closure under log rewind

Preservation invariant, universal quantification over dropping uncommitted global ops universal quantifying over local log rewind.

The PUSH/PULL Model




Theorem. Fragments of The Push/Pull model are opaque.

The PUSH/PULL Model




example: Don’t PULL uncommitted effects.

Other Applications.

The PUSH/PULL Model




• Pessimistic models of STM. • Pessimistic models of STM/Boosting. • Optimistic models of STM. • Optimistic models of STM/Boosting (TRANSACT’15) • Closed-nested transactions: don’t share until commit. • Dependent transactions: view uncommitted effects but

delay commit (or cascading aborts)

The PUSH/PULL Model




Mixing hardware TM with transactional boosting

Models that are yet-to-come.


Linearizable Commute

The PUSH/PULL Model




The PUSH/PULL Model




Eric KoskinenIBM Research, Yale University

United States

Matthew ParkinsonMicrosoft Research, Cambridge

United Kingdom

Thank you!

Documents

The Push/Pull Model of Transactions - Computer Science · The Push/Pull Model of Transactions ... Out-of-order PUSHing with redo-logs Application: Optimism vs. Pessimism