2017 SPTCC - Lock-free algorithms for Kotlin coroutinesneerc.ifmo.ru/sptcc/slides/slides-elizarov.pdf · Kotlin basic facts •Kotlin is a JVM language developed by JetBrains •General

Lock-freealgorithmsforKotlincoroutines

ItisallaboutscalabilityPresentedatSPTCC2017

/RomanElizarov@JetBrains

Speaker:RomanElizarov

• 16+yearsexperience• Previouslydevelopedhigh-perftradingsoftware@Devexperts• Teachconcurrent&[email protected]• Chiefjudge@NortheasternEuropeanRegionofACMICPC• NowworkonKotlin@JetBrains

Agenda

• Kotlincoroutinesoverview&motivationforlock-freealgorithms• Lock-freedoublylinkedlist• Lock-freemulti-wordcompare-and-swap• Combiningthemtogetmorecomplexatomicoperations (withoutSTM)

Kotlinbasicfacts

• KotlinisaJVMlanguagedevelopedbyJetBrains• Generalpurposeandstatically-typed• Object-orientedandfunctionalparadigms• OpensourceunderApache2.0• Reachedversion1.0in2016• Compatibilitycommitment• Nowatversion1.1

• OfficiallysupportedbyGoogleonAndroid

Kotlinis…

• Modern• Concise• Safe• Extensible• Pragmatic• Funtoworkwith!

Kotlinispragmatic

… andeasytolearn

Coroutines

Asynchronousprogrammingmadeeasy

Howdowewritecodethatwaitsforsomethingmostofthetime?

BlockingthreadsKotlin fun postItem(item: Item) {

val token = requestToken()val post = submitPost(token, item)processPost(post)

}

fun postItem(item: Item) {requestToken { token ->

submitPost(token, item) { post -> processPost(post)

}}

}

CallbacksKotlin

fun postItem(item: Item) {requestToken()

.thenCompose { token ->submitPost(token, item)

}.thenAccept { post ->

processPost(post) }

}

Futures/Promises/RxKotlin

CoroutinesKotlin fun postItem(item: Item) {

launch(CommonPool) {val token = requestToken()val post = submitPost(token, item)processPost(post)

}}

CSP&Actormodels

• Astyle ofprogrammingformodernsystems• Lotsofconcurrenttasks/jobs• Waitingmostofthetime• Communicatingallthetime

Sharedatabycommunicating

Kotlincoroutinesprimitives

• Jobs/Deferreds (futures)• join/await

• Channels• send&receive• synchronous&bufferedchannels

• Select/alternatives• Atomicallywaitonmultipleevents

• Cancellation• Parent-childhierarchies

Implementationchallenges

• Coroutinesarelikelight-weightthreads• Allthelow-level scheduling&communicationmechanismshavetoscale tolotsofcoroutines

Lock-freealgorithms

Buildingblocks

• Single-wordCAS(that’sallwehaveonJVM)• Automaticmemorymanagement(GC)• Practical lock-freealgorithms• Lock-FreeandPracticalDoublyLinkedList-BasedDeques UsingSingle-WordCompare-and-SwapbySundell andTsigas• APracticalMulti-WordCompare-and-SwapOperationbyTimothyL.Harris,KeirFraserandIanA.Pratt.

Doublylinkedlist

SN

1N

PS

PH T

sentinel sentinel

Usesamenodeinpractice

next linksformlogicallistcontentsprev linksareauxiliary

InsertPushRight (likeinqueue)

Doublylinkedlist (insert0)

SN

1N

PS

PH T

2N

P

create&init

1

2


SN

1N

PS

PH T

2N

P

CAS

RetryinsertonCASfailure


SN

1N

PS

PH T

2N

P

CAS

IgnoreCASfailure

”finishinsert”

RemovePopLeft (likeinqueue)

Doublylinkedlist(remove1)

SN

1N

PS

PH T

Markremovednode’snext link

Usewrapper objectformarkinpractice

Cachewrappersinpointed-tonodes

CAS

RetryremoveonCASfailure

1

2

Don’tuseAtomicMarkableReference

CAS


SN

1N

PS

PH T

Markremovednode’sprev link

RetrymarkingonCASfailure

”finishremove”


SN

1N

PS

PH T

CAS

”helpremove”– fixupnext links


SN

1N

PS

PH T

CAS

”correctprev”– fixupprev links

Statetransitions

Nodestates

Initnext:Okprev:Ok

prev.next:--next.prev:--

Insert1next:Okprev:Ok

prev.next:menext.prev:--

Insert2next:Okprev:Ok

prev.next:menext.prev:me

Remove1next:Remprev:Ok


Remove2next:Remprev:Rem



prev.next:++next.prev:me


prev.next:++next.prev:++

helpremove

correctprev

correctprev

1 2 3

4 5 6 7

Helping

Concurrentinsert

Concurrentinsert(0)

SN

1N

PS

PH T

2N

P

3N

P

I2

I3

Concurrentinsert(1)

SN

1N

PS

PH T

2N

P

3N

P

CASfail

CASok

I2

I3

Concurrentinsert(2)

SN

1N

PS

PH T

2N

P

3N

P

detectwrongprev(t.prev.next !=t)

I2

I3

Concurrentinsert(3)

SN

1N

PS

PH T

2N

P

3N

P

correctprev

I2

I3

Concurrentinsert(4)

SN

1N

PS

PH T

2N

P

3N

P reinit &repeat

I2

I3

Concurrentremove

Concurrentremove(0)

SN

1N

PS

PH T2

N

P

R1

R2

Concurrentremove(1)

SN

1N

PS

PH T2

N

P

R1

R1

R2

Concurrentremove(2)

SN

1N

PS

PH T2

N

P

R1

R2

Findsalreadyremoved

R1

R2

Concurrentremove(3)

SN

1N

PS

PH T2

N

P

R1

R2

helpremove

markprev

R1

R2

Concurrentremove(4)

SN

1N

PS

PH T2

N

P

R1

R2

Retrywithcorrectednext

R1

R2

Concurrentremove(5)

SN

1N

PS

PH T2

N

P

R1

R2

helpremove

R1

R2

Concurrentremove(6)

SN

1N

PS

PH T2

N

P

R1

R2

correctprev

R1

R2

Concurrentremove&insertWhenremovewins

Concurrentremove&insert(0)

SN

1N

PS

PH T

2N

P

create&init

R1

R1

I2


SN

1N

PS

PH T

2N

P

removefirst

R1

R1

I2


SN

1N

PS

PH T

2N

PCASfail

R1

R1

I2


SN

1N

PS

PH T

2N

P

detectwrongprev(t.prev.next -- removed)do“correctprev”R1

R1

I2


SN

1N

PS

PH T

2N

P

markprev

fixupnext

R1

R1

I2


SN

1N

PS

PH T

2N

P

R1

R1

I2

updateprev


SN

1N

PS

PH T

2N

P

R1

reinit &repeat

R1

I2

Concurrentremove&insertWheninsertwins


SN

1N

PS

PH T

2N

P

create&init

R1

R1

I2


SN

1N

PS

PH T

2N

P

R1

CAS

R1

I2


SN

1N

PS

PH T

2N

P

R1

R1

I2

willsucceedmarkingonremoveretry


SN

1N

PS

PH T

2N

P

R1

helpremove

markprev

R1

I2


SN

1N

PS

PH T

2N

P

R1

correctprev

R1

I2

Removeisover!


SN

1N

PS

PH T

2N

P

R1

correctprev

R1

I2

Takeaways

• Akindofalgo youneedapaperfor• Hardtoimprovew/owritinganotherpaper• Goodnews:stresstestsuncovermostimpl bugs• Badnews:whenstresstestfails,youuptolonghours• Morebadnews:hardtofindbugsthatviolatelock-freedomness ofalgorithm

Summary:whatwecando

• Insertitems(attheendofthequeue)• Removeitems(atthefrontofthequeue)• Traversethelist• Removeitemsatarbitrarylocations• InO(1)

Linearizability

• Insertlast• LinearizesatCASofnext

• Removefirst/arbitrary• Success– atCASofnext• Fail– atreadofhead.next

Moreaboutalgorithm

• Sundell &Tsigas algo supportsdeque operations• CanPushLeft &PopRight

• PopLeft issimple– readhead.next &remove• Butcannotlinearizethemallatcas points• PushLeft,PushRight,PopRight - Ok• PopLeft linearizesathead.next read(!!!)

Summaryofimpl notes

• UseGC(dropallmemorymanagementdetails)• Mergehead&tailintoasinglesentinelnode• Emptylistisjustoneobject(prev &nextontoitself)• Oneitem+=oneobject

• Reuse“removemark”objects• One-elementlistsreuseofptrs tosentinelallthetime

• Encapsulate!

SN

P1

N

PQ

ModsMorecomplexatomic operations

Basicmods(1)

• Insertitemconditionallyonprev tailvalue

SN

1N

PS

PH T

2N

P

check&bailoutbeforeCAS

Basicmods(2)

• Removeheadconditionallyonprev headvalue

SN

1N

PS

PH T

R1

check&bailoutbeforeCAS

Practicaluse-case:synchronouschannels

val channel = Channel<Int>()

// coroutine #1for (x in 1..5) {

channel.send(x * x) }

// coroutine #2repeat(5) {

println(channel.receive()) }

1

2

3

Senderswait

Sender#1H Sender#2 Sender#3 T

Moresenders

Incomingreceivers

Receiverremovesfirstifitisasendernode

Senderinsertslastifitisnotareceivernode

Receiverswait

Receiver#1H Receiver#2 Receiver#3 T

Morereceivers

Incomingsenders

Senderremovesfirstifitisareceivernode

Receiverinsertslastifitisnotasendernode

Sendfunctionsketch

fun send(element: T) {while (true) {

// try to add sender, unless prev is receiverif (enqueueSend(element)) break// try to remove first receiverval receiver = removeFirstReceiver()if (receiver != null) {

receiver.resume(element) // resume receiver break

}}

}

1

2

3

4

Channeluse-caserecap

• Usesinsert/removeopsconditionalontail/headnode• Canabort(cancel)waittoreceive/sendatanytimebyusingremove• Fullremoval-- nogarbageisleft

• Prettyefficientinpractice• Oneitemlists– one“garbage”object

Multi-wordcompareandswap(CASN)Buildevenbiggeratomicoperations

Use-case:selectexpression

val channel1 = Channel<Int>()val channel2 = Channel<Int>()

select {channel1.onReceive { e -> ... }channel2.onReceive { e -> ... }

}

Impl summary:register(1)

Selectstatus:NS

Channel1Queue

Channel2Queue

1. Notselected2. Selected


Selectstatus:NS

Channel1Queue

Channel2Queue

Addnodetochannel1queueifnotselected(NS)yet

N1


Selectstatus:NS

Channel1Queue

Channel2Queue

Addnodetochannel2queueifnotselected(NS)yet

N1 N2

Impl summary:wait

Selectstatus:NS

Channel1Queue

Channel2Queue

N1 N2

Impl summary:select(resume)

Selectstatus:S

Channel1Queue

Channel2Queue

N1

Makeselectedand removenodefromqueue

Impl summary:cleanuprest

Selectstatus:S

Channel1Queue

Channel2Queue

Removenon-selectedwaitersfromqueue

Double-CompareSingle-Swap(DCSS)BuildingblockforCASN

DCSSspecinpseudo-code

A B

fun <A,B> dcss(a: Ref<A>, expectA: A, updateA: A,b: Ref<B>, expectB: B) =

atomic { if (a.value == expectA && b.value == expectB) {

a.value = updateA}

}

1

2

34

DCSS:init descriptor

DCSSDescriptor(a,expectA,updateA,

b,expectB)

A BexpectA expectB

updateA

DCSS:prepare


b,expectB)

A B

CASptr todescriptorifa.value == expectA

expectA expectB

updateA

DCSS:readb.value


b,expectB)

A B

CASptr todescriptorifa.value == expectA

expectA expectB

updateA

DCSS:complete(whensuccess)


b,expectB)

A BexpectA expectB

updateACAStoupdatedvalueifastillpointstodescriptor

DCSS:complete(alternative)


b,expectB)

A BexpectA !expectB

updateA

DCSS:complete(whenfail)


b,expectB)

A BexpectA !expectB

updateACAStooriginalvalueifastillpointstodescriptor

DCSS:States

InitA:???

(desc created)

A:descAwasexpectA

prepok

A:???Awas!expectA

prepfail

onetread

1 2

A:updateABwasexpectB

success

A:expectABwas!expectB

4 5

fail

Anyotherthreadencounteringdescriptorhelpscomplete

Originatorcannotlearnwhatwastheoutcome

Lock-freealgorithmwithoutloops!

3

Caveats

• A&Blocationsmustbetotallyordered• orriskstack-overflowwhilehelping

• Onewaytolookatit:RestrictedDCSS(RDCSS)

DCSSMod:learnoutcome

A B

fun <A,B> dcssMod(a: Ref<A>, expectA: A, updateA: A,b: Ref<B>, expectB: B): Boolean =


a.value = updateAtrue

} else false

}

DCSSMod:init descriptor


b,expectB)

A BexpectA expectB

updateA

Outcome:UNDECIDED

Consensus

DCSSMod:prepare


b,expectB)

A BexpectA expectB

updateA

Outcome:UNDECIDED

DCSSMod:readb.value


b,expectB)

A BexpectA expectB

updateA

Outcome:UNDECIDED

DCSSMod:reachconsensus


b,expectB)

A BexpectA expectB

updateA

Outcome:SUCCESS

CAS(UNDECIDED,DECISION)

DCSSMod:complete


b,expectB)

A BexpectA expectB

updateA

Outcome:SUCCESS

DCSSMod:States

InitA:???

Outcome:UND(desc created)

A:descOutcome:UNDAwasexpectA

prepok

A:???Outcome:FAILAwas!expectA

prepfail

onetread

1 2A:desc

Outcome:SUCCBwasexpectB

success

A:descOutcome:FAIL

6

fail

A:expectA

A:updateA5

7

Stillnoloops!

3

4

Compare-And-SwapN-words(CASN)Theultimateatomicupdate

CASNspecinpseudo-code

A B

fun <A,B> cas2(a: Ref<A>, expectA: A, updateA: A,b: Ref<B>, expectB: B, updateB: B): Boolean =


a.value = updateA b.value = updateB

true} else

false}

1

2

3

4

5

Fortwowords,forsimplicity

CASN:init descriptor

DCSSDescriptor(a,expectA,updateA,b,expectB,updateB)

A BexpectA expectB

updateA

Outcome:UNDECIDED

updateB

CASN:prepare(1)


A BexpectA expectB

updateA

Outcome:UNDECIDED

updateBCAS

CASN:prepare(2)


A BexpectA expectB

updateA

Outcome:UNDECIDED

updateB

UseDCSStoupdateBifOutcome==UNDECIDED

DCSS

CASN:decide


A BexpectA expectB

updateA

Outcome:SUCCESS

updateB

CASoutcome

CASN:complete(1)


A BexpectA expectB

updateA

Outcome:SUCCESS

updateBCAS

CASN:complete(2)


A BexpectA expectB

updateA

Outcome:SUCCESS

updateBCAS

CASN:States

A:???B:???O:UND

A:descB:???O:UND

A:descB:descO:UND

A:updateAB:descO:SUCC

InitA:updateAB:updateBO:SUCC

1 2 3

5

A:descB:descO:SUCC

4

6A:???B:???O:FAIL

A!=expectA

A:descB:???O:FAIL

B !=expectB

onetread

A:expectAB:???O:FAIL

7 8

9

DCSS

PreventsfromgoingbackinthisSM

descriptorisknowntoother(helping)threads

UsingitinpracticeAllthelittlethingsthatmatter

ItiseasytocombinemultipleoperationswithDCSS/CASNthatlinearizeonaCASwithadescriptorparametersthatare

knowninadvance

Trivialexample:Treiber stack

1TOP 2

3 Newnode

CAS

expect

update

Let’sgodeeperIntounpublishedterritory

Doublylinkedlist:insertlast

OperationDescriptor

Aref:???expectA:SentinelupdateA:Node#2

…Outcome:UNDECIDED

Doublylinkedlist:insert(0)

SN

1N

PS

PH T

2N

P

CAShere

WeknowexpectedvalueforCASinadvanceWeknowupdatedvalueforCASinadvance

???canfillinAbeforeCAS&updateonretry

DCSShereisneeded(always!)


SN

1N

PS

PH T

2N

P

OperationDescriptor



DCSSDescriptor

affectednode:#1operationref


SN

1N

PS

PH T

2N

P

OperationDescriptor DCSSDescriptor

Helpersareaboundtostumbleuponthesamedescriptor

CAScanonlysucceedonlastnode

Competinginsertswillcomplete(help)usfirst





SN

1N

PS

PH T

2N

P

OperationDescriptor

desc isupdatedafter successfulDCSS

Aref:Node#1expectA:SentinelupdateA:Node#2


DCSSDescriptor



SN

1N

PS

PH T

2N

P

OperationDescriptorStayspointeduntiloperationoutcomeisdecided

Aref:Node#1expectA:SentinelupdateA:Node#2


Doublylinkedlist:removefirst

Doublylinkedlist:remove(0)

SN

1N

PS

PH T

R1

CAShere

2N

P

OperationDescriptor

Aref:???expectA:???

updateA:Rem[???]…

Outcome:UNDECIDED

BothnotknowninadvanceDeterministicf(expectA)


SN

1N

PS

PH T

R1

2N

P

OperationDescriptor

Aref:???expectA:???

updateA:Rem[???]…

Outcome:UNDECIDED

DCSSDescriptor

affectednode:#1

operationrefoldvalue:#2


SN

1N

PS

PH T

R1

2N

P

OperationDescriptor

Aref:???expectA:???

updateA:Rem[???]…

Outcome:UNDECIDED

ItlockswhatnodewearetoremoveCannotchangew/oremovalof#1Wedon’tsupportPushLeft!!!

DCSSDescriptor

affectednode:#1



SN

1N

PS

PH T

R1

2N

P

OperationDescriptor

Aref:Node#1expectA:Node#2updateA:Rem[#2]


desc isupdatedafter successfulDCSS

DCSSDescriptor

affectednode:#1



SN

1N

PS

PH T

R1

2N

P

OperationDescriptor

Aref:Node#1expectA:Node#2updateA:Rem[#2]


Stayspointeduntiloperationoutcomeisdecided

Closingnotes

• AllwecareaboutisCAS thatlinearizesoperation• Subsequentupdatesarehelpermoves• Invokeregularhelp/correctfunctions

• PerfectalgorithmtocombinewithoptionalHardwareTransactionalMemory(HTM)

Let’senjoywhatwe’veaccomplished

References

• Kotlinlanguage• http://kotlinlang.org

• Kotlincoroutinessupportlibrary• http://github.com/kotlin/kotlinx.coroutines

Thankyou

Anyquestions?

emailmetoelizarov atgmailrelizarov

Documents

2017 SPTCC - Lock-free algorithms for Kotlin coroutinesneerc.ifmo.ru/sptcc/slides/slides-elizarov.pdf · Kotlin basic facts •Kotlin is a JVM language developed by JetBrains •General