Toward Coordination-free and Reconfigurable Mixed ...Toward Coordination-free and Reconfigurable Mixed Concurrency Control Dixin Tang and Aaron J. Elmore, University of Chicago

Open access to the Proceedings of the 2018 USENIX Annual Technical Conference

is sponsored by USENIX.

Toward Coordination-free and Reconfigurable Mixed Concurrency Control

Dixin Tang and Aaron J. Elmore, University of Chicago

https://www.usenix.org/conference/atc18/presentation/tang

This paper is included in the Proceedings of the 2018 USENIX Annual Technical Conference (USENIX ATC ’18).

July 11–13, 2018 • Boston, MA, USA

ISBN 978-1-939133-02-1

Toward Coordination-free and Reconfigurable Mixed Concurrency Control

Dixin TangUniversity of Chicago

Aaron J. ElmoreUniversity of Chicago

Abstract

Recent studies show that mixing concurrency controlprotocols within a single database can significantly out-perform a single protocol. However, prior projects to mixconcurrency control either are limited to specific pairsof protocols (e.g mixing two-phase locking (2PL) andoptimistic concurrency control (OCC)) or introduce ex-tra concurrency control overhead to guarantee their gen-eral applicability, which can be a performance bottle-neck. In addition, due to unknown and shifting accesspatterns within a workload, candidate protocols shouldbe chosen dynamically in response to workload changes.This requires changing candidate protocols online with-out having to stop the whole system, which prior workdoes not fully address. To resolve these two issues, wepresent CormCC, a general mixed concurrency controlframework with no coordination overhead across candi-date protocols while supporting the ability to change aprotocol online with minimal overhead. Based on thisframework, we build a prototype main-memory multi-core database to dynamically mix three popular proto-cols. Our experiments show CormCC has significantlyhigher throughput compared with single protocols andstate-of-the-art mixed concurrency control approaches.

1 Introduction

With an increase in CPU core counts and main-memorycapacity, concurrency control has become a new bot-tleneck in multicore main-memory databases due to theelimination of disk stalls [8, 34]. New concurrency con-trol protocols and architectures focus on enabling highthroughput by fully leveraging available computation ca-pacity, while supporting ACID transactions. Some pro-tocols try to minimize the overhead of concurrency con-trol [10], while other protocols strive to avoid singlecontention points across many cores [27, 35]. How-ever, these single protocols are typically designed for

specific workloads, may only exhibit high performanceunder their optimized scenarios, and have poor perfor-mance in others. Consider H-Store’s concurrency con-trol protocol that uses coarse-grained exclusive partitionlocks and a simple single-threaded executor per parti-tion [10]. This approach is ideal for partitionable work-loads that have tuples partitioned such that a transactionis highly likely to access only one partition. But thisapproach suffers decreasing throughput with increasingcross-partition transactions [34, 27, 20]. Here, an opti-mistic protocol may be preferred if the workload mainlyconsists of read operations or a pessimistic protocol maybe ideal if the workload exhibits high conflicts. An ap-pealing solution to this tension is to combine differentprotocols such that each protocol can be used to processa part of workload that they are optimized for, and avoidbeing brittle to scenarios where single protocols suffer.

Efficiently mixing multiple concurrency control pro-tocols is challenging in several ways. First, it should notbe limited to a specific set of protocols (e.g. OCC and2PL [22, 28]), but be able to extend to new protocolswith reasonable assumptions. Second, the overhead ofmixing multiple protocols should be minimized such thatthe overhead is not a performance bottleneck in any sce-nario. A robust design should ensure that in any case themixed execution does not perform worse than any singlecandidate protocol involved. Finally, as many transac-tional databases back user-facing applications, dynami-cally switching protocols online in response to workloadchanges is necessary to maintain performance.

While several recent studies focus on mixed concur-rency control, they only address a part of the above chal-lenges. For example, MOCC [28] and HSync [22] are de-signed to mix OCC and 2PL with minimal mixing over-head but fails to extend to other protocols; Callas [31]and Tebaldi [23] provide a general framework that cancover a large number of protocols, but their overhead ofmixing candidate protocols is non-trivial in some scenar-ios and do not support online protocol switch thus failing

USENIX Association 2018 USENIX Annual Technical Conference 809

to address the workload changes. Our prior work [25]envisions an adaptive database that mixes concurrencycontrol, but does not includes a general framework to ad-dress these challenges.

In this paper, we present a general mixed concur-rency control scheme CormCC (Coordination-free andReconfigurable Mixed Concurrency Control) to sys-tematically address all the aforementioned challenges.CormCC decomposes a database into partitions accord-ing to workload access patterns and assigns a specificprotocol to each partition, such that a protocol can beused to process the parts of a workload they are opti-mized for. We then develop several criteria to regulatethe mixed execution of multiple forms of concurrencycontrol to maintain ACID properties. We show that un-der reasonable assumptions, this method allows correctmixed execution without coordination across differentprotocols, which minimizes the mixing overhead. In ad-dition, we develop a general protocol switching method(i.e. reconfiguration) to support changing protocols on-line with multiple protocols running together; the keyidea is to compose a mediated protocol compatible withboth the old and new protocol such that switch processdoes not have to stop all transaction workers while mini-mizing the impact on throughput and latency. To validatethe efficiency and effectiveness of CormCC, we developa prototype main-memory database on multi-core sys-tems that supports mixed execution and dynamic switch-ing of three widely-used protocols: a single-threadedpartition based concurrency control (PartCC) from H-Store [10], an optimistic concurrency control (OCC)based on Silo [27], and a two-phase locking based onVLL (2PL) [21]. 1

The main contributions of this paper over our visionpaper [25] are the following:

• A detailed analysis of mixed concurrency controldesign space, a general framework that is not limitedto a specific set of protocols to mix multiple formsof concurrency control without introducing extra over-head of coordinating conflicts across protocols.

• A general protocol switching method to reconfigurea protocol for parts of a workload without stopping thesystem or introducing significant overhead.

• A thorough evaluation of state-of-the-art mixedconcurrency control approaches, CormCC’s end-to-end performance over varied workloads, and the per-formance benefits and overhead of CormCC’s mixedexecution and online reconfiguration.

1Note that we use strong 2PL (SS2PL), but refer to it as 2PL

2 Related Work

With the increase of main memory capacity and the num-ber of cores for a single node, recent research focuseson improving traditional concurrency control protocolson modern hardware. We classify these works into opti-mizing single protocols, mixing multiple protocols, andadaptable concurrency control.

Optimizing Single Protocols Many recent projectsconsider optimizing a single protocol, which we believeare orthogonal to this project. H-Store [8, 10] developeda partitioned based concurrency control (PartCC) to min-imize concurrency control overhead. The basic idea isto divide databases into disjoint partitions, where eachpartition is protected by a lock and managed by a sin-gle thread. A transaction can execute when it has ac-quired the locks of all partitions it needs to access. Ex-ploiting data partitioning and the single-threaded modelis also adopted by several research systems [8, 10, 32, 18,19, 11]. Other projects propose new concurrency con-trols optimized for multi-core main-memory databases[6, 14, 27, 17, 35, 20, 13, 29, 30, 36, 12, 37, 16] or newdata structures to remove the bottlenecks of traditionalconcurrency control [34, 21, 15, 9].

Mixing Multiple Protocols Callas [31] and its succes-sive work Tebaldi [23] provide a modular concurrencycontrol mechanism to group stored procedures and pro-vide an optimized concurrency control protocol for eachgroup based on offline workload analysis; for transac-tion conflicts across groups, Callas and Tebaldi intro-duce one or multiple protocols in addition to protocolsfor each group to resolve them. While stored proce-dure oriented protocol assignment can process conflictswithin the same group more efficiently, the additionalconcurrency control overhead from executing both in-group and cross-group protocols can become the per-formance bottleneck for a main-memory database on amulti-core server. Section 3 shows a detailed discussion.In addition, the grouping based on offline analysis as-sumes workload conflicts are known upfront, which maynot be true in real applications. Our work differs fromCallas and Tebaldi in that our mixed concurrent controlexecution does not introduce any extra concurrency con-trol overhead, which greatly reduces the mixing over-head. Additionally, we do not assume a static work-load or require any knowledge about workload conflictsin advance, but allow protocols are chosen and reconfig-ured online in response to dynamic workloads. Someother projects exploit the mixed execution of 2PL orOCC [33, 6, 28, 22]. CormCC differs in that our frame-work is more general and can be extended to more pro-tocols.

Adaptable Concurrency Control Adaptable concur-rency control has been studied in several research works.

810 2018 USENIX Annual Technical Conference USENIX Association

1) One Protocol per Record/Transaction 2) One Protocol per Record, Multiple Protocols Per Transaction 3) Multiple Protocols per Record, One Protocol Per Transaction 4) Multiple Protocols per Record/Transaction

Figure 1: Design choices of mixed concurrency control

At the hardware level, ProteusTM [7] is proposed toadaptively switch across multiple transaction memory al-gorithms for different workloads, but cannot mix themto process different parts of a workload. Tai et al. [24]shows the benefits of adaptively switching between OCCand 2PL. RAID [4, 3] proposes a general way to changea single protocol online. CormCC differs in that it al-lows multiple protocols running in the same system andsupports to reconfigure a protocol for parts of workload.In this scenario, the presence of multiple protocols dur-ing the reconfiguration presents new challenges we areaddressing.

3 Design Choices

While combining multiple protocols into a single sys-tem can potentially allow more concurrency to improvethe overall throughput, it comes with the cost of higherconcurrency control overhead. In this section, we dis-cuss two key design choices to explore this trade-offand show how this trade-off motivates the design ofCormCC. Specifically, we consider whether a record canbe accessed (i.e. read/write) by multiple protocols andwhether a single transaction involves multiple protocols.Based on the design choices, the overhead can be de-fined as the cost of executing more than one protocol foreach transaction plus the cost of synchronizing the con-current read/write operations across different protocolsfor each record in the database. Note that in this paper,we assume all the transaction logic and the correspond-ing concurrency control logic of a transaction are exe-cuted by a single thread (i.e. transaction worker), whichis a common model in the design of mixed concurrencycontrol [31, 23, 28, 22, 6]. We now discuss the four pos-sible designs that are shown in Figure 1.

One Protocol per Record and per Transaction Thisis the simplest case, which is the left most part of Fig-ure 1. We see that each transaction (denoted by T) canonly choose one protocol (i.e. CC in Figure 1) and eachrecord (denoted by R) can be accessed via one proto-col. While the mixing overhead is minimal (based onour overhead definition), this design has very limited ap-plicability since each transaction can only access the par-tition of records managed by a specific protocol. To thebest of our knowledge, no previous work adopts it.

One Protocol per Record, Multiple Protocols perTransaction This design further allows that one trans-

action executes multiple protocols (shown in the seconddesign in Figure 1); it provides the flexibility that trans-actions can access any record and allows a specific proto-col to process all access to each record according to theiraccess patterns. On the other hand, the execution of asingle transaction may involve a larger set of instructionsfrom multiple protocols, which makes CPU instructioncache less efficient. However, according to our exper-iment in Section 6.5 this mixing overhead is very low.MOCC [28] adopts this design to mix OCC and 2PL, butdoes not have a general framework.

Multiple Protocols per Record, One Protocol perTransaction An alternative design (the third one in Fig-ure 1) is that each record can be accessed via multi-ple protocols and each transaction executes one protocol.This design is useful when the semantics of a subset oftransactions (e.g. from stored procedures) can be lever-aged by an optimized protocol. It raises a problem, how-ever, that co-existence of multiple protocols on the sameset of records should be carefully synchronized. One so-lution is to let all protocols share the same set of concur-rency control metadata and carefully design each proto-col such that all concurrent access to the same recordscan be synchronized without introducing additional co-ordination. This solution requires specialized design forall protocols and thus is limited in its applicability. Priorworks Hekaton [6] and HSync [22] adopt this design, butcan only combine 2PL and OCC.

Multiple Protocols per Record and per TransactionThis design (the fourth one in Figure 1) provides themost fine-grained and flexible mixed concurrency con-trol; each transaction can mix multiple protocols andeach record can be accessed via different protocols. Toprocess the concurrent access from different protocolsover the same records, additional protocols are intro-duced. For example, Callas [31] and its successive workTebaldi [23] organized protocols into a tree, where theprotocols in leaf nodes process conflicts of the assignedtransactions and the protocols in interior nodes processconflicts across its children. The overhead here is that foreach record access, multiple concurrency control logicshould be executed to resolve the conflicts across dif-ferent protocols over that record. Such overhead, as weshow in Section 6.3, can become a performance bottle-neck in the multi-core main-memory database.

Summary The above discussion (and our experi-ments) show that the second design, which lets each pro-


Preprocess Execution Validation Commit

CC1

CC2

CC3

OP1 OP2 OPn

Figure 2: An Example of CormCC execution

tocol exclusively process the access of a subset of recordsto minimize synchronization cost and allows protocolsare mixed within a transaction to provide mixing flex-ibility, strikes a good trade-off between leveraging theperformance benefits of single protocols and minimizingmixing overhead. CormCC draws its spirit and builds ageneral and coordination-free framework.

4 CormCC Design

We consider a main-memory database with multipleforms of concurrency control on a multi-core machine.Each table includes one primary index and zero or moresecondary indices. A transaction can be composed intoread, write (i.e. update), delete, and insert operations toaccess the database via either primary or secondary in-dices in a key-value way. The database is (logically)partitioned with respect to candidate protocols within thesystem, that is, each partition is assigned a single proto-col. Each protocol maintains an independent set of meta-data for all records. For all operations on the records ofa partition, the associated protocol executes its own con-currency control logic to process these operations (e.g.preprocess, reading/writing, commit), which we denoteas a protocol managing this partition. We use a con-currency control lookup table to store the mapping fromprimary keys to the protocol. The lookup table maintainsthe mapping for the whole key space and is shared by alltransaction workers. Prior work [26] shows that such alookup table can be implemented in a memory-efficientand fast way, and thus will not be the performance bottle-neck of CormCC. CormCC regards secondary indices aslogically additional tables storing entries to primary keys(not pointers to records); it adopts a dedicated protocol(e.g. OCC) to process all concurrent operations over sec-ondary indices. Transactions are routed to a global poolof transaction workers, each of which is a thread or a pro-cess occupying one physical core. This worker executesthe transaction to the end (i.e. commit or abort) with-out interruption. We additionally use a coordinator tomanage the online protocol reconfiguration for all trans-action workers. It collects statistical information periodi-cally from all workers, builds a new lookup table accord-ingly, and finally lets all workers use it. We first outlinethe CormCC protocol and then show the correctness ofCormCC. After that, we discuss online protocol recon-figuration within CormCC.

4.1 CormCC Protocol

CormCC divides a transaction’s life cycle into fourphases: Preprocess, Execute transaction logic (Execu-tion), Validation, and Commit. We adopt this four-phasemodel because most concurrency control protocols canfit into it. We use a transaction execution example inFigure 2 to explain the four phases.

Preprocess The preprocess phase executes concur-rency control logic that should be executed before thetransaction logic. Figure 2 shows that CormCC iter-ates over all candidate protocols (denoted as CC) andexecutes their preprocess phases respectively. Typicalpreprocess phase includes initializing protocol-specificmetadata. For example, 2PL Wait-die needs to acquire atransaction timestamp to determine the relative order toconcurrent transactions, or partition-based single-threadprotocol (e.g. PartCC) acquires locks in a predefined or-der for partitions the transaction needs to access.

Execution Transaction logic is executed in this phase.As shown in Figure 2, for each operation (denoted as OP)issued from the transaction, CormCC first finds the pro-tocol managing the record using the concurrency controllookup table. Then, CormCC utilizes the protocol’s con-currency control logic to process this operation. For ex-ample, if the transaction reads an attribute of a recordmanaged by 2PL, it acquires its read lock and returns theattribute’s value. Note that insert operations can find thecorresponding protocol in the lookup table even thoughthe record to be inserted is not in the database becausethe table stores the mapping of the whole key space.

Validation Each validation phase of all protocols areexecuted sequentially in this phase. If the transac-tion passes all validation, it enters the Commit phase;otherwise, CormCC aborts it. For example, if OCCis involved, CormCC executes its validation to verifywhether records read by OCC during Execution havebeen modified by other transactions. If yes, the trans-action is aborted; otherwise, it passes this validation.

Commit Finally, CormCC begins an atomic Commitphase by executing commit phases of all protocols. Forexample, one typical commit phase, like OCC, will ap-ply all writes to the database and make them visible toother transactions via releasing all locks. Note that anabort can happen in the Execution or Validation phase,in which case CormCC calls the abort functions of allprotocols respectively.

4.2 Correctness

We first show the criteria for CormCC to generate se-rializable schedule for a set of concurrent transactions(i.e. its result is equivalent to some serial history of thesetransactions) to guarantee the consistency of databases.


We then discuss how CormCC avoids deadlock, andshow that CormCC is recoverable (i.e. any committedtransaction has not read data written by an aborted trans-action).

Serializability To guarantee that CormCC is serializ-able, we require all candidate protocols are commit or-dering conflict serializable (COCSR). It means that iftwo transactions ti and t j have conflicts on a record r (i.e.ti and t j reads/writes r and at least one of them is a write)and t j depends on ti (i.e. ti accesses r before t j), then timust be committed earlier than t j.

Given that all candidate protocols are COCSR, wenow show that CormCC is also COCSR. Suppose thattwo transactions ti and t j have conflicts on a record setR, we consider two cases. First, if for any conflictedrecord r ∈ R the conflicted operation of ti accesses r be-fore t j, then we have t j depends on ti. Since any can-didate protocol is COCSR, for each conflicted record rthey ensure that ti is committed before t j. Therefore inthis case, CormCC can maintain COCSR property. Inthe second case, there exist two records r1 and r2 ∈ R,and ti accesses r1 before t j and t j accesses r2 before ti.We have t j depends on ti and ti depends on t j. In thiscase, r1 and r2 must be managed by two separate pro-tocols p1 and p2 since each single protocol is COCSRand it is not possible to form a conflict-cycle in one pro-tocol. Consider p1, which manages r1. Because it isCOCSR, it enforces that ti is committed earlier than t jsince t j depends on ti based on their conflicts on r1. Onthe other hand, p2 enforces that ti is committed earlierthan t j. Therefore, there is no valid commit time for bothti and t j to suffice the above two constraints. Thus, inthis case committing both transactions is impossible andthe COCSR property of CormCC is also maintained. Fi-nally, since COCSR is a sufficient condition for conflictserializable [2], CormCC is conflict serializable.

Preprocess Execution Validation

CC2

T1

T2

T1

T2

CC1

CC2

(1) (2)

CC1

Figure 3: Examples of deadlock across protocols

Deadlock Avoidance While each individual protocolcan provide mechanisms to avoid or detect deadlocks,mixing them using CormCC without extra regulationmay not make the system deadlock-free. One potentialsolution can be using a global deadlock detection mech-anism. However, this contradicts our spirit of not coordi-nating candidate protocols.

Alternatively, we examine the causes of deadlock andfind that under reasonable assumptions, CormCC canmix single protocols without coordination. Specifically,

the deadlock can happen within a single phase or acrossphases when two transactions wait for each other due totheir conflicts on records that are managed by separateprotocols. Figure 3 shows two such cases. The first caseshows that the single-phase deadlock happens becauseboth protocols CC1 and CC2 can make transactions T1and T2 wait within one phase. For the second case, we seethat T1 waits for T2 in the Execution phase due to CC1 andadditionally introduce conflicts to make T2 wait for itselfbased on CC2 in the Validation phase. Such deadlockis possible because a protocol (e.g. CC2) can introduceconflicts across phases, that is, the conflicts introducedby T1 in Execution are detected by T2 in Validation.

CormCC avoids the two cases by requiring each pro-tocol make transactions wait due to conflicts in no morethan one phase, and in each phase only one protocolmake transactions wait because of conflicts. If CormCCcan meet the two criteria and each protocol can avoid ordetect deadlocks, then CormCC is deadlock-free.

Recoverable To guarantee CormCC recoverable, werequire that each candidate protocol is strict, whichmeans that a record modified by a transaction is not vis-ible to concurrent transactions until the transaction com-mits. This ensures transactions never read dirty writes(i.e. uncommited writes) and thus are recoverable [2].Since each record written by a transaction is managedby a single strict protocol, the execution of CormCC willnever read dirty writes and is recoverable.

Supported Protocols A candidate protocol incorpo-rated in CormCC should be COCSR and strict. We find awide range of protocols can meet these criteria includingtraditional 2PL and OCC [2], VLL [21], Orthrus [20],PartCC from H-Store [10], and Silo [27].

To enforce that CormCC meets the criteria ofdeadlock-free, CormCC requires each protocol specifiesthe phases where it makes transactions wait. Based onthese specification, it is easy to detect whether the abovecriteria hold for a given set of protocols.

4.3 Online Reconfiguration

Online reconfiguration is to switch a protocol for a sub-set of records without stopping all transaction workers.CormCC uses a coordinator to manage this process. Atfirst, the coordinator collects statistical information fromall workers periodically and generates a new concurrencycontrol lookup table. When a worker completes a trans-action, it adopts the new lookup table while workers thathave not finished their current transactions still use theold one. After all workers use the new table, the old oneis deleted. We now introduce the challenge of supportingsuch online protocol switch.

Challenge The potential problem is that some transac-tion workers may use the new lookup table while others


T2

R1

Read

OCC

Write

SwitchW1 T1 T3 Write

2PL

W2

OCC

Validate CommitRead

Read

Figure 4: Problems during protocol reconfiguration

are using the old one. Figure 4 shows an example of thisproblem. Consider two workers W1 and W2, which ex-ecute transactions T1 and T2 respectively. Assume thatboth T1 and T2 access a record (i.e. R1) that is man-aged by OCC. During their execution, the coordinator in-forms that the protocol managing R1 should be switchedto 2PL. Therefore, after T1 finishes W1 checks this mes-sage, performs the switch (i.e. using 2PL to access R1),and starts a new transaction (T3). Since OCC and 2PLmaintain a different set of metadata, T3 and T2 are notaware of the conflict of T2 reading R1 and T3 writing R1,which may make database result in an inconsistent state.

Mediated Switching To address this issue, we pro-pose mediated switching; it adopts a mediated protocolthat is compatible with both old and new protocols. Dur-ing reconfiguration, the coordinator lets all workers asyn-chronously change the old protocol to the mediated one;After all workers use the mediated protocol, the coordi-nator then informs them to adopt the new protocol.

We compose a mediated protocol that can execute con-currency control logic of both old and new protocols.Specifically, a mediated protocol first executes the Pre-process logic of both old and new protocols; then, itenters the Execution phase, where for each record ac-cess the mediated protocol executes the Execution logicof both protocols. The mediated protocol’s Validation,Commit, and Abort also executes the correspondinglogic of both protocols. While it is easy to compose aprotocol that executes the logic of both protocols, oneproblem is how to unify different ways of applying mod-ifications (i.e. insert/delete/write) of different protocols.We find there are two ways to apply modifications: in-place modification during execution and lazy modifica-tion during commit phase. In the mediated protocol, wealways opt for lazy modification, which means storingthe modification in a local buffer during execution andapplying them when the transaction is committed. Sinceall protocols are strict, deferring the actual modificationto the commit phase does not violate correctness of pro-tocols. For example, 2PL performs in-place write, whileOCC writes the new value into a local buffer and appliesit in the commit phase. In our mediated protocol, wechoose the OCC approach of applying writes.

We use the example in Figure 5 to illustrate this pro-cess, where we need to switch the protocol managing R1from OCC to 2PL. The mediated protocol here will exe-cute the logic of both OCC and 2PL (denoted as OPCC).OPCC adopts the following logic:

T1 Switch

Switch

T2

R1 OCC

W1

T3

OPCC

W2

OCC

T4

Time

Upgrade

SwitchR1

2PL

W1

2PL

W2

OPCC

SwitchT3

T5

Time

Degrade

T6 T7

Figure 5: An Example of mediated switching

• If OPCC reads a record, it applies a read lock (2PL)and reads its value and timestamp into the read set(OCC).

• If OPCC writes a record, it applies a write lock(2PL) and stores the record along with new data intothe write set (OCC).

• In the validation phase, it locks all records in writeset (e.g. for critical section of Silo OCC)2 and thenvalidate the read set using OCC logic.

• In the commit phase, it applies all writes and releaselocks acquired by OCC and 2PL respectively.The switch via mediated protocol is composed of two

phases: upgrade and degrade. The protocol switch isinitiated when the coordinator finds that the protocol fora record set RS should be changed. It starts the up-grade phase by issuing a message to all workers to letthem switch the protocol for RS to the mediated proto-col. Each transaction worker checks for this message be-tween transactions and acknowledges the message to thecoordinator. During this asynchronous process, workersthat have received the message access RS using the medi-ated protocol, while other workers may access RS usingthe old protocol, which happens when they are runningtransactions that started before the switch. The left partof Figure 5 shows an example of upgrade phase. We seethat worker W2 finished T2 first; thus, it begins to accessR1 using OPCC (i.e. in transaction T3). At the sametime, T1 still accesses R1 using OCC. According to theabove OPCC description, we see that the conflicts on R1from W1 and W2 can be serialized because OPCC runsthe full logic of OCC. After all workers acknowledge theswitch for RS, the degrade phase begins with the coor-dinator messaging workers about the degrade to the newprotocol. Therefore, workers are using either the me-diated protocol or the new protocol, and serializability isguaranteed by the new protocol logic (e.g. 2PL in our ex-ample) used in both execution modes. The right part ofFigure 5 shows an example of degrade phase, where theconflicts over R1 can be serialized because OPCC alsoexecutes the full logic of 2PL.

2Note that OCC and 2PL use different sets of metadata; no deadlockcan exist between them


5 Prototype Design

We build a prototype main-memory multi-core databasethat can dynamically mix PartCC from H-Store [10],OCC from Silo [27], and 2PL from VLL [21] usingCormCC. PartCC partitions the database and associateseach partition an exclusive lock. Every transaction firstacquires all locks for the partitions it needs to read/writein a predefined order before the transaction logic is ex-ecuted. Then, the transaction is executed by a singlethread to the end without additional coordination. In SiloOCC, each record is assigned a timestamp. During trans-action execution, Silo OCC tracks read/write operations,and stores the records read by the transactions along withtheir associated timestamps into a local read set and allwrites into a local write set. In the validation phase, SiloOCC locks the write set and validates whether records inread set are changed using their timestamps. If the val-idation succeeds, it commits; otherwise, it aborts. VLLis an optimized 2PL by co-locating each lock with eachrecord to remove the contention of the centralized lockmanager.

Our prototype partitions primary indices and corre-sponding records using an existing partitioning algo-rithm [5]. Each partition is assigned with a transactionworker (i.e. thread or process) and each worker is onlyassigned transactions that will access some data in itspartition (the base partition), but may also access data inother partitions (the remote partition). CormCC selects aprotocol for each partition according to its access pattern.A partition routing table maintains the mapping from pri-mary keys to partition numbers and also correspondingprotocols. We use stored procedures as the primary in-terface, which is a set of predefined and parameterizedSQL statements. Stored procedures can provide a quickmapping from database operations to the correspondingpartitions by annotating the parameters that can be usedto identify a base partition to execute the transaction andother involved partitions [10].

The mixed execution of the three protocols start withPreprocess phase, where PartCC acquires all partitionlocks in a predefined order. Transactions may wait inthis phase because of partition lock requests. Next, trans-actions enter Execution, where CormCC uses PartCC,OCC, and 2PL to process record access operations ac-cording to which partition the record belongs to. Notethat only 2PL will make transactions wait in this phase,so transactions in the Execution phase will not wait forthose blocked in the Preprocess, which indicates dead-locks across PartCC and 2PL is impossible. After theExecution phase, Validation begins and OCC acquireswrite locks and validates the read set [27]. Only OCC re-quires transactions to wait; thus, they will not be blockedby previous phases. Finally, there is no wait in commit

phase; all protocols apply writes and release all locks.We show that such mixed execution is correct. First, forPartCC and 2PL if a transaction ti conflicts with anothertransaction t j, ti cannot proceed until t j commits or viceversa, so PartCC and 2PL are COCSR. Shang et al. [22]have proven that Silo OCC is also COCSR. Therefore,their mixed execution using CormCC maintains COCSR.In addition, each of PartCC, 2PL, and OCC can indepen-dently either avoid or detect deadlocks and make transac-tions conflict-wait in only one mutually exclusive phaseamong Preprocess, Execution, or Validation. Thus, theirmixed execution can also prevent or detect deadlocks. Fi-nally, all protocols are strict, so is their mixed execution.

To enable dynamic protocol reconfiguration, we buildtwo binary classifiers to predict the ideal protocol foreach partition. The detailed discussion of classifiers arepresented in a technical report [1].

6 Experiments

We now evaluate the effectiveness of mixed executionand online reconfiguration of CormCC. Our experimentsanswer four questions: 1) How does CormCC performcompared to state-of-the-art mixed concurrency controlapproaches (Section 6.3)? 2) How does CormCC adap-tively mix protocols under varied workloads over time(Section 6.4)? 3) What is the performance benefit andoverhead of mixed execution (Section 6.5)? 4) What isthe performance benefit and overhead of online reconfig-uration (Section 6.6)?

All experiments are run on a single server with fourNUMA nodes, each of which has a 8-core Intel XeonE7-4830 processor (2.13 GHz), 64 GB of DRAM and24 MB of shared L3 cache, yielding 32 physical coresand 256 GB of DRAM in total. Each core has a pri-vate 32 KB of L1 cache and 256 KB of L2 cache. Wedisable hyperthreading such that each worker occupies aphysical core. To eliminate network client latency, eachworker combines a client transaction generator.

6.1 Prototype ImplementationWe develop a prototype based on Doppel [17], an open-source multi-core main-memory transactional database.Clients issue transaction requests using pre-definedstored procedures, where all parameters are providedwhen a transaction begins, and transactions are exe-cuted to the completion without interacting with clients.Stored procedures issue read/write operations using in-terfaces provided by the prototype. Each transaction isdispatched to a worker that runs this transaction to theend (commit or abort).

Workers access records via key-value hash tables.Each worker thread occupies a physical core and main-


SSI

2PL

RL RL

PM NO DEL

OS SL

Figure 6: A three-layer configuration of Tebaldi

0.4

0.6

0.8

1

1.2

0 4 8 12 16 20 24 28 32

Thro

ught

put (

Mill

ions

txn/

s)

Number of non-partitionable partitions

2PLTebaldi

CormCC

Figure 7: Comparison under different partitionability

0.4

0.6

0.8

1

1.2

0 0.3 0.6 0.9 1.2 1.5

Thro

ught

put (

Mill

ions

txn/

s)

Theta of Zipf

2PLTebaldi

CormCC

Figure 8: Comparison unber different conflicts

tains its own memory pool to avoid memory allocationcontention across many cores [34]. A coordinator threadis used for extracting features for the prediction classi-fiers from statistics collected by workers and predictingthe ideal protocol to be used. Our prototype supportsautomatically selecting PartCC [10], Silo OCC [27],or No-Wait VLL [21] for each partition. Note that wehave compared 2PL variants No-Wait and Wait-Die, andfind that 2PL No-Wait performs best in most cases be-cause of lower synchronization overhead of lock man-agement [20]. We do not implement logging in CormCCsince prior work shows that logging is not a performancebottleneck [17, 38]. Our comparison additionally in-cludes a general mixed concurrency control frameworkbased on Tebaldi [23] (denoted as Tebaldi) and a hybridapproach of OCC and 2PL [22, 28] (denoted as Hybrid),that adopts locks to protect highly conflicted records butuses validation for the rest. We statically tune the set ofhighly conflicted records to make Hybrid have the high-est throughput. Specifically, for our highly conflictedworkload we protect 1000 mostly-conflicted records foreach partition and for our lowly conflicted workload norecords will be locked and we use OCC for them.

6.2 Benchmarks & Experiment Settings

We use YCSB and TPC-C in our experiments. Wegenerate one table for YCSB that includes 10 millionrecords, each with 25 columns and 20 bytes for eachcolumn. Transactions are composed of mixed read andread-modify-write operations. The partitioning of YCSBis based on hashing its primary keys. TPC-C simulatesan order processing application. We generate 32 ware-houses and partition the store according to warehouseIDs except the Item table, which is shared by all workers.We use the full mix of five procedures.

To generate varied workloads, we tune three parame-ters. The first is the percentage of cross-partition trans-actions ranging from 0 to 100. We set the number ofpartitions a cross-partition transaction will access as 2;The second is the mix of stored procedures for a work-load. Note that YCSB only has one stored procedure,and we tune the number of operations per transaction andthe ratio of read operations. Finally, we vary data accessskewness. We use Zipf to generate record access distri-

bution within a partition. T heta of Zipf can be variedfrom 0 to 1.5. This means for TPC-C within a partitiondetermined by WarehouseID, we skew record access forrelated tables. We also vary these parameters to train ourclassifiers for protocol prediction. The detailed configu-ration of training classifiers is illustrated in [1].

6.3 Comparison with Tebaldi

Tebaldi [23] is a general mixed concurrency controlframework that groups stored procedures according totheir conflicts and build a hierarchical concurrency con-trol protocols to address in-group and cross-group con-flicts. Figure 6 shows a three-layer configuration forTPC-C. We see that NewOrder (NO) and Payment (PM)are in the same group and their conflicts are managed byruntime pipeline (RL). Runtime pipeline is an optimized2PL that can leverage the semantics of stored proceduresto pipeline conflicted transactions. Delivery (DEL) aloneis in another group also run by runtime pipeline. Con-flicts between the two groups are processed by 2PL. Or-derStatus (OS) and StockLevel (SL) are executed in a sep-arate group. Their conflicts with the rest of the groupsare processed by Serializable Snapshot Isolation (SSI).For every operation issued by a transaction, it needs toexecute all concurrency control logic from the root nodeto the leaf node to delegate the conflicts to specific pro-tocols. For example, any operation issued by NewOrderneeds to go through the logic of SSI, 2PL, and RL. Whilethis approach can process conflicts in a more fine-grainedway, the overhead of multiple protocols for all opera-tions can limit the performance. Another restriction ofTebaldi is that it relies on the semantics of stored proce-dures to assign protocols. For workloads including datadependent behaviour (e.g. hot keys or affinity betweenkeys) and not having stored procedures with rich seman-tics (i.e. YCSB in our test), Tebaldi can only use oneprotocol to process the whole workload. In our test, weuse this 3-layer configuration for Tebaldi, which has thebest performance for TPC-C [23]. Note that we use anoptimized Runtime Pipeline reported in [31], which caneliminate the conflicts between Payment and NewOrder.

We first compare CormCC with Tebaldi, imple-mented in our prototype, using TPC-C over mixing well-partitionable and non-partitionable workloads. We par-


0 0.5

1 1.5

2 2.5

3 3.5

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

Thro

ught

put (

Mill

ions

txn/

s)

Time (s)

PartCCOCC2PL

HybridCormCC

Figure 9: Holistic test for CormCC under YCSB varied workloads over time

0

0.5

1

1.5

2

2.5

3

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

Thro

ught

put (

Mill

ions

txn/

s)

Time (s)

PartCCOCC2PL

HybridCormCC

Figure 10: Holistic test for CormCC under TPC-C varied workloads over time

0.5

1

1.5

2

2.5

0 25 50 75 100

Thro

ughp

ut R

atio

to S

ingl

e P

roto

cols

Time(s)

Max Avg Min

Figure 11: CormCC throughput ratio to single protocols for YCSB

0.5

1

1.5

2

2.5

0 25 50 75 100

Thro

ughp

ut R

atio

to S

ingl

e P

roto

cols

Time(s)

Max Avg Min

Figure 12: CormCC throughput ratio to single protocols for TPC-C

tition the database into 32 warehouses and start our testwith a well-partitionable workload (i.e. each partitionreceives 100% single-partition transactions), and then in-crease the number of non-partitionable warehouses (i.e.each receives 100% cross-partition transactions) by aninterval of 4. Throughout this test, we use default trans-action mix of TPC-C. Since both Tebaldi and CormCCuse 2PL as their candidate protocol, our test additionallyincludes the results of 2PL.

Figure 7 shows the performance results of three proto-cols. CormCC first adopts PartCC and then proceeds tomix PartCC and 2PL for workloads with mixed partition-ability. When the workload becomes non-partitionable,2PL is used by CormCC. We see that CormCC alwaysperforms better than Tabaldi and 2PL because it canleverage the partitionable workloads. Tebaldi alwaysperforms slightly worse than 2PL because of its concur-rency control overhead from multiple protocols. Whilesuch overhead is not substantial in a distributed environ-ment as shown in the original paper [23], it can becomea bottleneck in a main-memory multi-core database dueto the elimination of network I/O operations.

To highlight Tebaldi’s performance benefits of effi-ciently processing conflicts, we increase the access skew-ness within each warehouse by varying the theta of Zipfdistribution from 0 to 1.5 with an interval 0.3. Here,we choose 16 warehouses as partitionable and the restas non-partitionable. Figure 8 shows that the throughput

of all protocols increases at first, because more accessskewness introduces better access locality and improvesCPU cache efficiency. Then, high conflicts dominate theperformance and the throughput decreases for all proto-cols. We see that with higher conflicts Tebaldi graduallyoutperforms 2PL, and suffers less throughput loss in theworkload with very high conflicts.

These tests show that while Tebaldi can efficiently pro-cess conflicts, it comes with a non-trivial concurrencycontrol overhead. In addition, Tebaldi needs to know theconflicts of a workload a priori such that it can utilizethe static analysis [23] to make an efficient configurationoffline. In contrast, CormCC mixes protocols with mini-mal overhead, requires no knowledge of conflicts before-hand, and can dynamically choose protocols online.

6.4 Tests on Varied Workloads

We evaluate the holistic benefits of CormCC by runningYCSB with randomizing benchmark parameters every 5seconds. We compare the same randomized run (e.g.same parameters at each interval) with fixed protocolsand Hybrid. CormCC collects features every second andconcurrently selects the ideal protocol for each partition.

We randomly vary five parameters: i) read rate chosenin 50%, 80%, and 100%; ii) number of operations pertransaction, selected between 15 and 25; iii) theta of Zipffor data access distribution; chosen from 0, 0.5, 1, and


0

1

2

3

4

5

0 4 8 12 16 20 24 28 32

Thro

ught

put (

Mill

ions

txn/

s)

Number of non-partitionable partitions

PartCCOCC2PL

HybridCormCC

Figure 13: Measuring partitionability

0

1

2

3

4

0 4 8 12 16 20 24 28 32

Thro

ught

put (

Mill

ions

txn/

s)

Number of read-only partitions

PartCCOCC2PL

HybridCormCC

Figure 14: Measuring read/write ratio

0

10

20

30

40

50

1/2 PartCC 1/2 OCC

1/2 PartCC 1/2 2PL

1/2 OCC 1/2 2PL

1/3 PartCC 1/3 OCC 1/3 2PL

PartCC OCC 2PL

Thro

ug

hp

ut

(Thousa

nds

txn/s

/core

)

Single Protocols Mixed Protocols

Figure 15: Testing the overhead of mixed execution

1.5; iv) the number of partitions that have cross-partitiontransactions (with the remaining partitions as well parti-tionable): we randomly choose the number from 0 to 32with the interval 4; v) the percentage of cross-partitiontransactions for partitions in (iv), randomly selected be-tween 50% and 100%.

The test starts with a well-partitionable workload of80% read rate, 15 operations per transaction, and a uni-form access distribution (i.e. theta = 0). Figure 9 showsthe test results of every 2 seconds for 100 seconds in to-tal. We see that in almost all cases CormCC can eitherchoose the best protocol or find a mixed execution to out-perform any candidate protocols and Hybrid approach,while not experiencing long periods of throughput degra-dation due to switching. CormCC can achieve at most2.5x, 1.9x, 1.8x, and 1.7x throughput of PartCC, OCC,2PL, and Hybrid respectively.

We additionally test the performance variations ofCormCC under randomized varied workloads of TPC-C. We partition the database into 32 warehouses, andhave each worker collect features every second and se-lect the ideal protocol for each warehouse at runtime. Wepermute four parameters to generate varied workloads.First, we randomly select a transaction mix in 10 can-didates, where one is default transaction mix of TPC-Cand other nine are randomly generated. Then, we varythe three parameters: record skew for related tables, thenumber of warehouses that have cross-partition transac-tions, and the percentage of cross-partition transactionsin the same way as YCSB test. We report the results ofevery 2 seconds in Figure 10. The test starts with well-partitionable default transaction mix of TPC-C and variesworkload every 5 seconds. We see that it has similar be-haviours of Figure 9, where CormCC can almost alwaysperform the best. In this test, CormCC can achieve atmost 2.8x, 2.4x, 1.7x, and 1.8x throughput of PartCC,OCC, 2PL, and Hybrid respectively.

We then report the ratios of the mean throughput ofCormCC (after protocol switching) to that of the worstand best single protocols (labeled by max and min respec-tively) for each varied workload (i.e. every 5s) of bothbenchmarks in Figure 11 and Figure 12. We addition-ally report the ratio of the mean throughput of CormCCto the average throughput of three single fixed protocols(labelled by avg) in each varied workload. We see that

the highest ratio CormCC can achieve for YCSB andTPC-C is 2.2x and 2.6x respectively. For 55% work-loads of YCSB and 85% workloads of TPC-C, the av-erage ratio is at least 1.2x. The lowest ratio in YCSBand TPC-C test is 0.91x and 0.94x respectively, whichmeans that for the two benchmarks CormCC can achieveat least 91% and 94% throughput of the best protocol dueto wrong protocol selection for some partitions. Theseresults show that CormCC can achieve significant per-formance gains when a wrong protocol is selected for aworkload, can improve the throughput over single proto-cols for a wide range of varied workloads, and is robustto dynamic workloads.

6.5 Evaluating Mixed ExecutionIn this subsection, we first evaluate the performance ben-efits of CormCC over single protocols and Hybrid ap-proaches, and then test the overhead of CormCC.

We first show how mixed well-partitionable and non-partitionable workloads based on YCSB benchmark in-fluence the relative performance of CormCC to otherprotocols. We partition the database into 32 partitions,and start our test with a well-partitionable workload andthen increase the number of non-partitionable partitionsby an interval of 4. In this test, each transaction includes80% read operations. For these tests, we use transactionsconsisting 20 operations and skew record access withineach partition using Zipf distribution with theta = 1.5.

Figure 13 shows that CormCC always performs bestbecause it starts with PartCC and then adaptively mixesPartCC for partitionable workloads and 2PL for highlyconflicted non-partitionable counterpart. Compared toCormCC, the performance of PartCC degrades rapidlydue to high partition conflicts and other protocols cannottake advantage of partitionable workloads.

We then test workloads with the increasing percent-age of read operations. Initially, operations accessingeach partition include 80% read operations; we increasethe number of partitions receiving 100% read opera-tions by an interval of 4. In this test, our workload in-cludes 16 partitions having 100% cross-partition trans-actions among them, while the others are only accessedby single-partition transactions.

Figure 14 shows that CormCC has remarkable


throughput improvement over other protocols by com-bining the benefits of PartCC, OCC, and 2PL. Specif-ically, CormCC first mixes PartCC and 2PL, and thenapplies OCC for non-partitionable and read-only parti-tions. While Hybrid can adaptively mix OCC and 2PL,it is sub-optimal due to failing to leverage the benefitsof PartCC. In these tests, the speed-ups of CormCC overPartCC, OCC, 2PL, and Hybrid can be up to 3.4x, 2.2x,1.9x, and 2.0x respectively.

Next, we test the overhead of CormCC. We first exe-cute transactions using CormCC and track the percent-age of each transaction’s operations executed on recordsowned by a specific protocol (e.g. 1/2 of the transaction’srecords use OCC and 1/2 of the transaction’s records use2PL). With the percentages collected, we execute a mixof transactions where a corresponding percentage of thetransactions are executed exclusively on a single proto-col (e.g. 1/2 of the transactions are only OCC and 1/2 areonly 2PL), and compare the throughputs of the two ap-proaches. Note that to test the overhead without involv-ing the performance advantages of CormCC over singleprotocols, we use a single core to execute all transactions.

Figure 15 shows a micro-benchmark to evaluate mixedexecution overhead. We execute 50,000 transactions,each having 20 operations with 50% read operations,with the rest as read-modify-write operations. Key ac-cess distribution is uniform. The dataset is partitionedinto 32 partitions; 10 of them are managed by OCC, 10of them are for 2PL, and 12 are PartCC. The “mixed pro-tocols” shows the average throughput of CormCC withdifferent percentage of operations executed by differentprotocols; the “single protocol” results show the averagethroughput of a single protocol (e.g. 100% of transac-tions use OCC), or using single protocols to exclusivelyexecute a corresponding percentage of transactions. Wefind that our method has roughly the same throughputas a mix of “single protocols”, which shows that theoverhead of mixed concurrency control is minimal inCormCC. This is largely due to the fact that we do notadd extra meta-data operations to synchronize conflictsacross protocols.

6.6 Evaluating Mediated Switching

To evaluate the performance benefits and overhead ofmediated switching (denoted as Mediated), we com-pare it with a method of stopping all protocol execu-tion and applying the new protocol (denoted as StopAll).In our test, we perform a protocol switch from OCCto 2PL using five YCSB workloads with uniform keyaccess distribution. The first workload only includesshort-lived transactions with each having 10 read and 10read-modify-write operations. The other workloads in-clude a mix of short and long-running transactions. We

0

0.2

0.4

0.6

Short-only Long (0.5s) Long (1s) Long (2s) Long (4s)

Thro

ug

hp

ut

(Mill

ions

txn/s

)

OCC 2PL StopAll Mediated

Figure 16: Testing mediated switching

generate long-running transactions by introducing clientthink/wait time to short transactions. The long transac-tions last 0.5s, 1s, 2s, and 4s for the four workloads re-spectively and are dedicated to one worker. We collectthroughput every second and report the average through-put during protocol switching. We ensure that switchhappens at the start, end, and middle of a long runningtransaction, which represent that switch waits for little,whole, and half of the transaction respectively, and re-port three test cases for each mixed workload.

As shown in Figure 16, we see that Mediated andStopAll have a minimal throughput drop compared to2PL when the workload only includes short transac-tions. When long-running transactions are introduced,StopAll suffers due to waiting for the completion of long-running transactions, while Mediated can still maintainhigh throughput during the switch because Mediateddoes not stop all workers, but let them adopt both 2PLand OCC (i.e. upgrade phase); then, the coordinator no-tifies all workers to adopt 2PL (i.e. degrade phase) afterthe long transaction ends. Mediated protocol can achieveat least 93% throughput of OCC or 2PL due to the over-head of executing the logic of two protocols.

In addition, we perform the same test for all otherpairwise protocol switching. We find that the over-head is minimal under short-only workload. Whenlong-running transactions are introduced, the maximumthroughput drop is about 20% during protocol switch-ing from PartCC to OCC. This is acceptable comparedto StopAll, which cannot process new transactions in theswitch process. These experiments show that Mediatedcan maintain reasonable throughput during a protocolswitch, even in the presence of long transactions.

7 Conclusion

By exploring the design space of mixed concurrency con-trol, CormCC presents a new approach to generally mix-ing multiple concurrency control protocols, while not in-troducing coordination overhead. In addition, CormCCproposes a novel way to reconfigure a protocol for partsof a workload online with multiple protocols running.Our experiments show that CormCC can greatly outper-form static protocols, and state-of-the-art mixed concur-rency control approaches in various workloads.


References

[1] Toward Coordination-free and ReconfigurableMixed Concurrency Control (Technical Report).https://newtraell.cs.uchicago.edu/

files/tr_authentic/TR-2018-06.pdf.

[2] BERNSTEIN, P. A., HADZILACOS, V., ANDGOODMAN, N. Concurrency Control and Recov-ery in Database Systems. Addison-Wesley Long-man Publishing, 1986.

[3] BHARGAVA, B. K., HELAL, A., FRIESEN, K.,AND RIEDL, J. Adaptility experiments in the RAIDdistributed data base system. In Ninth Sympo-sium on Reliable Distributed Systems, SRDS 1990,Huntsville, Alabama, USA, October 9-11, 1990,Proceedings (1990), pp. 76–85.

[4] BHARGAVA, B. K., AND RIEDL, J. A model foradaptable systems for transaction processing. IEEETrans. Knowl. Data Eng. 1, 4 (1989), 433–449.

[5] CURINO, C., ZHANG, Y., JONES, E. P. C., ANDMADDEN, S. Schism: a workload-driven approachto database replication and partitioning. PVLDB 3,1 (2010), 48–57.

[6] DIACONU, C., FREEDMAN, C., ISMERT, E.,LARSON, P., MITTAL, P., STONECIPHER, R.,VERMA, N., AND ZWILLING, M. Hekaton: SQLserver’s memory-optimized OLTP engine. In Pro-ceedings of the ACM SIGMOD International Con-ference on Management of Data, SIGMOD 2013,New York, NY, USA, June 22-27, 2013 (2013),pp. 1243–1254.

[7] DIDONA, D., DIEGUES, N., KERMARREC, A.,GUERRAOUI, R., NEVES, R., AND ROMANO,P. Proteustm: Abstraction meets performancein transactional memory. In Proceedings of theTwenty-First International Conference on Architec-tural Support for Programming Languages and Op-erating Systems, ASPLOS ’16, Atlanta, GA, USA,April 2-6, 2016 (2016), pp. 757–771.

[8] HARIZOPOULOS, S., ABADI, D. J., MADDEN, S.,AND STONEBRAKER, M. OLTP through the look-ing glass, and what we found there. In Proceedingsof the ACM SIGMOD International Conference onManagement of Data, SIGMOD 2008, Vancouver,BC, Canada, June 10-12, 2008 (2008), pp. 981–992.

[9] JUNG, H., HAN, H., FEKETE, A. D., HEISER, G.,AND YEOM, H. Y. A scalable lock manager formulticores. In Proceedings of the ACM SIGMOD

International Conference on Management of Data,SIGMOD 2013, New York, NY, USA, June 22-27,2013 (2013), pp. 73–84.

[10] KALLMAN, R., KIMURA, H., NATKINS, J.,PAVLO, A., RASIN, A., ZDONIK, S. B., JONES,E. P. C., MADDEN, S., STONEBRAKER, M.,ZHANG, Y., HUGG, J., AND ABADI, D. J. H-store: a high-performance, distributed main mem-ory transaction processing system. PVLDB 1, 2(2008), 1496–1499.

[11] KEMPER, A., AND NEUMANN, T. Hyper: Ahybrid oltp&olap main memory database systembased on virtual memory snapshots. In Proceed-ings of the 27th International Conference on DataEngineering, ICDE 2011, April 11-16, 2011, Han-nover, Germany (2011), pp. 195–206.

[12] KIM, K., WANG, T., JOHNSON, R., AND PAN-DIS, I. ERMIA: fast memory-optimized databasesystem for heterogeneous workloads. In Proceed-ings of the 2016 International Conference on Man-agement of Data, SIGMOD Conference 2016, SanFrancisco, CA, USA, June 26 - July 01, 2016(2016), pp. 1675–1687.

[13] KIMURA, H. FOEDUS: OLTP engine for a thou-sand cores and NVRAM. In Proceedings of the2015 ACM SIGMOD International Conference onManagement of Data, Melbourne, Victoria, Aus-tralia, May 31 - June 4, 2015 (2015), pp. 691–706.

[14] LARSON, P., BLANAS, S., DIACONU, C., FREED-MAN, C., PATEL, J. M., AND ZWILLING,M. High-performance concurrency control mech-anisms for main-memory databases. PVLDB 5, 4(2011), 298–309.

[15] LEVANDOSKI, J. J., LOMET, D. B., AND SEN-GUPTA, S. The bw-tree: A b-tree for new hardwareplatforms. In 29th IEEE International Conferenceon Data Engineering, ICDE 2013, Brisbane, Aus-tralia, April 8-12, 2013 (2013), pp. 302–313.

[16] MU, S., CUI, Y., ZHANG, Y., LLOYD, W., ANDLI, J. Extracting more concurrency from dis-tributed transactions. In 11th USENIX Sympo-sium on Operating Systems Design and Implemen-tation, OSDI ’14, Broomfield, CO, USA, October6-8, 2014. (2014), pp. 479–494.

[17] NARULA, N., CUTLER, C., KOHLER, E., ANDMORRIS, R. Phase reconciliation for contendedin-memory transactions. In 11th USENIX Sympo-sium on Operating Systems Design and Implemen-tation, OSDI ’14, Broomfield, CO, USA, October6-8, 2014. (2014), pp. 511–524.


[18] PANDIS, I., JOHNSON, R., HARDAVELLAS, N.,AND AILAMAKI, A. Data-oriented transaction ex-ecution. PVLDB 3, 1 (2010), 928–939.

[19] PANDIS, I., TOZUN, P., JOHNSON, R., AND AIL-AMAKI, A. PLP: page latch-free shared-everythingOLTP. PVLDB 4, 10 (2011), 610–621.

[20] REN, K., FALEIRO, J. M., AND ABADI, D. J. De-sign principles for scaling multi-core OLTP underhigh contention. In Proceedings of the 2016 Inter-national Conference on Management of Data, SIG-MOD Conference 2016, San Francisco, CA, USA,June 26 - July 01, 2016 (2016), pp. 1583–1598.

[21] REN, K., THOMSON, A., AND ABADI, D. J.Lightweight locking for main memory databasesystems. vol. 6, pp. 145–156.

[22] SHANG, Z., LI, F., YU, J. X., ZHANG, Z., ANDCHENG, H. Graph analytics through fine-grainedparallelism. In Proceedings of the 2016 Interna-tional Conference on Management of Data, SIG-MOD Conference 2016, San Francisco, CA, USA,June 26 - July 01, 2016 (2016), pp. 463–478.

[23] SU, C., CROOKS, N., DING, C., ALVISI, L.,AND XIE, C. Bringing modular concurrency con-trol to the next level. In Proceedings of the 2017ACM International Conference on Management ofData, SIGMOD Conference 2017, Chicago, IL,USA, May 14-19, 2017 (2017), pp. 283–297.

[24] TAI, A. T., AND MEYER, J. F. Performabilitymanagement in distributed database systems: Anadaptive concurrency control protocol. In MAS-COTS ’96, Proceedings of the Fourth Interna-tional Workshop on Modeling, Analysis, and Sim-ulation On Computer and Telecommunication Sys-tems, February 1-3, 1996, San Jose, California,USA (1996), pp. 212–216.

[25] TANG, D., JIANG, H., AND ELMORE, A. J. Adap-tive concurrency control: Despite the looking glass,one concurrency control does not fit all. In CIDR2017, 8th Biennial Conference on Innovative DataSystems Research, Chaminade, CA, USA, January8-11, 2017, Online Proceedings (2017).

[26] TATAROWICZ, A., CURINO, C., JONES, E. P. C.,AND MADDEN, S. Lookup tables: Fine-grainedpartitioning for distributed databases. In IEEE28th International Conference on Data Engineer-ing (ICDE 2012), Washington, DC, USA (Arling-ton, Virginia), 1-5 April, 2012 (2012), pp. 102–113.

[27] TU, S., ZHENG, W., KOHLER, E., LISKOV, B.,AND MADDEN, S. Speedy transactions in mul-ticore in-memory databases. In ACM SIGOPS24th Symposium on Operating Systems Principles,SOSP ’13, Farmington, PA, USA, November 3-6,2013 (2013), pp. 18–32.

[28] WANG, T., AND KIMURA, H. Mostly-optimisticconcurrency control for highly contended dynamicworkloads on a thousand cores. PVLDB 10, 2(2016), 49–60.

[29] WANG, Z., MU, S., CUI, Y., YI, H., CHEN, H.,AND LI, J. Scaling multicore databases via con-strained parallel execution. In Proceedings of the2016 International Conference on Management ofData, SIGMOD Conference 2016, San Francisco,CA, USA, June 26 - July 01, 2016 (2016), pp. 1643–1658.

[30] WU, Y., CHAN, C. Y., AND TAN, K. Transactionhealing: Scaling optimistic concurrency control onmulticores. In Proceedings of the 2016 Interna-tional Conference on Management of Data, SIG-MOD Conference 2016, San Francisco, CA, USA,June 26 - July 01, 2016 (2016), pp. 1689–1704.

[31] XIE, C., SU, C., LITTLEY, C., ALVISI, L.,KAPRITSOS, M., AND WANG, Y. High-performance ACID via modular concurrency con-trol. In Proceedings of the 25th Symposium on Op-erating Systems Principles, SOSP 2015, Monterey,CA, USA, October 4-7, 2015 (2015), pp. 279–294.

[32] YAO, C., AGRAWAL, D., CHEN, G., LIN, Q.,OOI, B. C., WONG, W., AND ZHANG, M. Ex-ploiting single-threaded model in multi-core in-memory systems. IEEE Trans. Knowl. Data Eng.28, 10 (2016), 2635–2650.

[33] YU, P. S., AND DIAS, D. M. Analysis of hybridconcurrency control schemes for a high data con-tention environment. IEEE Trans. Software Eng.18, 2 (1992), 118–129.

[34] YU, X., BEZERRA, G., PAVLO, A., DEVADAS,S., AND STONEBRAKER, M. Staring into theabyss: An evaluation of concurrency control withone thousand cores. PVLDB 8, 3 (2014), 209–220.

[35] YU, X., PAVLO, A., SANCHEZ, D., AND DE-VADAS, S. Tictoc: Time traveling optimistic con-currency control. In Proceedings of the 2016 Inter-national Conference on Management of Data, SIG-MOD Conference 2016, San Francisco, CA, USA,June 26 - July 01, 2016 (2016), pp. 1629–1642.


[36] YUAN, Y., WANG, K., LEE, R., DING, X., XING,J., BLANAS, S., AND ZHANG, X. BCC: reducingfalse aborts in optimistic concurrency control withlow cost for in-memory databases. PVLDB 9, 6(2016), 504–515.

[37] ZHANG, Y., POWER, R., ZHOU, S., SOVRAN,Y., AGUILERA, M. K., AND LI, J. Transactionchains: achieving serializability with low latency ingeo-distributed storage systems. In ACM SIGOPS24th Symposium on Operating Systems Principles,SOSP ’13, Farmington, PA, USA, November 3-6,2013 (2013), pp. 276–291.

[38] ZHENG, W., TU, S., KOHLER, E., AND LISKOV,B. Fast databases with fast durability and recov-ery through multicore parallelism. In 11th USENIXSymposium on Operating Systems Design and Im-plementation, OSDI ’14, Broomfield, CO, USA, Oc-tober 6-8, 2014. (2014), pp. 465–477.


Documents

Toward Coordination-free and Reconfigurable Mixed ...Toward Coordination-free and Reconfigurable Mixed Concurrency Control Dixin Tang and Aaron J. Elmore, University of Chicago