Bringing Transactional Guarantees to MongoDB

Bringing Transactional Guarantees to MongoDB

Paul Robinson [email protected]

@pfrobinson

Agenda

•ACID Transactions

•Compensating Transactions

•Code Example

•Today and Planned

Transactions with RDBS

Update balance and create an order atomically

id username item size

0 0 Stumpjumper L

id username email voucher

0 paul.robinson paul… 3000

Invoices

Users

id username item size

0 0 Stumpjumper L

1 0 Zesty L

id username email voucher

0 paul.robinson paul… 200

Invoices

Users

{ id: "<ObjectID1>", username: "paul.robinson", email: "[email protected]" voucher: 3000, invoices: { {"Stumpjumper", "L"}, } }

Transactions with Document Stores

Update balance and create an order atomically

{ id: "<ObjectID1>", username: "paul.robinson", email: "[email protected]" voucher: 200, invoices: { {"Stumpjumper", "L"}, {"Zesty", "L"} } }

But, sometimes this isn’t possible…

Change Multiple Documents{ user: ‘Paul’ balance: 1000 }

{ user: ‘Fred’ balance: 0 }

{ user: ‘Paul’ balance: 700 }

{ user: ‘Fred’ balance: 300 }

E.g. Money Transfer, audited delete, integration

Why doesn’t MongoDB support multi-document transactions?

Scaling MongoDB

Router (mongos)

Shard

Master

D1

Slave 1

D1

Slave 2

D1

Shard

Master

D2

Slave 1

D2

Slave 2

D2

Multi-Document Transactions

Client

Shard

D1Router

(mongos)Shard

D2

D1


Client

Shard

D1Router

(mongos)Shard

D2


Client

Shard

D1Router

(mongos)Shard

D2

Client D1?

Client


ClientD2

Shard

D1Router

(mongos)Shard

D2

Client D1?

Client


Client

Shard

D1Router

(mongos)Shard

D2

Client D1?

D1Client


Client

Shard

D1Router

(mongos)Shard

D2

Done

ClientD1

(Multi-document) ACID doesn’t scale …

Don’t throw out transactions altogether!

Extended Transactions

• Umbrella term

• Alternatives to ACID

Guarantees

ACIDNone Extended

ACID Vs Compensating Transactions

• ACID 1. Lock each resource 2. Commit/rollback each resource !

• Compensating 1. Commit each resource 2. Confirm/compensate each resource

Compensating Transactions

• Eventually consistent

• Relaxed isolation

• Can move locks to application logic

• Support Long running tasks

Transaction OptionsAtomic Atomic Batch Traditional

ACIDCompensating!Transactions

Single Doc ✓ ✓ ✓ ✓Multi Doc O ✓ ✓ ✓Sharding ✓ O O ✓

Multi stores O O ✓ ✓ACID ✓ ✓ ✓ O

App Unaware ✓ ✓ ✓ O

Existing Patterns

• Compensating Transaction

• Just a pattern

• Needs implementing

• Recovery is hard!

MongoDB

Application (incl transaction and recovery code)

Narayana’s Solution in WildFly 8

• Middleware

• Transaction Manager driven

• Annotation-based API

• Based on Sagas

MongoDB

Narayana

Application

Protocol Details

Starting State{ user: ‘A’ bal: 1000 }

{ user: ‘B’ bal: 0 }

Log Compensation Handler 1{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }



<txid> pending <comp1>

Update Document 1{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 700 tx: <txid> }






Log Compensation Handler 2{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }









<txid> pending <comp1> <comp2>

Update Document 2{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }








{ user: ‘B’ bal: 300 tx: <txid> }





Update Transaction Log{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }















<txid> committing <comp1> <comp2>

Confirm Document 1{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }



















Confirm Document 2{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }



















{ user: ‘A’ bal: 700 }



Completed{ user: ‘A’ bal: 1000 }

{ user: ‘A’ bal: 1000 }



















{ user: ‘A’ bal: 700 }



{ user: ‘A’ bal: 700 }

{ user: ‘B’ bal: 300 !}

Code Example

Money Transfer

• Each user has a balance

• Move money between users

• Update both documents atomically

public class BankingService { ! @Inject AccountManager accountManager; ! @Compensatable public void transferMoney(String fromAccount, String toAccount, Integer amount) { ! accountManager.debitAccount(fromAccount, amount); accountManager.creditAccount(toAccount, amount); }

Service

public class AccountManager { ! @Inject AccountDAO accountDAO; @Inject private CompensationManager compensationManager; @Inject CreditData creditData; ! @TxCompensate(UndoCredit.class) public void creditAccount(String account, Integer amount) { ! //High value transfers (over 500) are not allowed with this service if (amount > 500) compensationManager.setCompensateOnly() && return; creditData.setToAccount(account); creditData.setAmount(amount); accountDAO.incrementBalance(account, amount); } …

Account Manager (credit part)

… @Inject DebitData debitData; ! @TxCompensate(UndoDebit.class) public void debitAccount(String account, Integer amount) { ! debitData.setFromAccount(account); debitData.setAmount(amount); ! accountDAO.incrementBalance(account, -1 * amount); } }

Account Manager (debit part)

@CompensationScoped public class CreditData implements Serializable { ! private String toAccount; private Integer amount; … } !@CompensationScoped public class DebitData implements Serializable { ! private String fromAccount; private Integer amount; … }

Compensation Data

public class UndoCredit implements CompensationHandler { ! @Inject CreditData creditData; @Inject AccountDAO accountDAO; ! public void compensate() { ! if (creditData.getToAccount() != null) { accountDAO.incrementBalance( creditData.getToAccount(), -1 * creditData.getAmount()); } } }

Compensation Handler (Credit)

What we have today

• Compensating-Transactions API

• MongoDB Example

• Other Quickstarts

• Blog post series

• Ships with WildFly 8.Final+

What’s Planned

• MongoDB RM

• Compensating state Recovery

• Throughput/Scalability study

• RDBMS integration

• Other language support

Getting Started• Explore the quickstarts http://github.com/jbosstm/quickstart/

• Give feedback (forum) http://community.jboss.org/en/jbosstm

• Track issues http://issues.jboss.org/browse/JBTM

• Subscribe to Blog http://jbossts.blogspot.co.uk/

• Contact me [email protected], @pfrobinson

Software

Bringing Transactional Guarantees to MongoDB