Return of the transaction king

Ryan Knight - @knight_cloud

Return of the Transaction King

Brief History of Data

RDBMS + ACID Transactions

Atomic

Consistent

Isolated

Durable

Traditional ACID Transactions

Did not Scale with the Internet

Leader w/ Read Replicas

Distributed Locks

CAP Theorem Forced Choices

Global Scale or Strong Consistency?

Rise of Eventual Consistency

Eventual Consistency

Really a Necessary Evil

Last Write Wins

Event Sourcing

Write now and figure it out later

● DNS, IoT, Time Series, Monitoring, Web Metrics, Followers, etc.

● Modify Business to Fit - Allow for Inconsistency○ Inventory - Oversell and cancel order if

necessary○ Bank - Overdraft and charge

Where Event Sourcing Works

We live in a stateful and finite world! - James Ward

But ….

Where Event Sourcing BreaksWhen Order of Events Matters

Can not write from 2 hosts and guarantee ordering

Reconcile Diverging History of Events

Possible to Lose History

Strong Consistency

Differing Views of Strong Consistency

Write / Read Local Quorum

No Read before Write

No Compare and Swap

No Snapshot Isolation

Write SkewTwo concurrent transactions each determine what they are writing based on reading a data set which overlaps what the other is writing

begriffs.com

Can we Ignore Consistency?

Pushes consistency problem into the application where they are far harder to solve

Eventual Consistency really is a necessary Evil

Consistency in the Application Tier

Events Sourcing w/ Command

Serialization

Serialization Points - Kafka

Consumers

What happens when they fail?

Cloud Native / Microservices

Partitioning of the Application Domain and Related Data

Increased Coordination between Data

Number of Microservices Increases Number of Interactions

Exponential Growth in Complexity

Cloud Native + NoSQL - ACID==========Rube Goldberg machine

ACID Transactions

Atomic

Consistent / Correct

Isolated

Durable

ACID TRANSACTIONS

+ EVENT SOURCING

============RETURN OF

TRANSACTIONS

Modern Transactional Databases

Fauna

Google Spanner

CockroachDB

GLOBAL SCALE + STRONG CONSISTENCY

Value of Transactions

Decrease Application Tier Complexity

Reduce Cognitive Overhead

Increased Developer Productivity

Increased Focus on Business Value

● Deal with People's Money ● Financial Transactions● Inventory ● Stock Exchange● Auctions● Points

When do you need ACID TXN?

Do we really have to sacrifice AP to get C?But there are two important caveats: first, you only need forfeit something during an actual partition, and even then there are many mitigations. Second, the actual theorem is about 100% availability, while the interesting discussion here is about the tradeoffs involved for realistic high availability.

Eric Brewer Spanner Paper

Problems with CAP, and Yahoo’s little known NoSQL system - Daniel Abadi

Tradeoff between latency (L) and consistency under normal conditions

Google Spanner

● Private global network along with GPS receivers and atomic clocks.

● Atomic Clocks for Serializability Guarantees● AdWords & Google Play for 5+ years● Google Cloud Platform Control Plane ● No Scheduled Downtime● Can do maintenance by moving data between nodes.

CockroachDBHybrid Logical Clock similar to a vector clock for ordering of transactions

Challenges with clock skew

Waits up to 250 MS on reads

What is Fauna?

Globally distributed, transactional database

Born out of Twitter and the Calvin Pape

Functional / Type-Safe Query Language

Indexes, Constraints and Views

Fauna Transaction Resolution

Resolves transactions before committing to DB

Execute phase and Commit phase

All Transaction w/ in a epoch our ordered and applied as a batch

Uses a RAFT replicated log that derives a total order of all transactions.

Distributed ACID Transaction Engine

Consensus protocols from Raft and Calvin Paper

Can span thousands of records and indexes

Designed for high throughput and WAN replication

Ensures multi-node and multi- datacenter correctness

Security, Identity, IsolationNative Security and Access Control

Row-level security, identity, and isolation

Complete Data Isolation across datasets

Quality of Service Managed Resource Scheduler

Operating System Like Design

Single Cluster for Differing and Competing Workloads

Temporality

FaunaDB keeps all instances of a data, not overwriting them but creating new ones when a write is performed. This is useful especially when auditing data, verifying its evolution over time.

Fauna Data RepresentationDatabase

Class(es)

Instanc(es)

Arrays and Scalar Types

Nested Data

Database

Database(s)

Class(es)

Class(es)

InstancesEach record stored as an object.

All objects are instances of a specific class.

Every instance has an identifier called a ref.

An instance’s ref encodes its class along with a unique id

FunctionalTransaction Scope

Not traditional begin / end transaction

Entire Function / Query Block is scope of transaction

Paradigm will feel natural to Functional Programmers

Similar to Traditional RDBMS Indexes Terms and Values

Transform, cover, and order instance data

Enforce unique constraints.

Referenced explicitly in query expressions;

Optimizer does not make index applications

Indexes

Example “Smart Contract”

● User Nodes w/ Simulated Account Balance

● Server Node acts as Intermediary and

Notary on Contract

● User Nodes Propose Borrowing from Peers

● Peer Accepts or Rejects Proposal

● Contract is Created between Peers Lending

Fixed Amount