analternativeviewofadistributed databaseSergejJurecko

ActorDB

IntroWhatitis.Whatisitfor.

DistributedSQLdatabase

1

What is it?

Features:• Nosinglepointoffailure• Horizontallyscalable• Consistent- usingRaftconsensusalgorithm• OntopofSQLiteandLMDB• MySQLandThriftconnectors

Easy to use

• Workswellevenasasinglenode

• ConfiguredwithSQLandactordb_console

2

Explicitsharding

3

What’s the catch?

Thismeans:• Manysmalldatabases,notasinglelargeone• Goodatconcurrency• Databasesper:user,entity,service• Naturalshardingkey

Examples

• actormytype(myactor);select*fromtab;

• actort(a1,a2)create;insertintotabvalues(1,’aa’);

• actort(*); {{RESULT}}select*fromtab;

• actort(*whereidlike‘u%’);{{RESULT}}select*fromtab;

4

File sync

• Lotsofusers,lotsofdata

• Filehierarchy

• CTE,Foreignkeys,Transactions

• Queriesalwaysper-user

5

Great for

• SaaS

• Mobileappbackends

• Microservices

6

Microservices

• Splitaproblemintosmallerchunks

• Decouple

• Solveproblemsonthecommunicationlayer

• Inherentlyadistributedsystem

7

Microfail

8

Microservices with ActorDB

• Stillrelational

• Distributedtransactions

• Asmanytypesasyouwant

9

ActorDB vs Traditional SQL

• Codefollowsyourdatamodel

• TraditionalSQLdatabasesleadtomonoliths

• ActorDB forcesdecoupling

10

How it worksSQLite,LMDB

SQLite

• CTE,foreignkeys,FTS,JSON

• Embeddable

• Fast

• Stable

• Verybasicstoragelayer

11

LMDB

• Transactions

• Compact

• Nowriteamplification

• Goodwithlargevalues

12

SQLite + LMDB

• SQLitewal interface:findpage,readpage,writepage

• SQLiteisunmodifiedoutsidewal.c

• SQLite->WAL(compression,replication)->LMDB

13

No wal?

• Thereisawal,it’sjustinLMDB

• EverySQLitepagehasXversions

• Checkpointsaredeletes

14

TODO

• ExternalWAL

• Higherwriteconcurrency

• Somewriteamplification

15

ShardingSpreadingdataacrossnodes.

ActorDB -> Clusters -> Actor

• ActorDB dividesdataintoclusters

• Aclusterisareplicationunit

• Anactorliveswithinacluster

• Everynodehas~4shards

16

ActorDB -> Clusters -> Actor

17

actormytype(myactor);select*frommytab;

18

Find the shard

Whathappens:• hash(myactor)->Integer• Integerdeterminesshard• Shardbelongstoanode• Anodeisassignedtoacluster

What is a shard?

• Ashardisanactor

• Ashardisaregistryofactors

• FromXtoYofhashspace

• Ownedbyanode,livesinacluster

19

4 to 400 shards

• Startwith1node

• Addasecond

• Start“stealing”actors,fromtoptomiddle

• Redirectrequestsduringmovement

20

Actor Copy/Move

1. Stopdoingcheckpoints

2. Frompage1toN,startcopying

3. Copyuntilcaughtup

4. Lockandfinishup

21

KV storeAndwhyit’sawesome.

Shards expanded

• Storedatadirectlyintoshards

• Expandedshardschema

• Lotskeys,notalotofdataperkey

• actormykvtype(mykey);UPDATEactorsSETval =val+1WHEREid=’mykey';

22

Foreignkeystoactorstable

23

Sub tables

ResultsinaKVstorewith:• Structureddata• Consistentpartialupdates• Localjoins

Distributed file system

• Hugenumberoffiles

• Smallamountofdataperfile

• Actor-per-fileimpractical

• KVstoretotherescue

24

Distributed file system

• actorfilesystem(abc123); insertorignoreintoactors,…;insertintonodevalues(‘abc123’,’server1’); insertintouser_file(12,123, ‘abc123’);

25

Future plans

Future plans

• Georeplication

• Kafkalikepubsub

• WALforLMDB

• Map-reduce(luajit)

26

That is it!Thankyou.