Functional Database Strategies

Scalæ by the Bay 2016

by Jason Swartz@swartzrock

Step One

Buy A Functional Database

Silly! Databases Aremutable, not

functional!

Well?Aren’t they?

1. Immutable Rows2. Window Functions3. Events, Not State4. DB Interactions

Agenda

Let’s Talk AboutImmutable Tables

Create-Read-Update-Delete

GET /issuesGET /issues/{id}POST /issuesPUT /issues/{id}

Issue Endpoints

How Do These EventsAffect The Database?

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=103. PUT /issues/1 done=true

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

Not Bad.

Do You Know How We Got Here?

Why is ‘assignee’ NULL?

Mutable Table Rows Lose History

Immutable Table Rows Keep Their History

Let’s Try To Lock Down

Our State

1. POST /issues title=’Config ELB’

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

1. POST /issues title=’Config ELB’2. PUT /issues/1 assignee=10

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

Tables Are Mutable, But Table Rows Should Be

Immutable

In Other Words, Tables Should Be

Append-Only

How Do You Make An Append-Only

Table?

One: Don’t Let Your DB User Make

Changes

Grant select, insert on issues to my-db-user;

-- tested on Postgresql

Thank You! Goodbye!

Two: Pick The RightColumns

1. GET /issues/1

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:19 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+

+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | NULL | true |+------+-------------+------------+----------+-------+

create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false)

1. GET /issues/1

+------+-------------+------------+------------+----------+-------+| id | created | issue_id | title | assignee | done |+------+-------------+------------+------------+----------+-------+| 1 | 09-18 18:13 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+| 2 | 09-18 18:16 | 1 | Config ELB | 10 | false |+------+-------------+------------+------------+----------+-------+| 3 | 09-18 18:19 | 1 | Config ELB | NULL | false |+------+-------------+------------+------------+----------+-------+

+------+-------------+------------+------------+----------+-------+| 4 | 09-18 18:24 | 1 | Config ELB | NULL | true |+------+-------------+------------+------------+----------+-------+

select * from issueswhere issue_id = :issue_idorder by id desc limit 1;

That’s The Basics Of Immutability

In Table Rows

Agenda

SQL:2003 expands Groups into Windows

Works Great InPostgresql

● Aggregation functions, eg sum(), rank(), avg()● Window definitions with over()● Grouping with partition by, order

Window Functions

+------+------------+------------+----------+-------+| id | issue_id | title | assignee | done |+------+------------+------------+----------+-------+| 1 | 1 | Config ELB | NULL | false |+------+------------+------------+----------+-------+| 2 | 1 | Config ELB | 10 | false |+------+------------+------------+----------+-------+| 3 | 2 | Bug to fix | 11 | false |+------+------------+------------+----------+-------+| 4 | 2 | Bug to fix | 11 | true |+------+------------+------------+----------+-------+

with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day')select * from latest_issues where row_number = 1

+------+------------+------------+----------+-------+------------+| id | issue_id | title | assignee | done | row_number |+------+------------+------------+----------+-------+------------+| 1 | 1 | Config ELB | NULL | false | 2 |+------+------------+------------+----------+-------+------------+| 2 | 1 | Config ELB | 10 | false | 1 |+------+------------+------------+----------+-------+------------+| 3 | 2 | Bug to fix | 11 | false | 2 |+------+------------+------------+----------+-------+------------+| 4 | 2 | Bug to fix | 11 | true | 1 |+------+------------+------------+----------+-------+------------+

That WasWindow

Functions

Agenda

You Know How To Maintain State

Do We Still Need State?

Let’s Talk AboutEvent-Sourcing

+------+-------------+------------+----------+-------+| id | updated | title | assignee | done |+------+-------------+------------+----------+-------+| 1 | 09-18 18:13 | Config ELB | NULL | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:16 | Config ELB | 10 | false |+------+-------------+------------+----------+-------+| 1 | 09-18 18:24 | Config ELB | 10 | true |+------+-------------+------------+----------+-------+

Events

States

Now We’re Storing Events,

Not States

create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text)

1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’

+----+-------------+----------+------------+---------+| id | created | issue_id | originator | payload |+----+-------------+----------+------------+---------+| 14 | 09-18 18:50 | 1 | 4a482... | <...> |+----+-------------+----------+------------+---------+

Create Events AndSimulate The State

1. Create-Issue

Issue(“Config ELB”, null, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue

Issue(“Config ELB”, 10, false);

Real Events

Virtual States

1. Create-Issue2. Assign-Issue3. Complete-Issue

Issue(“Config ELB”, 10, true);

Real Events

Virtual States

So Why UseEvent-Sourcing?

1. High Write Performance2. Potential for Command/Query Separation3. Auditable4. Replayable5. Undo-able6. Monitorable

Reasons For Event-Sourcing

It’s Like Having Control Over The Versions Of

Your State Changes

It’s Like Having Control Over The Versions Of

Your Data

It’s Like GitFor Your Data

1. Frankly, It’s Weird2. Requires Events. No Events, No Event-Sourcing.3. As Of November 2016, It’s Still Non-Standard

Reasons Against Event-Sourcing

Wait! We’reScala

developers!

Who Cares About BeingNon-Standard?

That About Sums Up Event Sourcing

Agenda

Remember That YourDatabase

is mutable.

Avoid SharingYour State

● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST● Execute state changes at THE EDGE

Safe Database Interactions

DoobieA Typelevel Project

Are these steps? Or a Monad?

That About Sums Up Database

Interactions

Okay, Actually That’s The Entire Talk

Unless There’s More Time

Scalæ by the Bay 2016

by Jason Swartz@swartzrock

Thank You For Attending

THIS SPACELEFT BLANK

Functional Database Strategies

Engineering

Dynamic Functional Dependencies and Database Aging

FUNCTIONAL SPECIFICATION FOR NP ADMINISTRATION DATABASE

24HOP Session - Database Administration Strategies

Database Caching Strategies Using Redis Web Services – Database Caching Strategies using Redis Page 2 • Database integrated caches: Some databases, such as Amazon Aurora, offer

Functional level strategies

Functional Strategies 1

Objects in relational database schemes with functional ...€¦ · formally defined and characterized for relational database schemes with functional, inclusion, and exclusion dependencies

Lecture 09: Functional Dependencies, Database Design

31362852 Functional Level Strategies Plan and Policies

A FUNCTIONAL DATABASE

Tomato Functional Genomics Database: a comprehensive ... · Tomato Functional Genomics Database: a comprehensive resource and analysis package for tomato functional genomics Zhangjun

Functional Treatment Ideas Strategies in Adult … · Functional Treatment Ideas & Strategies in Adult ... Functional Treatment Ideas & Strategies in Adult Hemiplegia ... therapy

Strategies for functional modeling

Evaluation Strategies for Functional Logic Programming

Formulation of Functional Strategies

Functional level strategies-Plan and Policies

Web Based_ Functional Strategies

Business and Functional Level Strategies

BioFNet: biological functional network database for ... Bioinform-2013-Kurata.pdf · BioFNet: biological functional network database for analysis and synthesis of biological systems

Lecture 8 Implementing strategies: functional level issues