NoSQLContemplating a Singularity

About me…

• Dave Smith (aka Dizzy)

• VP, Eng. @ Orchestrate

• History

• Jabber / XMPP

• Ping Identity

• Basho

Thesis

Future database systems will be adaptive aggregates of what we now call a “database".

“NoSQL” is an expanding singularity in database systems.

Timeline of Forms

1960s

Hierarc

hical

Relatio

nal

1970s

NoSQL

1990s

Object-O

riente

d

1980s• BerkeleyDB - 1994

• Lucene - 1999

• Memcache - 2003

• Hadoop - 2005

• Bigtable Paper - 2006

• Dynamo Paper - 2007

• Change in types of queries - “Not only SQL”

• Discards assumptions about location and consistency of data

• Tectonic shifts in workloads

NoSQL as a Singularity

Implications(Theoretical)

• No one data structure will be optimal or even sufficient

• Consistency will necessarily be relaxed

• Architectural flexibility will be essential

Implications(Practical)

• Multiple databases will be required

• DBAs, DevOps and Devs will all have to deal with this increased complexity

• Design your systems with this assumption; don’t assume writes will be instantly available

• Capacity planning gets much harder

• Reasoning about failure will get much, much harder

The Future

• “Database” will be a sprawling system of systems

• Accommodates multiple indices, consistency tradeoffs

• Indexing will NOT be synchronous

• Adaptivity will be key to keeping queries fast, not just a good general purpose index