View
216
Download
0
Category
Tags:
Preview:
Citation preview
: what’s all the buzz about?
http://nosql-database.org/
Next generation databases are:• Non-relational, • Distributed, • Open-source,• Horizontal scalable
Often more characteristics: Schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge data amount
List of NoSQL databases [122+]• Wide Column Store / Column FamiliesHBase, Cassandra, Hypertable, Cloudata, Cloudera, Amazon SimpleDB• Document StoresCouchDB, MongoDB, Terrastore, ThruDB, OrientDB, RavenDB, Citrusleaf, SisoDB• Key Value / Tuple StoreAzure Table Storage, MEMBASE, Riak, Redis, Chordless, GenieDB, Scalaris, Tokyo Cabinet / Tyrant, KeyspaceBerkeley DB, MemcacheDB, Faircom C-Tree, Mnesia, LightCloud, Hibari, HamsterDB, STSdb, Pincaster, RaptorDB• Eventually Consistent Key Value StoresAmazon Dynamo, Voldemort, Dynomite, KAI• Graph DatabasesNeo4J, Infinite Graph, Sones, InfoGrid, HyperGraphDB, Trinity, AllegroGraph, Bigdata, DEX, OpenLink, Virtuoso, VertexDB, FlockDB• Object Databasesdb4o, Versant, Objectivity, Gemstone, Progress, Starcounter, Perst, Caching, ZODB, NEO, PicoLisp, Sterling• More and more databases
So what’s wrong with relational databases?
Main principals of RDBMS
• SQL• ACID• Atomic “all or nothing”• Consistent means that data moves from one correct
state to another correct state, with no possibility that readers could view different values that don’t make sense together.
• Isolated means that transactions executing concurrently will not become entangled with each other.
• Durable once a transaction has succeeded, the changes will not be lost.
Shortcomings of RDBMS• Transactions under heavy load• Complexities of vertical scaling• 2 phase commit (2PC) protocol
Sharding
If you can’t split it, you can’t scale it (Randy Shoup, distinguished architect, eBay)
• Sharging approach• Feature-based shard or functional segmentation• Key-based sharding• Lookup table
• Shared-nothing or Cassandra like sharding
The real question is not “What’s wrong with relational
databases?” but rather, “What problem do you have?”
Brewer’s CAP Theorem
Availability
Consistency Partition Tolerance
Brewer’s CAP Theorem
Availability
ConsistencyPartition Tolerance
Amazon Dynamo derivatives:Cassandra, Voldemort, Riak, CouchDB
Neo4j, Google Big Table and its derivatives: MongoDB, Redis, Hypertable
Relational: MySQL, Oracle, MSSQL
in 50 words or less
Apache Cassandra is an open source, distributed, decentralized, elastically scalable, highly available, fault-tolerant, tuneably consistent, column-oriented database that bases its distribution design on Amazon’s Dynamo and its data model on Google’s Bigtable. Created at Facebook, it is now used at some of the most popular sites on the Web.
Cassandra case studies
Cassandra outlines
• BASE (Basically Available Soft-state Eventual consistency) and not ACID (Atomicity, Consistency, Isolation, Durability)
• Distributed and decentralized• Elastic scalability• High availability and fault tolerance• Tunable consistency
Use cases for Cassandra
• Large deployments• Lots of writes, statistics and analysis• Geographical distribution• Evolving applications
Writes
Commit log
Write
Memtable
SSTable SSTable
• No reads• No seeks• Fast• Sequential disk access • Atomic within a column family• Any node• Always writable (hinted hand-off)• ≈ 0.2 ms
Threshold
Reads
Read
SSTable SSTable
• Bloomfilter field to determine whether a provided key is in the SSTable
• Index field for quick read• Any node• Read repair• ≈ 15 ms
Memtable
Bf Idx Bf Idx
The tenets of column-oriented model
• Keyspace Outer container, that contains column families (is sort of like a relational database)
• Column Family Logical division that associates similar data (very roughly analogous to tables in the relational world)
• Column Name/value pair (and a client-supplied timestamp of when it was last updated)
• Super Column Family Container for super columns sorted by their names
• Super Column Structure with name and set of dependent columns
Column Family\Column
Column FamilyA container for columns sorted by their names. Column Families are referenced and sorted by row keys.
row key
column name 1
column value 1
column name n
column value n
ColumnA name value pair (contains also a time-stamp for conflict resolution on the server side)
column name
column value
+ timestamp : long
: byte[]
: byte[]
row key
super column name 1super column name m
column name 1
column value 1
column name n1
column value n1
column name 1
column value 1
column name nm
column value nm
Super Column FamilyA container for super columns sorted by their names. Like Column Families, Super Column Families are referenced and sorted by row keys.
Super Column Family\Super ColumnSuper Column
A sorted associative array of columns.
column name 1
column value 1
column name n
column value n
super column name
row key
super column name 1super column name m
column name 1
column value 1
column name n1
column value n1
column name 1
column value 1
column name nm
column value nm
Addressing Super Column Family
• Five-dimensional hash• [Keyspace][ColumnFamily][Key][SuperColumn][SubColumn]
row key
column name 1
column value 1
column name n
column value n
Addressing Column Family
• Four-dimensional hash• [Keyspace][ColumnFamily][Key][Column]
Cassandra client optionsThrift (12 different languages)Avro (data serialization system)
Java: Hector: http://github.com/rantav/hector (abstraction over thrift) Pelops: http://github.com/s7/scale7-pelops (abstraction over thrift) CQL: JDBC driver for Cassandra version starting from 0.8 (SQL like language) Hector JPA: https://github.com/riptano/hector-jpa (ORM client) Cassandrelle: http://demoiselle.sf.net/component/demoiselle-cassandra / (documentation ???) Kundera: http://code.google.com/p/kundera / (buggy ???)
Python: Pycassa, TelephusGrails: grails-cassandra.NET: Aquiles, FluentCassandraRuby: CassandraPHP: phpcassa, SimpleCassie
Cassandra\RDBMS query differences
• No update query• Record-level atomicity on writes• No duplicate keys• Basic write properties: consistency level
(ZERO, ANY, ONE, QUORUM, ALL)• Basic read properties: consistency level (ONE,
QUORUM, ALL)
IntegratingHadoop (http://hadoop.apache.org) is a set of open source projects that deal with large amounts of data in a distributed way.• Hadoop Distributed File System (HDFS): a distributed file system that provides high-
throughput access to application data.• Hadoop MapReduce: a software framework for distributed processing of large data sets
on compute clusters.
Other Hadoop-related projects at Apache include:• Cassandra™: a scalable multi-master database with no single points of failure.• Hive™: a data warehouse infrastructure that provides data summarization and ad hoc
querying.• Mahout™: a Scalable machine learning and data mining library.• Pig™: a high-level data-flow language and execution framework for parallel computation.
The end
Questions?
Recommended