C* path

#CASSANDRAEU CASSANDRASUMMITEU

C* Path:Denormalize your data

Eric Zoerner | Software Developer, eBuddy BV Cassandra Summit Europe 2013 London

dinsdag 22 oktober 13


Topics

• About eBuddy

• Introducing C* Path

• How does it work?

• Design and Challenges

• Cassandra Data Model

• Futures



About eBuddy



XMS



Cassandra ineBuddy Messaging Platform

• User Data Service





• User Discovery Service






• Persistent Session Store







• Message History







• Message History

• Location-based Discovery



Some Statistics

• Current size of data– 1,4 TB total (replication of 3x); 467 GB actual data

• 16 million sessions (11 million users plus groups)

• Almost a billion rows in one column family(inverse social graph)



C* Path



The Problem (a “classic”)

Complex Object

name: Stringbirthdate: Datenickname: String

Person

street: Stringcity: Stringprovince: StringpostalCode: StringcountryCode: String

Address

*1

name: Stringnumber: String

Phone*

1

??

??

??

? ?

Key-Value Store(RDB table, NoSQL, etc.)



Some Strategies

Serialization!



Some StrategiesSerialization!

Normalization!

Personid

John

birthdate

Jack

1979-11-30

110 1985-04-06

Mary111 Mary

name nickname

person_id

001

003

street

New York

78 Hoofd Str

456 Singel

110 123 Main St

Amsterdam110 002

address_id city

London111

Address

person_id

mobile

mobile

phone

+44030393

+44884800

110 +15551234

111 home

name

111

Phone



Some StrategiesSerialization!

Normalization!

Decomposition!

Personid

John

birthdate

Jack

1979-11-30

110 1985-04-06

Mary111 Mary

name nickname

person_id

001

003

street

New York

78 Hoofd Str

456 Singel

110 123 Main St

Amsterdam110 002

address_id city

London111

Address

person_id

mobile

mobile

phone

+44030393

+44884800

110 +15551234

111 home

name

111

Phone

name/ John

addresses/@0/street 123 Main St.

phones/@0/number +31123456789

... ...



Strategies Comparison

✔ ✘ ✔

✔ ✘ ✔

✔ ✔

✘ ✔ ✔

✔ ✔ ✘

Serialization Normalization Decomposition

Single Write

Single Read

Consistent Updates not enforced

Structural Access

Cycles



C* Path

Open Source Java Library for decomposing complex objects into Path-Value pairs —and storing them in Cassandra

https://github.com/ ebuddy/c-star-path

* Artifacts available at Maven Central.


https://github.com/ebuddy/c-star-path



C* Path: Decomposition

• Easy to Use • Simple API





• Good for Cassandra because:

– Structural Access: Write parts of objects without reading first





• Good for Cassandra because:

– Structural Access: Write parts of objects without reading first

– Good for denormalizing data, can read or write large complex objects with one read or write operation



How does it work?



API Example - Write to a Path

StructuredDataSupport<UUID> dao = … ;UUID rowKey = … ;Pojo pojo = … ;





Path path = dao.createPath(“some”, “path”, ”to”,”my”,”pojo”);






dao.writeToPath(rowKey, path, pojo);



API Example - Read from a Path




API Example - Read from a Path


Pojo pojo = dao.readFromPath(rowKey, path, new TypeReference<Pojo>() { });



API Example - Delete

dao.deletePath(rowKey, path);



API Example - Batch Operations

BatchContext batch = dao.beginBatch();

dao.writeToPath(rowKey1, path, pojo1, batch);dao.writeToPath(rowKey2, path, pojo2, batch);dao.deletePath(rowKey3, path, pojo3, batch);

dao.applyBatch(batch);



Read or write at any level of a path

Person person = …;

Path path = dao.createPath(“x”);dao.writeToPath(rowKey, path, person);



Read or write at any level of a path

Person person = …;

Path path = dao.createPath(“x”);dao.writeToPath(rowKey, path, person);

Path pathToName = path.withElements(“name”);String name = dao.readFromPath(rowKey, pathToName, stringTypeReference);



Write Implementation: Decomposition

• Step 1:

– Convert domain object into basic structure of Maps, Lists, and simple values. Uses the jackson (fasterxml) library for this and honors the jackson annotations




• Step 1:


• Step 2:

– Decompose this basic structure into a map of paths to simple values (i.e. String, Number, Boolean), done by Decomposer




• Step 1:


• Step 2:

– Decompose this basic structure into a map of paths to simple values (i.e. String, Number, Boolean), done by Decomposer

• Step 3:

– Write this map as key-value pairs in the database



Example Decomposition - step 1

name: Stringbirthdate: Datenickname: String

Person

street: Stringcity: Stringprovince: StringpostalCode: StringcountryCode: String

Address

*1

name: Stringnumber: String

Phone*

1

Simplify structure into regular Maps, Lists, and simple values




Simplify structure into regular Maps, Lists, and simple values

Map

name = "John" birthdate = "-39080932298" nickname="Jack" addresses=<List>

[0] = <Map>

[1] = <Map>

street="Singel 45"

place="Amsterdam"

street="123 Main"

place="New York"

phones=<List>

[0] = <Map>

name="mobile"

number="+31651234567"



path value

name/ “John”

birthdate/ “-39080932298”

nickname/ “Jack”

addresses/@0/street “123 Main St.”

addresses/@0/place “New York”

addresses/@1/street “Singel 45”

addresses/@1/place “Amsterdam”

phones/@0/name “mobile”

phones/@1/number "+31651234567"




Read implementation: Composition

• Step 1:

– Read path-value pairs from database




• Step 1:


• Step 2:

– “Merge” path-value maps back into basic structure(Maps, Lists, simple values), done by Composer




• Step 1:


• Step 2:

– “Merge” path-value maps back into basic structure(Maps, Lists, simple values), done by Composer

• Step 3:

– Use Jackson to convert basic structure back into domain object using a TypeReference



Design & Challenges



Path Encoding

• Paths stored as strings

• Forward slashes in paths (but hidden by Path API)

• Path elements are internally URL encoded allowing use of special characters in the implementation

• Special characters: @ for list indices(@0, @1, @2, ...)



Challenge: “Shrinking Lists”

➀ Write a list.

x/@0/ “1”

x/@1/ “2”dao.writeToPath(key, “x”, {“1”,”2”});



➀ Write a list.➁ Write a shorter list.

x/@0/ “1”


x/@0/ “3”

x/@1/ “2”dao.writeToPath(key, “x”, {“3”});




➀ Write a list.➁ Write a shorter list.➂ Read the list.

x/@0/ “1”


x/@0/ “3”

x/@1/ “2”dao.writeToPath(key, “x”, {“3”});

dao.readFromPath(key, “x”, new TypeReference<List<String>>() {});

{“3”,”2”}


✘



Solution:Implementation writes a list terminator value.

x/@0/ “1”

x/@1/ “2”

x/@2/ 0xFFFFFFFF

dao.writeToPath(key, “x”, {“1”,”2”});

x/@0/ “3”

x/@1/ 0xFFFFFFFF

x/@2/ 0xFFFFFFFF

dao.writeToPath(key, “x”, {“3”});

dao.readFromPath(key, “x”, new TypeReference<List<String>>() {});

{“3”}


✔

✔



Solution:Implementation writes a list terminator value.


✔

Unfortunately, this is only a partial solution, because it is still possible to read “stale” list elements using a positional index in the path.

This can be avoided by doing a delete before a write, but for performance reasons the library will not do that automatically.

Conclusion: The user must know what they are doing and understand the implementation.



Challenge: Inconsistent UpdatesBecause objects can be updated at any path, there is no protection against a write “corrupting” an object

structure

x/address/street/ “Singel 45”

x/name/ “John”

Path path = dao.createPath(“x”);dao.writeToPath(key, path, person1);



Challenge: Inconsistent UpdatesBecause objects can be updated at any path, there is no protection against a write “corrupting” an object

structure


x/name/ “John”

Path path = dao.createPath(“x”);dao.writeToPath(key, path, person1);

path = dao.createPath(“x”,”name”);dao.writeToPath(key, path, person1);


x/name/ “John”

x/name/address/street/ “Singel 45”

x/name/name/ “John”✘



Challenge: Inconsistent Updates

Solution:Don’t do that!

✔

* If it does happen...

The implementation provides a way to still get the “corrupted” data as simple structures, but an attempt to convert to a now incompatible POJO will fail.

Conclusion: The user must know what they are doing and understand the implementation.



Issue: Sorting

Question:What about sorting path elements as something other than strings, such as numerical or time-based UUID elements?



Issue: Sorting


Instead of storing paths as strings, the implementation could have used DynamicComposite.



Issue: Sorting


Instead of storing paths as strings, the implementation could have used DynamicComposite.

We tried it.



Issue: Sorting


It can work. CQL supports it as a user-defined type.

Unfortunately it causes cqlsh to crash, making it difficult to “browse” the data.



Issue: Sorting


It is still in consideration to use DynamicComposite for paths in a future version.



Cassandra Data Model



Thriftx/address/street/ “Singel 45”

x/name “John”

… …

<UUID>

row key column name column value

column family

- OR -

super column family

(coming soon)

xxaddress/street/ “Singel 45”name “John”… …

<UUID>

row keysuper column name



Thrift

ColumnFamilyOperations<K,String,Object> operations = new ColumnFamilyTemplate<K,String,Object>( keyspace,KeySerializer,StringSerializer,StructureSerializer);

StructuredDataSupport<K> dao = new ThriftStructuredDataSupport<K>(operations);

Thrift implementation relies on the Hector client.



CQLCREATE TABLE person (

key text, path text, value text, PRIMARY KEY (key, path) )

• Cannot use the path itself as a column name because it is “dynamic”

• Dynamic column family



CQL: Data Model Constraints

• Need to do a range (“slice”) query on the path ⇒ path must be a clustering key

• Also, the path must be the first clustering key, since otherwise we would need to have to provide an equals condition on previous clustering keys in a query.

• One might try putting a secondary index on the path instead of making it a clustering key, but this doesn’t work since Cassandra indexes only work with equals conditionsBad Request: No indexed columns present in by-columns clause with Equal operator

CREATE TABLE person ( key text, path text, value text, PRIMARY KEY (key, path) )



CQL

StructuredDataSupport<K> dao = new CqlStructuredDataSupport<K>(String tableName, String partitionKeyColumnName, String pathColumnName, String valueColumnName, Session session);

CQL implementation relies on the DataStax Java driver.



And the rest…



Planned Features

• Sets with simple values: element values stored in path

• DynamicComposites?

• Multiple row reads and writes

• Slice queries on path ranges



Credits and Acknowledgements

• Thanks to Joost van de Wijgerd at eBuddy for his ideas and feedback

• jackson JSON Processor, which is core to the C* Path implementationhttp://wiki.fasterxml.com/JacksonHome

• Image credits:

Slide image name author link

Some Strategies binary noegranado http://www.flickr.com/photos/43360884@N04/6949896929/


http://wiki.fasterxml.com/JacksonHome

http://wiki.fasterxml.com/JacksonHome

http://www.flickr.com/photos/43360884@N04/6949896929/





C* Path

Open Source Java Library for decomposing complex objects into Path-Value pairs —and storing them in Cassandra

https://github.com/ ebuddy/c-star-path

* Artifacts available at Maven Central.




Technology

C* path