Cassandra 2.1 boot camp, Protocol, Queries, CQL

©2014 DataStax

Tyler Hobbs

Protocol, Queries, and Cell Names

1

[email protected] @tylhobbs

Friday, September 12, 14

mailto:[email protected]


Native Protocol•Driver <=> Cassandra Communications•doc/native_protocol_v3.spec


Native Protocol•Handshake•Set protocol version, compression, auth, etc


Native Protocol•Multiplexed operations•Unique request ID per-operation per-connection•Responses can come out of order•Pushed notifications from the server

driver c*123

213


Native Protocol•PREPARE•EXECUTE•QUERY•BATCH


Statement Preparation•Driver sends PREPARE with query•C* replies with:• Statement ID• Query parameter metadata• Result set metadata

•Driver prepares against all nodes it will use


Lexing and Parsing•ANTLR•Cql.g is the grammar•Minimal checking of statement validity•Creates “Raw” statements and terms• No type checking yet


Raw -> Prepared•Term.Raw -> Term.(Non)Terminal• Terminal is fully processed, nothing to do after preparing

• String literals, int literals, etc• Pure function calls (e.g. intAsBlob(42))

• NonTerminal has bind markers or non-pure function calls• Example: now()

•Type checking performed where possible•Term.Raw.prepare()


Raw -> Prepared•Statement.Raw -> ParsedStatement.Prepared• Is the query valid and supported?• Organize restrictions on partitioning, clustering and

normal columns• Check validity of restrictions, secondary indexes,

ordering, filtering, etc.• SelectStatement.Raw.prepare()


Raw -> Prepared•ParsedStatement.Prepared is cached, ID returned

to driver•QueryProcessor


Statement Execution•EXECUTE with ID and binary parameters


Statement Execution•Check permissions• QueryProcessor.processStatement -> Statement.checkAccess()

•Check Statement Validity•Bind parameters• Term.NonTerminal -> Term.Terminal• Evaluate functions• Check parameter types


Statement Execution•Build slice ends for reads• Pick start, end based on reversal• Are we getting multiple slices?• SelectStatement.makeFilter()


Statement Execution•Set up paging• Use last seen primary key as starting point• Paging state held by driver


Statement Execution•Pass command to StorageProxy• StorageProxy.read(), mutate(), cas()

• Dispatched to replicas• Low-level cells returned


Statement Execution•Process Results• Order columns correctly• Filter empty rows• Order rows• Trim excess rows to match LIMIT• Aggregate (count(), for now)


Statement Classes•SchemaAlteringStatement• CREATE/ALTER/DROP KEYSPACE, etc

•ModificationStatement• INSERT, UPDATE, DELETE

•SelectStatement


SchemaAlteringStatement•Update local schema•Broadcast to all live nodes•Notify drivers with pushed notification


ModificationStatement•When doing LWT/CAS• Prepare -> Read -> Propose -> Commit• Build slice ends for read, execute read• Check conditions against read row• If CAS fails, return existing row as result


Cells•Subject to change for 3.0•Name/value pair

• Sorted by name


CellsCREATE TABLE versioned_docs (id uuid,

version timeuuid,

fieldname text,

fieldvalue blob,

PRIMARY KEY (id, version, fieldname))

•id is the partitioning key•version and fieldname are clustering columns•Within a partition, rows are ordered by clustering

columns


Sparse Format•CQL3 Format•Cell name format:

• (clustering_1, ..., clustering_n, column_name)


Sparse FormatCREATE TABLE versioned_docs (id uuid,

version timeuuid,

fieldname text,

fieldvalue blob,




version timeuuid,

fieldname text,

fieldvalue blob,


INSERT INTO ... VALUES( D5B...0A5, 3AF4...D93, ‘title’, ‘Monthly Report’)

(3AF4..D93, 'title', 'fieldvalue')


Sparse Format•Add one extra component for collections:

• (clustering_1, ..., column_name, collection_element)



version timeuuid,

fields map<text, blob>,

PRIMARY KEY (id, version))



version timeuuid,

fields map<text, blob>,

PRIMARY KEY (id, version))

INSERT INTO ... VALUES ( D5B...0A5, 3AF4...D93, {‘title’: ‘Monthly Report’})

(3AF4..D93, 'fields', 'title')


Sparse Format• If there are no clustering columns, the cell name

is just the column name

CREATE TABLE users ( id uuid PRIMARY KEY, name text, age int)

•Cell names:(‘name’) and (‘age’)


Sparse Format• If COMPACT STORAGE is used, we drop the

composite

CREATE TABLE users ( id uuid PRIMARY KEY, name text, age int)WITH COMPACT STORAGE

•Cell names: ‘name’ and ‘age’


Dense Format•Used for COMPACT STORAGE tables with

clustering columns

CREATE TABLE sensor_readings ( id uuid, time timestamp, value float, PRIMARY KEY (id, time))WITH COMPACT STORAGE


Dense Format•Don’t store column name in cell name

CREATE TABLE sensor_readings ( id uuid, time timestamp, value float, PRIMARY KEY (id, time))WITH COMPACT STORAGE

INSERT INTO ... VALUES ( D5B...0A5, 1410279227535, 0.64)

1410279227535


Dense Format•Multiple clustering keys result in composite cell

nameCREATE TABLE sensor_readings ( id uuid, time timestamp,

attribute text, value float, PRIMARY KEY (id, time, attribute))WITH COMPACT STORAGE

•Cell name: (1410279227535, ‘temperature’)


Formats•Sparse Composite

• CQL3 tables

•Sparse Simple• “Static” compact storage, no clustering columns• e.g. “users” table

•Dense Simple• compact storage w/ one clustering column• e.g. sensor readings

•Dense Composite• compact storage w/ multiple clustering columns• e.g. sensor readings w/ multiple fields


Internal Classes• CellName and CellNameType

• Sparse/Dense

• Simple/Composite


Questions?

@tylhobbs

[email protected]




Technology

Cassandra 2.1 boot camp, Protocol, Queries, CQL