View
444
Download
0
Category
Preview:
DESCRIPTION
Presenters: Tammer Saleh, Director of Product, Cloud Foundry Services at Pivotal Pivotal is dedicated to bringing best-of-breed data services to Pivotal CF, and there is no other open source data technology with as much potential as Cassandra. We’ll discuss the strategies and techniques for deploying and managing a multi-user Cassandra installation that integrates with Cloud Foundry. - Making Cassandra manage itself - Single-tenant versus Multi-tenant usage - Deploying Cassandra with BOSH - Cloud Foundry services architecture.
Citation preview
Pivotal #CassandraSummit
Cassandra on Pivotal CF
Pivotal #CassandraSummit
Tammer SalehDirector of Product
Pivotal CF Services, London !
http://tammersaleh.com
Overview of Pivotal CF!Services API!
Why Cassandra!Building Cassandra!
Automating Cassandra Operations
Pivotal #CassandraSummit
Overview of Pivotal CF
Pivotal #CassandraSummit
$ cf push is golden.
$ cf create-service p-cassandra development-plan eventsdb $ cf bind-service eventsdb dashboard $ cf push dashboard $ open https://dashboard.mypcf.com
Pivotal #CassandraSummit
Pivotal #CassandraSummit
Pivotal #CassandraSummit
vSphere
BOSH provisioning / configuration / orchestrationOperations
Manager
AWS OpenStack
Console
Service Foo
Broker DB
Service Bar
Broker DB
Service Baz
Broker DB
Runtime
DEA Cloud Controller
Router
DEA
DEA DEA
Health Manager
SSO
Services APIPivotal #CassandraSummit
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Application Instances
Cloud Controller
Router
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Cloud Controller
Router
Service
Service Instances
Service Broker
Application Instances
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Cloud Controller
Router
Service
Service Instances
Service Broker
Application Instances
Catalog
$ cf marketplace
vSphere / AWS / OpenStack
BOSHOperations Manager
CF Runtime
Cloud Controller
Router
Service
Service Instances
Service Broker
Application Instances
Provision instance
Done
$ cf create service foo
CF Runtime
Application Instances
Cloud Controller
Router
Service
Service Instances
Service Broker
Provision binding
Binding info
$ cf bind service my_app foo
$VCAP_SERVICES={ "pgsql": [ { "name": "UUID", "plan": "small", "credentials": { "uri": “postgres://u:p@service_ip” } } ], … }
Binding info
CF Runtime
Application Instances
Cloud Controller
Router
Service
Service Instances
Service Broker
$ cf bind service my_app foo
App instance reads and writes directly to
service instance.
GET /v2/catalog What’s available?
PUT /v2/service_instances/:id Create service
DELETE /v2/service_instances/:id Delete service
PUT /v2/service_instances/:id/service_bindings/:id Create binding
DELETE /v2/service_instances/:id/service_bindings/:id Delete binding
Pivotal CF Services
Insanely fast KV storage - The AK-47 of Databases
The World's Leading Graph DB
Distributed, real-time search and analytics engine
Simple and Fast NoSQL DB
High-performance memory object cache.
Next generation MySQL replacement.
Enterprise Hadoop Distribution.
S3 Compatible Blobstore
Highly distributed, write-heavy KV/column store.
Our first production-grade service.
"In terms of scalability, there is a clear winner throughout our experiments.
Cassandra achieves the highest throughput for the maximum number of
nodes in all experiments."
- University of Toronto, 2012
Why ?
Why ?
Cassandra is truly cloud-scale.
Why ?
Why ?
Multi-datacenter deployments are commonplace in
the cloud.
Building Cassandra
Pivotal #CassandraSummit
Pivotal #CassandraSummit
Plan InstanceService Binding
Services API Redux
Pivotal #CassandraSummit
InstanceWhat is an ?
InstanceWhat is an ?
Scales to the size of a single cluster.!
Single Keyspace.!
Good for development and testing.
Pivotal #CassandraSummit
Part of a Shared Cluster
VM VM VM
Cassandra Node Cassandra Node Cassandra Node
Instance 1 (Keyspace)
Instance 2 (Keyspace)
Instance 3 (Keyspace)
Keyspace visibilityPivotal #CassandraSummit
Noisy Neighbours
Pivotal #CassandraSummit
Quotas
Pivotal #CassandraSummit
InstanceWhat is an ?
True production grade.!
No noisy neighbours.!
Expensive.
Pivotal #CassandraSummit
Cluster of VMs
Instance 1 (Cluster of VMs)
Cassandra VM Cassandra VMCassandra VM
Instance 2 (Cluster of VMs)
Instance 3 (Cluster of VMs)
Cassandra VM Cassandra VMCassandra VM
Cassandra VM Cassandra VMCassandra VM
InstanceWhat is an ?
Looks identical to production.!
Fairly good noisy neighbour isolation.!
Constrained resources per instance.
Pivotal #CassandraSummit
Cluster of Containers
VM VM VM
Instance 1 (Cluster)
Cassandra Node
Instance 2 (Cluster)
Instance 3 (Cluster)
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Cassandra Node
Automating Cassandra Operations
Pivotal #CassandraSummit
BOSH
Pivotal #CassandraSummit
BOSH is…• Predictable!• Repeatable!• IaaS agnostic!• Built for large-scale deployments
Pivotal #CassandraSummit
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent
hello!
hey !there!
ahoy!
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent IaaS
STONITH!
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
VM
Agent IaaS
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
IaaS
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
IaaS
Provision!
BOSH Resurrector
Pivotal #CassandraSummit
BOSH Director
VM
Agent
VM
Agent
IaaS
VM
Agenthowdy!
NTPd
Pivotal #CassandraSummit
Pivotal #CassandraSummit
NTPd!Accuracy vs Precision
Pivotal #CassandraSummit
NTPd!!
https://blog.logentries.com/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-2-solutions/
Cassandra Node
Cassandra Node
Cassandra Node
Pivotal #CassandraSummit
NTPd!!
https://blog.logentries.com/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-2-solutions/
Cassandra Node
Cassandra Node
Cassandra Node
NTPd!Server
Pivotal #CassandraSummit
NTPd!!
https://blog.logentries.com/2014/03/synchronizing-clocks-in-a-cassandra-cluster-pt-2-solutions/
Cassandra Node
Cassandra Node
Cassandra Node
NTPd!Server
Internet
NTPd!Server
NTPd!Server
NTPd!Server
Running repairs
Pivotal #CassandraSummit
Running repairs
Pivotal #CassandraSummit
1. When decommissioning the Node.
2. When the node has been down for N minutes.
3. Once per node per week within GC_GRACE_PERIOD
Services Are the Key to the Success of
Pivotal CF
Pivotal #CassandraSummit
Tammer Salehtsaleh@pivotal.io
http://tammersaleh.com
Thank you.
Pivotal #CassandraSummit
If this interests you, we’re hiring CF engineers in SF and London.
Recommended