Building Blocks for the Future of Television
DataStax Webinar 5/6/2014Boris Wolf
Driving Innovation at CSV
Evangelize
InfrastructureCross Platform
Services
ExpandTV Platform / X1
ExtendHome Platform
Demonstrate
Syndicateto other platforms
Experiment
An entirely new cloud-based, personalized, and interactive TV experience.
X1 Platform Overview
Delivering anengaging, interactive, and immersive sports viewing experience.
X1 Sports App
Some Examples of C* Usage in X1
• DVR (recording history etc.)• Personalization• Preferences• CMB (message bus)
CMB – Cloud Message Bus
• CMB = CQS + CNS• An Open Source General Purpose High Throughput
Distributed Message Bus Infrastructure for the Cloud• Queuing and Notification Services• API compatible with AWS SQS/SNS• Built on C* and Redis
Receive()
Receive()
MM MReceive()
CQS – Queuing Service
Send(M)
M
M
T
http://…
http://…
http://…
…
…
CQS
SQS
CQS
Publish(M)
M
M
M
M
M
M
SubscribersCNS – Notification Service
CMB – Cloud Message Bus
• CMB = CQS + CNS• An Open Source General Purpose High Throughput
Distributed Message Bus Infrastructure for the Cloud• Queuing and Notification Services• API compatible with AWS SQS/SNS• Built on C* and Redis
CMB – Tradeoffs
• Guaranteed delivery (even if recipients loose messages!)
• Best effort ordering• Duplicates are possible on occasion
t=0
Receive()
t=0
VTO=30sec
Process()
t=5
VTO=25secDelete()
t=5
t=0
Receive()
Receive()
t=0
VTO=30sec
Receive()
Process()
t=5
VTO=25sec
Process()
Receive()
t=5
VTO=25sec
Receive()
t=25
VTO=5sec
Receive()
t=30
Receive()
t=30
VTO=30secProcess()
t=35
Delete()
VTO=25sec
t=35
SQS / SNS API CompatibilityAmazonSQSClient sqs = new AmazonSQSClient(new BasicAWSCredentials("Z2DV…", "bH2U…."));
sqs.setEndpoint("http://localhost:6059"); // create queue String queueUrl = sqs.createQueue(new CreateQueueRequest("myqueue")).getQueueUrl(); // send message sqs.sendMessage(new SendMessageRequest(queueUrl, "my message")); // receive messages List<Message> messages = sqs.receiveMessage(new ReceiveMessageRequest(queueUrl)).getMessages(); for (Message m : messages) {
System.out.println(m.getBody());}
CMB Web UI
• We actually do: As fail-over option, for proto-typing, …
• CMB as Open Source alternative to avoid vendor lock-in
Why not just use AWS SQS/SNS?
CMB – Cloud Message Bus
• CMB = CQS + CNS• An Open Source General Purpose High Throughput
Distributed Message Bus Infrastructure for the Cloud• Queuing and Notification Services• API compatible with AWS SQS/SNS• Built on C* and Redis
CQS Implementation - C* and RedisCMB API Nodes• Embedded Jetty implementing SQS and SNS APIsCassandra• Cross-DC persistence and replication• Proven horizontal scalabilityRedis• Avoid duplicates• Help with best effort ordering• Handle Visibility Timeout (VTO)
Case Study X1 DVR
Scheduler WS
Recorder WS X1 STB
XRE
CQS
CQS
XRE
Worker Service
X1 STBX1 STB
Deployment details:• 2 DC Deployment, 8-Node C* Ring in each (local
quorum reads and writes)• 10 Redis Shards (each DC)• 12 CMB Nodes (each DC)
Case Study X1 DVR
SQS APIs Params
SendMessage() MessageBody=…
ReceiveMessage() MaxNumberOfMessages=10VisibilityTimeout=4WaitTimeSeconds=20
DeleteMessage() ReceiptHandle=1383970619866:0:ed006cabb22dcc80e44f1967ac259f3b_0_65:2902396753389907544:-7877921303127914669
Case Study X1 DVR
Case Study X1 DVR
• Follow SQS/SNS APIs as they evolve (Dead Letter Queue, Mobile Push)
• C* 2.0 features, CQL3, binary protocol drivers • Make operations / scaling / capacity planning
easier• Offer CMB as a Service
Opportunities for Improvement
Thank You!
http://github.com/Comcast/cmb
http://groups.google.com/forum/#!forum/cmb-user-forum
Recommended
