How companies-use-no sql-and-couchbase-10152013

How Companies use NoSQL and Couchbase

Dipti Borkar Director, Product Management

Anil KumarProduct Management

Two kinds of Database Management System

OLTP / OLTP like

Operational Stores

Data warehouse orAnalytics system

NoSQL + Big Data

Map-reduce against huge datasets to analyze and find insights and answers

Operational database for web and mobile apps with high performance at scale

Common Use CasesSocial Gaming

• Couchbase stores player and game data

• Examples customers include: Zynga

• Tapjoy, Ubisoft, Tencent

Mobile Apps

• Couchbase stores user info and app content

• Examples customers include: Kobo, Playtika

Ad Targeting

• Couchbase stores user information for fast access

• Examples customers include: AOL, Mediamind, Convertro

Session store

• Couchbase Server as a key-value store

• Examples customers include: Concur, Sabre

User Profile Store

• Couchbase Server as a key-value store

• Examples customers include: Tunewiki

High availability cache

• Couchbase Server used as a cache tier replacement

• Examples customers include: Orbitz

Content & Metadata Store

• Couchbase document store with Elastic Search

• Examples customers include: McGraw Hill

3rd party data aggregation

• Couchbase stores social media and data feeds

• Examples customers include: LivePerson

Use Case: High-Availability Caching

High availability caching

RDBMS

Application Layer

User Requests

Cache Misses and Write Requests

Read-Write Requests

Couchbase Distributed Cache

• Application objects

• Popular search query results

• Session information

• Heavily accessed web landing pages

Use Case: High-Availability Caching

Data Cached in Couchbase?

• Speed up RDBMS

• Consistently low response times for document / key lookups

• High-availability 24x7x365

• Replacement for entire caching tier

Application characteristic

Use Case: High-Availability Caching

• Low latency in sub-milliseconds with consistently high read / write throughput using built-in cache

• Always-on operations even for database upgrades and maintenance with zero down time

Why NoSQL?

Use Case: Session Store

Session Store

Use Case: Session Store

Data stored in Couchbase?

• Extremely fast access to session data using unique session ID

• Easy scalability to handle fast growing number of users and user-generated data

• Always-on functionality for global user base

Application characteristic

• Session values or Cookies (stored as key-value pairs)

• Examples include: items in a shopping cart, flights selected, search results, etc.

Use Case: Session Store

• Low latency in sub-milliseconds with consistently high read / write throughput for session data via the built-in object-level cache

• Linear throughput scalability to grow the database as user and data volume grow

• Always-on operations even particularly high availability using Couchbase replication and failover

• Intra cluster and cross cluster (XDCR) replication for globally distributed active-active platform

Why NoSQL?

Use Case: Globally Distributed User Profile Store

User ID / Profile Store

Use Case: Globally Distributed User Profile Store

Data stored in Couchbase?

• Extremely fast access to individual profiles

• Always online system as multiple applications access user profiles

• Flexibility to add and update user attributes

• Easy scalability to handle fast growing number of users

Application characteristic

• User profile with unique ID

• User setting / preferences

• User’s network

• User application state

Use Case: Globally Distributed User Profile Store

• Low latency and high throughput for very quick lookups for millions of concurrent users using built-in cache

• Intra cluster and cross cluster (XDCR) replication for high availability and disaster recovery

• Active-active geo-distributed system to handle globally distributed user base

• Online admin operations eliminate system downtime

Why NoSQL?

Use Case: Data Aggregation

Data Aggregation

Use Case: Data Aggregation

Data stored in Couchbase?

• Flexibility to store any kind of content

• Flexibility to handle schema changes

• Full-text Search across data set

• High speed data ingestion

• Scales horizontally as more content gets added to the system

Application characteristic

• Social media feeds: Twitter, Facebook, LinkedIn

• Blogs, news, press articles

• Data service feeds: Hoovers, Reuters

• Data form other systems

Use Case: Data Aggregation

• JSON provides schema flexibility to store all types of content and metadata

• Fast access to individual documents via built-in cache, high write throughput

• Indexing and querying provides real-time analytics capabilities across dataset

• Integration with ElasticSearch for full-text search

• Ease of scalability ensures that the data cluster can be grown seamlessly as the amount of user and ad data grows

Why NoSQL?

Use Case: Content and Metadata Store

Data Aggregation

Use Case: Content and Metadata Store

Data stored in Couchbase?

• Flexibility to store any kind of content

• Fast access to content metadata (most accessed objects) and content

• Full-text Search across data set

• Scales horizontally as more content gets added to the system

Application characteristic

• Content metadata

• Content: Articles, text

• Landing pages for website

• Digital content: eBooks, magazine, research material

Use Case: Content and Metadata Store

• Fast access to metadata and content via object-managed cache

• JSON provides schema flexibility to store all types of content and metadata

• Indexing and querying provides real-time analytics capabilities across dataset

• Integration with ElasticSearch for full-text search

• Ease of scalability ensures that the data cluster can be grown seamlessly as the amount of user and ad data grows

Why NoSQL?

McGraw Hill Education Labs Learning portal

Use Case: Content and metadata store

Building a self-

adapting, interactive learning

portal with Couchbase

As learning move online in great numbers

Growing need to build interactive learning environments that

Scale!

Scale to millions of learners

Serve MHE as well as third-party content

Including open content

Support learning apps

010100100111010101010101001010101010

Self-adapt via usage data

The Problem

• Allow for elastic scaling under spike periods

• Ability to catalog & deliver content from many

sources

• Consistent low-latency for metadata and stats access

• Require full-text search support for content

discovery

• Offer tunable content ranking & recommendation

functions

Backend is an Interactive Content Delivery Cloud that must:

XML Databases

SQL/MR Engines

In-memory Data Grids

Enterprise Search Servers

Experimented with a combination of:

The Challenge

Architecture

LivePerson – Real time visitor engagement

Use Case: 3rd party data aggregation with analytics

Real time Analytics for

LivePerson's customers

LiveEngage DASHBOARD

LivePerson: Leading customer engagement platform

Requirements Requirements Requirements

• High throughput, really fast

• Linear scale

• Searchable (Views and M/R)

• Supports both K/V & Document store

• Cross data center replication

• “Always on”, Resilience solution

The Problem

13 TBper month ~1 PB

In total 1.8 BVisits per month

VOLUME

Couchbase Java SDK

Application serverTomcat

M/R views

cluster

M/R views

cluster

XDCR

REST API

Couchbase Java SDK

Storm Topology

Couchbase Java SDK

Storm Topology

Architecture

Questions?

Thank you!

dipti@couchbase.com