Upload
dipti-borkar
View
797
Download
0
Tags:
Embed Size (px)
Citation preview
How Companies use NoSQL and Couchbase
Dipti Borkar Director, Product Management
Anil KumarProduct Management
Two kinds of Database Management System
OLTP / OLTP like
Operational Stores
Data warehouse orAnalytics system
NoSQL + Big Data
Map-reduce against huge datasets to analyze and find insights and answers
Operational database for web and mobile apps with high performance at scale
Common Use CasesSocial Gaming
• Couchbase stores player and game data
• Examples customers include: Zynga
• Tapjoy, Ubisoft, Tencent
Mobile Apps
• Couchbase stores user info and app content
• Examples customers include: Kobo, Playtika
Ad Targeting
• Couchbase stores user information for fast access
• Examples customers include: AOL, Mediamind, Convertro
Session store
• Couchbase Server as a key-value store
• Examples customers include: Concur, Sabre
User Profile Store
• Couchbase Server as a key-value store
• Examples customers include: Tunewiki
High availability cache
• Couchbase Server used as a cache tier replacement
• Examples customers include: Orbitz
Content & Metadata Store
• Couchbase document store with Elastic Search
• Examples customers include: McGraw Hill
3rd party data aggregation
• Couchbase stores social media and data feeds
• Examples customers include: LivePerson
Use Case: High-Availability Caching
High availability caching
RDBMS
Application Layer
User Requests
Cache Misses and Write Requests
Read-Write Requests
Couchbase Distributed Cache
• Application objects
• Popular search query results
• Session information
• Heavily accessed web landing pages
Use Case: High-Availability Caching
Data Cached in Couchbase?
• Speed up RDBMS
• Consistently low response times for document / key lookups
• High-availability 24x7x365
• Replacement for entire caching tier
Application characteristic
Use Case: High-Availability Caching
• Low latency in sub-milliseconds with consistently high read / write throughput using built-in cache
• Always-on operations even for database upgrades and maintenance with zero down time
Why NoSQL?
Use Case: Session Store
Session Store
Use Case: Session Store
Data stored in Couchbase?
• Extremely fast access to session data using unique session ID
• Easy scalability to handle fast growing number of users and user-generated data
• Always-on functionality for global user base
Application characteristic
• Session values or Cookies (stored as key-value pairs)
• Examples include: items in a shopping cart, flights selected, search results, etc.
Use Case: Session Store
• Low latency in sub-milliseconds with consistently high read / write throughput for session data via the built-in object-level cache
• Linear throughput scalability to grow the database as user and data volume grow
• Always-on operations even particularly high availability using Couchbase replication and failover
• Intra cluster and cross cluster (XDCR) replication for globally distributed active-active platform
Why NoSQL?
Use Case: Globally Distributed User Profile Store
User ID / Profile Store
Use Case: Globally Distributed User Profile Store
Data stored in Couchbase?
• Extremely fast access to individual profiles
• Always online system as multiple applications access user profiles
• Flexibility to add and update user attributes
• Easy scalability to handle fast growing number of users
Application characteristic
• User profile with unique ID
• User setting / preferences
• User’s network
• User application state
Use Case: Globally Distributed User Profile Store
• Low latency and high throughput for very quick lookups for millions of concurrent users using built-in cache
• Intra cluster and cross cluster (XDCR) replication for high availability and disaster recovery
• Active-active geo-distributed system to handle globally distributed user base
• Online admin operations eliminate system downtime
Why NoSQL?
Use Case: Data Aggregation
Data Aggregation
Use Case: Data Aggregation
Data stored in Couchbase?
• Flexibility to store any kind of content
• Flexibility to handle schema changes
• Full-text Search across data set
• High speed data ingestion
• Scales horizontally as more content gets added to the system
Application characteristic
• Social media feeds: Twitter, Facebook, LinkedIn
• Blogs, news, press articles
• Data service feeds: Hoovers, Reuters
• Data form other systems
Use Case: Data Aggregation
• JSON provides schema flexibility to store all types of content and metadata
• Fast access to individual documents via built-in cache, high write throughput
• Indexing and querying provides real-time analytics capabilities across dataset
• Integration with ElasticSearch for full-text search
• Ease of scalability ensures that the data cluster can be grown seamlessly as the amount of user and ad data grows
Why NoSQL?
Use Case: Content and Metadata Store
Data Aggregation
Use Case: Content and Metadata Store
Data stored in Couchbase?
• Flexibility to store any kind of content
• Fast access to content metadata (most accessed objects) and content
• Full-text Search across data set
• Scales horizontally as more content gets added to the system
Application characteristic
• Content metadata
• Content: Articles, text
• Landing pages for website
• Digital content: eBooks, magazine, research material
Use Case: Content and Metadata Store
• Fast access to metadata and content via object-managed cache
• JSON provides schema flexibility to store all types of content and metadata
• Indexing and querying provides real-time analytics capabilities across dataset
• Integration with ElasticSearch for full-text search
• Ease of scalability ensures that the data cluster can be grown seamlessly as the amount of user and ad data grows
Why NoSQL?
McGraw Hill Education Labs Learning portal
Use Case: Content and metadata store
Building a self-
adapting, interactive learning
portal with Couchbase
As learning move online in great numbers
Growing need to build interactive learning environments that
Scale!
Scale to millions of learners
Serve MHE as well as third-party content
Including open content
Support learning apps
010100100111010101010101001010101010
Self-adapt via usage data
The Problem
• Allow for elastic scaling under spike periods
• Ability to catalog & deliver content from many
sources
• Consistent low-latency for metadata and stats access
• Require full-text search support for content
discovery
• Offer tunable content ranking & recommendation
functions
Backend is an Interactive Content Delivery Cloud that must:
XML Databases
SQL/MR Engines
In-memory Data Grids
Enterprise Search Servers
Experimented with a combination of:
The Challenge
Architecture
LivePerson – Real time visitor engagement
Use Case: 3rd party data aggregation with analytics
Real time Analytics for
LivePerson's customers
LiveEngage DASHBOARD
LivePerson: Leading customer engagement platform
Requirements Requirements Requirements
• High throughput, really fast
• Linear scale
• Searchable (Views and M/R)
• Supports both K/V & Document store
• Cross data center replication
• “Always on”, Resilience solution
The Problem
13 TBper month ~1 PB
In total 1.8 BVisits per month
VOLUME
Couchbase Java SDK
Application serverTomcat
M/R views
cluster
M/R views
cluster
XDCR
REST API
Couchbase Java SDK
Storm Topology
Couchbase Java SDK
Storm Topology
Architecture
Questions?
Thank you!