Couchbase Sydney Meetup
Couchbase Architecture and ScalabilityKarthik Babu Sekar
SolutionsArchitect,WWFieldOperations@Couchbase- LinkedIn - https://www.linkedin.com/in/karthikbabusekar
- Twitter- https://twitter.com/SKBhere- E-mail – [email protected]
©2016 Couchbase Inc.©2015 Couchbase Inc.
History of Couchbase
2
©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase provides a complete Data Management solution
3
Highavailabilitycache
Key-valuestore
Documentdatabase
Embeddeddatabase Syncmanagement
Multi-purpose capabilities support a broad range of apps and use cases
Enterprises often start with cache, then broaden usage to other apps and use cases
©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase meets today’s & tomorrow’s requirements
4
Flexible data model
Consistent performance at scale
High availability
Easy,affordablescalability
24x365
©2016 Couchbase Inc.©2015 Couchbase Inc.
Common Use-Cases
5
360DegreeCustomerView
ProfileManagement
Catalog FraudDetection
ContentManagement
InternetofThings
DigitalCommunication
RealTimeBigData
MobileApplications
Personalization
©2016 Couchbase Inc.©2015 Couchbase Inc.
What makes Couchbase unique?
6
Performance & scalability leaderSubmillisecondlatencywithhighthroughput;memory-centricarchitecture
Multi-purpose
Enterprise class Web Console
Easytodeploy&manage;integratedAdminConsole,single-clickclusterexpansion&rebalance
Cache,keyvaluestore,documentdatabase,andlocal/mobiledatabaseinsingleplatform
Always-on availability
Datareplicationacrossnodes,clusters,anddatacenters
Enterprises choose Couchbase for several key advantages
24x365
©2016 Couchbase Inc.©2015 Couchbase Inc.
§ Consolidatedcacheanddatabase
§ Tunememoryrequiredbasedonapplicationrequirements
Multi-purpose database supports many uses
77
Tunable built-in cache
Flexible schemas with JSON
Couchbase Lite
§ RepresentdatawithvaryingschemasusingJSONontheserveroronthedevice
§ IndexandquerydatawithJavascript views
§ LightweightembeddedDBforalwaysavailableapps
§ SyncGatewaysyncsdataseamlesslywithCouchbaseServer
©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase delivers always-on availability
8
High Availability
Disaster Recovery
Backup&Restore
§ In-memoryreplicationwithmanualorautomaticfailover
§ Rack-zoneawarenesstominimizedataunavailability
§ Memory-to-memorycrossclusterreplicationacrossdatacentersorgeos
§ Active-activetopologywithbi-directionalsetup
§ FullbackuporIncrementalbackupwithonlinerestore
§ Deltanodecatch-upsforfasterrecoveryafterfailures
24x365
©2016 Couchbase Inc.©2015 Couchbase Inc.
Simplified administration for exceptional ease of use
Online upgrades and operations
Built-in enterprise class admin console
Restful APIs
§ Onlinesoftware,hardwareandDBupgrades
§ Indexing,compaction,rebalance,backup&restore
§ Performalladministrativetaskswiththeclickofabutton
§ Monitorstatusofthesystemvisualatclusterlevel,databaselevel,serverlevel
§ AlladminoperationsavailableviaUI,RESTAPIsorCLIcommands
§ IntegratethirdpartymonitoringtoolseasilyusingREST
©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase leads in performance and scalability
Auto Sharding Memory-memory XDCR
MultiDimensionalScaling
§ Nomanualsharding
§ Databasemanagesdatamovementtoscaleout– nottheuser
§ Market’sonlymemory-to-memorydatabasereplicationacrossclustersandgeos
§ Providesdisasterrecover/datalocality
§ Hugelysimplifiesmanagementofclusters
§ Easytoscaleclustersbyaddinganynumberofnodes
Service-BasedArchitecture
Multi-Dimensional Scalability (MDS) – Optionally Scale each service independently:• Data• Index• Query
MDS is the architecture that enables independent
scaling of data, query, and indexing workloads while
being managed as one cluster.
©2015 Couchbase Inc.
Full Cluster Architecture
12
STORAGE
Couchbase Server 1
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 2
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 3
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 4
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 5
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 6
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService
©2015 Couchbase Inc.
Full Cluster Architecture
13
STORAGE
Couchbase Server 1
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 2
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 3
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 4
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 5
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService STORAGE
Couchbase Server 6
SHARD7
SHARD9
SHARD5
SHARDSHARDSHARD
Managed Cache
Cluster ManagerClusterManager
ManagedCache
Storage
DataService
IndexService
QueryService
Couchbase Architecture: Single Node
14
High level architecture - Single Node
CouchbaseServernodesareidentical
Twocorecomponentsü ClusterManagerü DataManager
Threeindependentservicesü DataServiceü IndexServiceü QueryService
CouchbaseServerNode
DataManager ClusterManager
ManagementRESTAPIWebUI
Erlang/OTP
Node/Cluster
Coordination
DataService
Storage
ManagedCache
Get/Set
MapReduceViews
QueryService
QueryEngine
QueryPlanCache
Storage
IndexService
IndexingEngine
ManagedCache
Storage
Cluster ManagerManagementandcoordination
Responsibleforü UI/RESTinterfaceü Configuration/Administrationü Coordinatingclusterrebalancingü Processmonitoringü Statisticsü No dataprocessing
Erlang/OTPcodebaseü DevelopedbyEricssonü Telco-switchgradeperformance
CouchbaseServerNode
ClusterManager
ManagementRESTAPIWebUI
Erlang/OTP
Node/Cluster
Coordination
DataManager
DataService
Storage
ManagedCache
Get/Set
Map-ReduceViews
QueryService
QueryEngine
QueryPlanCache
Storage
IndexService
IndexingEngine
ManagedCache
Storage
:8091 AdminConsole/API
Node Manager
Copyright © 2015 Couchbase, Inc.
Threesupportedworkloads
DataServiceü Get/Setofindividualdocumentsü Distributedsecondaryindexes(Views)IndexServiceü Globalsecondaryindexesü IndexmaintenanceandprovisionQueryServiceü N1QL(SQLsupersetforJSON)ü Executionplans,indexcoordination
C/C++andGocodebasesü Veryfastandmemoryefficient
evenwithlargememoryfootprint CouchbaseServerNode
ClusterManager
ManagementRESTAPIWebUI
Erlang/OTP
Node/Cluster
Coordination
DataManager
DataService
Storage
ManagedCache
Get/Set
Map-ReduceViews
QueryService
QueryEngine
QueryPlanCache
Storage
IndexService
IndexingEngine
ManagedCache
Storage
Singlenodetype/codebaseü Allservicesavailableoneverynodeü Servicesenabledindependently
Allowsclustertobeconfiguredforü Varyinghardwarecapacitiespernodeü Applicationworkloadtuning
CouchbaseServerNode
ClusterManager
ManagementRESTAPIWebUI
Erlang/OTP
Node/Cluster
Coordination
DataManager
QueryService
DataService
IndexService
IndexService
IndexingEngine
ManagedCache
Storage
QueryService
QueryEngine
QueryPlanCache
Storage
DataService
Storage
ManagedCache
Get/Set
Map-ReduceViews
Multi-Dimensional Scalability
Copyright © 2015 Couchbase, Inc.
©2015CouchbaseInc. 19
CouchbaseDataAccess
• Everything is built on top of Key Value
• A Document store is a special case of Key-Value
• Views provide aggregation and real-time analytics through incremental map-reduce
• Global Secondary Indexes provide low latency/high throughput indexes
• N1QL is a language that provides a powerful and expressive way of accessing documents
Key Differentiators - N1QL
N1QL is a Next-generation NoSQL query language for JSON.
20
§ JOINS§ Powerful Extensions for JSON § ODBC/JDBC drivers available
Introduction to N1QL
SQL query language for documents
• SELECT * FROM WHERE/LIKE/JOIN/GROUP/etc, CREATE INDEX
• Extended for JSON to support nested and hierarchical data structures
• Support for map-reduce views and Global Secondary Indexes
• Query (DQL), Manipulation (DML), Description (DDL)
• ODBC/JDBC drivers
SQL 4 documents - examples
2222
Listingproductsinacategory Findingthemostpopularproductsinacategory
GlobalSecondaryIndexes
NewStorageOptionsforGlobalSecondaryIndex1. Memory-OptimizedGlobalSecondaryIndexes(MOI)
• OptimizedforMemory:smallmemoryfootprint,optimizedforlowestlatencyqueries• FasterIndexing:freshindexesunderheavymutationswithlockfreeindexmaintenance• PredictablelowlatencywithQueryandKeyBasedOperations,eveninpresenceoflarge
numberofindexes
1. StandardGlobalSecondaryIndexes• OptimizedforStorage+Memory:efficientspilltodiskwithForestDBasthestorage
engine• AdvancedIOPerformance: newcircularwritesthatminimizecompactionoverheadof
appendonlywrites
©2016CouchbaseInc.
Mobile
Search
CacheOLAP
Data Warehouse
Hadoop
RDBMS
©2016CouchbaseInc.
Mobile
Search
CacheOLAP
Data Warehouse
Hadoop
RDBMS
©2016CouchbaseInc.
Search
OLAP
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
Search
OLAP
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
OLAP
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
INTRODUCING SEARCH
Structured Query AND SearchOne platform.
|
Search Feelinglucky?
©2016CouchbaseInc.
OLAP
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
OLAP
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
INTRODUCING REALTIME ANALYTICS
You may also like these products
Thank you for your order!Order #: 3423943
Paid Partner Content
Operations & Analytics. One platform.
©2016CouchbaseInc.
OLAP
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
Data Warehouse
Hadoop
{N1QL}
©2016CouchbaseInc.
INTRODUCING REALTIME ANALYTICS
You may also like these products
Thank you for your order!Order #: 3423943
Paid Partner Content
Operations & Analytics. One platform.
©2016CouchbaseInc.
Data Warehouse
Hadoop
THE PLATFORM FOR YOUR DIGITAL APPLICATIONS
©2016CouchbaseInc.
THE DATA PLATFORM FOR THE DIGITAL ECONOMY
Unified Adminstration
Unified Programming Interface
Data Query Index SearchMobileReplication Analytics
{N1QL}
XDCR: Cross Data Center Replication
Application can access both clusters (master – master)Scales out linearlyDifferent from intra-cluster replication (“CP” versus “AP”)
XDCR: Flexible topologies
One-one, one-many, many-oneDifferently sized and resourced clusters supported
Couchbase Architecture: Mobile
43
©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase Mobile Overview
©2016 Couchbase Inc.©2015 Couchbase Inc.
Always Available and Fast
45
©2016 Couchbase Inc.©2015 Couchbase Inc.
Native Support
46
©2016 Couchbase Inc.©2015 Couchbase Inc.
Couchbase Mobile Solution
47
Key Take-aways
Couchbase Mobile = Couchbase Lite + Couchbase Sync Gateway + Couchbase Server
Couchbase Lite: Only NoSQL database for mobile devices (phones, tables, embedded systems)
Couchbase Sync Gateway: Offline/Online synchronization
48
©2016 Couchbase Inc.©2015 Couchbase Inc.
Backup and Restore with Couchbase§ Zer0-downtimebackupandrestore
§ Built-inutilities:cbbackup /cbrestore§ Full,differentialandcumulativebackupavailableper-bucket.§ Restorefromanypoint,toanybucketortopology
Demo & Q&A