Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Solving real-time IoT challenges:

Couchbase Connect 2016Mohanraj Umapathy,

Verizon Labs, Director – IoT Platform

Couchbase at Verizon Labs

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Agenda

• IoT overview

• Verizon solutions for IoT/ThingSpace

• Verizon Labs real-time data pipeline

• Benefits of Couchbase

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

A new definition of IoT

3

IoT is transforming businesses through an aware and autonomous network

that provides actionable insight and better business intelligence.

AwareThe asset must be able

to sense something.

AutonomousThe asset must send data

somewhere automatically.

ActionableThe data must

inform decisions.

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

4

1. IDC, Worldwide Internet of Things Spending by Vertical Market 2015–2018 Forecast, IDC #256255, June 2015.

Market adoption

Growing faster in some

industries than others

• Manufacturing, utilities and

transportation

are leading.

• Retail, healthcare and

consumer are catching up

fast.1

IoT worldwide

installed base

• 10.3 billion in 2014

• 29.5 billion in 20201

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or

distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 5

How are you going to make sure your

IoT implementation succeeds?

Put it together correctly • Successful IoT applications require

integrated system design, security

and management.

Keep it running smoothly • A complete solution involves ongoing

development, deployment, integration

and device management.

Harvest the reward efficiently • To extract business value, you need to

cost-effectively store, analyze and make

sense of collected data.

Overcoming the challenges

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

ThingSpace IoT Ecosystem

6

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

ThingSpace IoT

ThingSpace platformVerizon network

End-to-end device integration and management

ApplicationsDevices

Connectivity

management

Device

management

Reporting and

analytics

Security and

compliance

Ze

ro-t

ou

ch

on

bo

ard

ing A

PIs

an

d d

eve

lop

er

po

rtal

Wired access

LTE/5G

Traditional and

IoT cores

7

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Real-Time Intelligent Data Pipeline

Data Ingestion Insights

Device

Sensor

Cloud

Streaming Data

Event Processing Pipeline

Visualization

AI Devices

Map

Analytics

Rules

Action

Storage

Machine Learning

Transformation

Filter

Relationship

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Verizon Labs IoT/ThingSpace & Couchbase

Problem

Solved

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Breakoutwith Mohan Umapathy

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

• Introduction

• Why Couchbase?

• Solving IoT challenges in active-active data centers

• Lesson learned with Couchbase in production

• POC: Couchbase, Mesos, Docker and Spark – 3M ops/sec

Agenda

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Introduction

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Mohanraj UmapathyVerizon Labs, Director – IoT Platform

Mohan leads the Internet of Things (IoT) Platform

team that develops highly available and scalable

IoT platform services at Verizon Labs.

Prior to joining Verizon Labs, Mohan was Director

at Qualcomm leading the IoT Platform effort.

Mohan spent most of his career developing fault-

tolerant distributed messaging systems and rule

engines for IoT use cases.

Mohan enjoys playing chess and sports that his

kids sign up for.

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or

distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 14

• Mohanraj Umapathy

• Yifeng Chen

• Abhishek Jamloki

• Steven Le

Team Credits

• To be updated by

Couchbase

• Alex Ma

• Couchbase support team

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

ThingSpace IoT ecosystem

ThingSpace platformVerizon network

End-to-end device integration and management

ApplicationsDevices

Connectivity

management

Device

management

Reporting and

analytics

Security and

compliance

Ze

ro-t

ou

ch

on

bo

ard

ing A

PIs

an

d d

eve

lop

er

po

rtal

Wired access

LTE/5G

Traditional and

IoT cores

15

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Why Couchbase?

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Verizon Labs IoT/ThingSpace & Couchbase

Problem

Solved

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Couchbase hardware & software

Type Specification

CPU E5-2697v3

Memory 256GB

Disk 2TB SSD (RAID 1)

OS Centos 6.7

Couchbase Server 3.1.1

JVM Oracle JRE 1.7u 64-bit

18

Specs

• Do not tune if Couchbase meets your

performance requirements

• Consult your OS and Couchbase

professional for tuning parameters

• Benchmark, benchmark and

benchmark!

Rules

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Couchbase performance

19

Number

of Nodes Size

Stable

Rate (s)

Stable CAS

Rate (s)

Max Rate

(s)

Average

CPU IOPS Rate

3 2.5 KB 150,000 47,000 250,000 30% 7000

Operation Events Events > 100ms

Read 1 billion 7012 (0.0007%)

Write 200 million 2620 (0.0013%)

Total 1,200,000,000 9622

Total Doc Memory Resident <5ms

100M 100% 99.5%

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Solving device report challenges

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Use case requirement

Complex and dynamic IoT device report

• Multi-tenancy and grouping of IoT devices

• Arbitrary search and sort on more than 35 device fields

Scaling to support devices from several thousand enterprise customers

Reduce time to market (TTM) for new device features

Always on, Active-Active DCs and 99.999%

21

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Our Challenges

Exponential growth of IoT devices in our system

• Device report from portal has unpredictable response time

• Query from RDBMS does not meet our latency requirement

Difficult to maintain legacy stored procedure

• >5000+ lines

• Join queries

RDBMS high replication latency

Time to market (TTM) and capacity projection

22

High Latency

Legacy

Exponential growth

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or

distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 23

Solution Overview

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Datagrid

Datagrid cluster

• Distributed, memory-first and modular data-centric system

• Built on top of Couchbase, ElasticSearch and etc

• Near-real time or real time data analytics depending on queries.

Datagrid client

• CAS and query data from Datagrid clusters

Sync Service

• Incremental data synchronization between DBMS and Datagrid cluster

24

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Datagrid

25

Product Version

Couchbase (CB) 3.1.1

ElasticSearch (ES) 1.7.1

CB-ES Plugin 2.1.1

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Lessons learned in production

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Minimize human touchpoints

• DevOps Automation Framework

• Vagrant, Docker and Ansible/Puppet

• End-to-end cluster installation and validation via REST API

27

DevOps automation

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Alert and monitoring

28

Basic metric

per node

Cluster level Node level Bucket level

Host CPU

Process CPU

Disk I/O

Network I/O

Node failures

File system

Size (number of nodes)

Active tasks

Number of connections

Health (up/down)

CPU usage (sys/usr/iowait)

RAM usage

Major page faults

Disk space usage

Swap usage (should be

always 0)

Disk IOPS

Disk IO latency

Get ops rate

CAS ops rate

Set ops rate

Ram quota usage

Active document resident ratio

XDCR level (2 XDCR queues X2 DCs):

• Outgoing queue size

• Outgoing replication rate

• Incoming replication rate

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Understand your workflow

Watch and tune your clusters to suit your workflow:

Compaction

Add/Remove Nodes

Security Updates

29

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Couchbase, Mesos, Docker, Spark and AlluxioProof of Concepts

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Real-Time Intelligent Data Pipeline

Data Ingestion Insights

Device

Sensor

Cloud

Streaming Data

Event Processing Pipeline

Visualization

AI Devices

Map

Analytics

Rules

Action

Storage

Machine Learning

Transformation

Filter

Relationship

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Basic streaming dataflow

32

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Data pipeline demo

Processing 500,000 accounting events/sec to find aggregated IoT device usages.

• POC using Datagrid, built on top of Couchbase (14 nodes), Spark (60 nodes),

Kafka (6 nodes), Mesos, Docker

• Couchbase cluster sustains 3.5 millions operations/sec

• Video: https://www.youtube.com/watch?v=7k4qypwYiRs

33

Type Rate/s Nodes CPU per node Memory per node

Device Traffic

Simulator

500,000 5 8 8GB

Spark 500,000 60 4 16GB

Couchbase 3,500,000 14 20 128GB

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Performance improvement

Achieved 50% higher processing rate with Spark on Alluxio, an open source memory

distributed storage

34

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Performance improvement

35

Achieved 50% higher

processing rate with Spark on

Alluxio, an open source

memory distributed storage

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Our next steps

Evaluating Couchbase 4.5.x

N1QL

Memory-optimized indexes

Integrated full-text search

Partial updates and reads

Extra use cases with Couchbase

36

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

We’re hiring

37

Verizon Website: http://www.verizon.com/about/careers

My email: mohanraj.umapathy@one.verizon.com

Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.

Thank you.

38