PaNDAA Big Data Approach to OSS Analytics

John Evans

Distinguished Consulting Engineer

Trevor Smith

Technical Leader

• Virtualisation / automation / orchestration has made real-time service provisioning possible

• Open source big data / analytics technologies now being widely applied outside of big web companies

• OSS architectures simply not keeping pace with the rest of industry

• No coherent industry direction on how OSS needs to change in the presence of these new technologies

Context

Simplified OSS / BSS Stack

OSS

BSS

Network and

Services

CustomerOrder

Order Mgmt

Provisioning & Activation

Service Data

Accounting and Monitoring

Billing and Reporting

Bills and Reports

Orchestration ?OSS Analytics

Relationship Between Orchestration and OSS Analytics

Orchestration OSS Analytics

Data CenterCoreAccess &

Aggregation

User

Sta

te Data

Related as loosely coupled

but tightly integrated systems

OSS analytics is responsible for

collecting data from the

infrastructure, monitoring and

analysis

The “F_APS” in FCAPS

Orchestration is responsible

for service provisioning and

pushes state to the

infrastructure

The “C” in FCAPS

• OSS analytics applications can be addressed by performing a query function against the entire OSS data set

• Fault management = (event data)

• Performance management = (metric data)

• Billing mediation = (event data, metric data)

• Capacity management = (metric data)

• Security analytics = (metric data, route data)

OSS Analytics is a big data problem!

Fix the data first

“Analytics is only as good as the

availability and quality of the data that

can be processed — so you need to

fix the data first”

In "How to Compete on Analytics",

Davenport lays out 4 quadrants of

analytics maturity…

…but how to get to quadrant 4?

Problem space is addressed

by a platform, not a product

Relationship Between Orchestration and Analytics

Orchestration OSS Analytics

Data CenterCoreAccess &

Aggregation

User

Sta

te Data

Related as loosely coupled

but tightly integrated systems

OSS analytics is responsible for

collecting data from the

infrastructure, monitoring and

analysis

The “F_APS” in FCAPS

Orchestration is responsible

for service provisioning and

pushes state to the

infrastructure

The “C” in FCAPS

We have platforms for

orchestration in Cisco NSO,

OpenDaylight, Openstack

We need a companion

platform for OSS Analytics

PaNDAPlatform for Network Data Analytics

Platform for Network Data Analytics

• Vision:

• Simple, scalable, open big data / analytics platform

• Forms a generic big data analytics platform supporting different types of analytics applications for cloud based networks and services• Operational Intelligence, e.g.

OSS

• Business intelligence, e.g. BSS

• Leverage rapid innovation in Big Data analytics space

Orchestration

Controllers

Customer

Devices

Applications

QoE Monitoring

Infr

astr

uct

ure

an

d

serv

ice-

leve

l dat

aC

ust

om

er-l

evel

dat

a

Data

Distribution

Data Store

& Processing

Open Data Platform

Producers:

Data aggregation

Event Data

Log Data

Metric Data

Network Telemetry

Data Sources

App

App

App

App

App

App

App

Consumers:

Data analysis

Applications

Inventory Topology Geography Geography

Context:

Horizontally

Scalable Data

Platform

Applications

App

App

App

Live stream

Platform for Network Data Analytics

Orchestration

Controllers

Customer

Devices

Applications

QoE Monitoring

Infr

astr

uct

ure

an

d

serv

ice-

leve

l dat

aC

ust

om

er-l

evel

dat

a

Data

Distribution

Data Store

& Processing

Master Data

Store

Open Data Platform

Batch

Processing

Stream

processing

Real Time

Data Store

Deep H

isto

rical Q

uery

Real T

ime Q

uery

Producers:

Data aggregation

Event Data

Log Data

Metric Data

Network Telemetry

Data Sources

Inventory Topology Geography Geography

Context:

• Principles

• Decouple data aggregation (publishers) from data analysis (consumers) – allow any app tto access any data source

• Simple, scalable, open data distribution platform

• Immutable dataset with minimal filtering/processing on ingress

• Analytics based approach to analysis functions

• Support for streaming apps, real-time queries and batch processing

Live stream

App

App

App

App

App

App

App

Consumers:

Data analysis

Applications

App

Platform for Network Data Analytics

Data

Distribution

Data Store

& Processing

Master Data

Store

Open Data Platform

Batch

Processing

Stream

processing

Real Time

Data Store

Deep H

isto

rical Q

uery

Real T

ime Q

uery

Producers:

Data aggregation

Event Data

Log Data

Metric Data

Network Telemetry

Data Sources

Capacity Analytics

Billing (Mediation)

Business Intelligence

Fault Analysis

PerfAnalysis

Log Analytics

Security and Threat Analysis

Inventory

Consumers:

Data analysis

Applications

Inventory Topology Geography Geography

Context:

Live stream

SNMP

Logs

SNMP

Monit,

Collectd,

Logstash,

Ceilometer

Netflow

Benefits

• An open system architecture

• Collect data once – allow any analysis application to mine any data source, leveraging the full value of the OSS dataset

• Extensible – add new OSS analysis functions quickly and seamlessly with minimum of development cost

• Leverage rapid innovation in Big Data analytics space

Spark

Streaming

Logstash

Logstash

pmacct

Kafka

Spark

Hbase

Impala

ELK

OpenSoc

Demo

See us in the DevNet zone

Online at panda.cisco.com

Hands-on with our Learning Labs at developer.cisco.com/site/panda/

Want to know more?

Thank you