background image: 960x540 pixels - send to back of slide and set to 80% transparency

Advanced Analytics for Telecommunications Bob Glithero, Principal Product Marketing Manager Vineet Goel, Product Manager

background image: 960x540 pixels - send to back of slide and set to 80% transparency

Agenda

•  Pivotal – Hortonworks Partnership •  Challenges in Customer Experience •  HDB: Hadoop-Native Analytics Database

for Hortonworks Data Platform •  Sample Use Cases •  For More Information

Pivotal HDB + Hortonworks Hadoop Partnering for Faster Value from Data

●  Leaders in open-source Hadoop ●  Managing, analyzing, and operationalizing data at

scale ●  Joint support for ODPi promotes interoperability in

Hadoop

+Pivotal and Hortonworks’ strategic partnership marries Pivotal’s best-in-class SQL on Hadoop, analytical database, with Hortonworks’ best-in class expertise and support for Hadoop.

You’re the third person I’ve been handed off to! Can’t anyone help me?

4

I’m not seeing any alarms...why are our customers having poor service?

5

Managing Experience is Complicated

Then •  Basic handsets, embedded applications •  Simpler services - voice, SMS, WAP •  Experience influenced mostly inside the network

Now •  From phones to hand-held computers •  Massive data volume, velocity, and variety from millions of apps and

services •  MNOs held responsible for all aspects of service, whether inside or

outside the network

CSPs Increasingly Competing on QoE

Trying to understand how network performance impacts experience

When service is degraded, CSPs need to quickly understand:

Is the problem inside or outside the network? Which subscribers are impacted? What needs attention first?

Common Operator Challenges

Network Operations Customer Care Marketing

Increase monetization, offset voice, SMS revenue loss

Reduce churn and credits, cost to serve

Reduce complexity, increase visibility, increase QoE

background image: 960x540 pixels - send to back of slide and set to 80% transparency

Operators are turning to their data to solve these challenges How do we analyze data in an efficient, cost-effective way to transform customer experience?

High performance, interactive SQL queries on Hadoop HDB: The Hadoop Native SQL Database

●  Highly efficient MPP (massively parallel processing)

●  Low-latency

●  Petabyte scalability

●  ACID transaction support

●  SQL-92, 99, 2003 compatibility

●  Advanced cost-based optimizer

DATA LAKE SQL App

BUSINESS ANALYSTS

DATA SCIENTISTS

Advanced Analytics Performance

Exceptional MPP performance, low latency, petabyte scalability, ACID

reliability, fault tolerance

Most Complete Language Compliance

Higher degree of SQL compatibility, SQL-92, 99, 2003, OLAP, leverage

existing SQL skills

Best-in-class Query Optimizer

Maximize performance and do advanced queries with confidence

Elastic Architecture for Scalability

Scale-up/down or scale-in/out, expand/shrink clusters on the fly

Tightly integrated w/MADlib Machine

Learning Advanced MPP analytics, data science at

scale, directly on Hadoop data

HDB / HAWQ Advantages

MAD

●  DiscoverNewRela/onships●  EnableDataScience●  AnalyzeExternalSources●  QueryAllDataTypes!

Mul/-levelFaultTolerance

GranularAuthoriza/on

ResourcePools+YARN

Mul$-tenancy+Security

ANSISQLStandard

OLAPExtensions

JDBCODBCConnec/vity

MPPArchitecture

OnlineExpansion

Hadoop/HDFS

PetabyteScale

Cost-BasedOpXYZizer

DynamicPipelining

ACID+Transac/onal

AmbariManagement

MachineLearning

DataFedera/on

LanguageExtensions

Hardened,10+YearsTested,Produc/onProven

Opera$ons+Extensibility

HDFSNa/veFileFormats

●  ManageMul/pleWorkloads●  PetabyteScaleAnaly/cs●  Sub-secondPerformance

●  LeverageExis/ngSkills&Tools

●  EasilyIntegratewithOtherTools

Compression+Par//oning

Core

compliance

●  WellIntegratedwithHortonworksDataPlaZorm

HDB + HDP Marketecture

13

Faster Insight with In-Database Analytics

Pivotal HDB / Apache HAWQ (incubating) Low-latency, MPP analytic

database with full ANSI SQL support running natively on

Hortonworks HDP

Apache MADlib (incubating) Scale out, SQL-based

machine learning within HDB/HAWQ, Greenplum, and

PostgreSQL databases

+

14

Top MADlib Use Cases

•  Fraud detection •  Risk analysis •  Customer experience •  Marketing •  Predictive maintenance

Telco uses HDB to analyze and improve call quality

2bn call records per day •  Overwhelmed traditional data warehouse

Hadoop and HDB •  5x data stored at half the cost •  Familiar SQL interface to analyze 3 months

worth of dropped call data

DATA LAKE

16

How could a network operations team apply analytics to improve experience for its network services?

What Data Is Needed?

Service Assurance Customer Care Marketing • Network Performance data (GTP probe data)

• HTTP Click Stream Records

• Flow Records

• Network & Device Reference Data

• Topology and location

• HTTP Click Stream Records

• Flow Records

• Network Performance data (GTP probe data)

• CRM data (account, device information)

• Service Request Records

• HTTP Click Stream Records

• Flow Records

• CRM data (account, device information)

Constructing KQIs from performance indicators

84% Speed Latency Effective Throughput

Integrity

Drops Time-Outs Cut-Offs Failures

Retainability

Failure % Response time Access time

Accessibility Voice QoE

Data capture Data science

•  xDRs •  NetFlow •  Probes

Data processing

Accessibility

Quality

Retainability

In-Database Analytics with HDB and MADlib

Application/ Content Data

•  Raw Usage •  Logs •  (HTTP, Flow, Other)

HDFS

HBase

Hive

HDB/HAWQ In-DB Analytics Network Data

•  Probes (GTP-C/U) •  xDRs

•  Case management •  CRM •  Billing •  Device inventory •  Network topology •  Geolocation maps B/OSS Data

PXF

PXF

MPP Query Execution

ANSI SQL

•  SQL-based •  Over 50 data science

functions •  UDFs

•  Offline modeling •  Batch queries •  Reporting/viz with

SQL-based tools

+

Native or PXF

20

How could marketing teams use analytics to better target subscribers for promotions and advertising?

Blended Mobile ARPU is Declining

Loss of voice and SMS ARPU from competition, free apps Data revenues not offsetting voice, SMS losses MNOs seeking new monetization options

Source: IHS Technology Mobile ARPU Forecast, 2016

Need for Behavioral Insights

•  CSPs need to maximize subscriber yields to offset declining revenues

•  Marketers have little information to market to anonymous prepaid subscribers

•  Need to protect current revenue from competition from over-the-top (OTT) apps and services

Morning: New York •  Starts on Samsung Galaxy S6 •  On CNN, sees news on earthquake •  Donates via Red Cross Society •  Later: Switches to iPad – same account

plan •  Checks market close on WSJ.com

A Day in the Life: User Perspective

Evening: Boston •  Checks Facebook page •  Streams Netflix

SubscriberId StartTimeStamp EndTimeStamp URL User AgentRK2FQ9PWZVW52 2015 04 28 06 37 04 512 2015 04 28 06 37 04 543http://www.cnn.com Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1 RK2FQ9PWZVW52 2015 04 28 06 37 05 546 2015 04 28 06 37 04 623http://www.cnn.com/world Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1

RK2FQ9PWZVW52 2015 04 28 06 37 19 529 2015 04 28 06 37 19 599http://www.cnn.com/2015/04/28/asia/flight-delhi-nepal-earthquake/index.html Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1

RK2FQ9PWZVW52 2015 04 28 06 37 23710 2015 04 28 06 37 23 770http://www.cnn.com/2015/04/28/asia/kathmandu.jpgMozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1 RK2FQ9PWZVW52 2015 04 28 06 37 45919 2015 04 28 06 37 45988http://adclick.g.doubleclick.net/pics/click/?= Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1

RK2FQ9PWZVW52 2015 04 28 06 37 34957 2015 04 28 06 37 34996http://www.google-analytics.com/__utm.gif?utmwv=4.9mi Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1

RK2FQ9PWZVW52 2015 04 28 06 42 09 883 2015 04 28 06 42 10 467http://www.cnn.com/2015/04/25/world/nepal-earthquake-how-to-help/index.html Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) CNN/2.1.1 ….. (images being loaded here) …….

RK2FQ9PWZVW52 2015 04 28 06 43 03 234 2015 04 28 06 06 12 334http://www.nrcs.org Mozilla/5.0 (Linux; U; Android 4.4.2; en-US; SAMSUNG-SM-N900A Build/KOT49H) ….. ….. ….. ….. …

RK2FQ9PWZVW52 2015 04 28 09 45 05 7322015 04 28 09 45 05

812 http://wsj.comMozilla/5.0 (iPad; CPU OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari

….. ….. ….. … …

RK2FQ9PWZVW52 2015 04 28 17 03 14 204 2015 04 28 17 03 14 269http://wsj.comMozilla/5.0 (iPad; CPU OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari

….. ….. ….. … …RK2FQ9PWZVW52 2015 04 28 18 19 56 459 2015 04 28 18 19 56 509https://69.63.178.45

….. ….. ….. … …RK2FQ9PWZVW52 2015 04 28 21 23 25 754 2015 04 28 21 23 25 876http://23.13.201.71 netflix-ios-app

A Day in the Life: Data Perspective

•  Capture and collate raw subscriber data •  Sessionize and enrich clickstream data with location, device. and other data, calculate subscriber

usage metrics

SubscriberId DeviceNAME PUBLISHERCategory-

SubcategoryApplication

Name SESSION START SESSION END PAGE_VIEWS HITS BYTES LOCATION

RK2FQ9PWZVW52Samsung Galaxy S6 CNN NewsNews-International

News CNN App 2015 04 28 06 37 04 512 2015 04 28 06 42 10 467 4 45 539123 NY

RK2FQ9PWZVW52 Samsung Galaxy S6 Red CrossNon Profit &

Charities-Institutions Browser 2015 04 28 06 43 03 234 2015 04 28 06 53 03 874 2 7 383372 NY

RK2FQ9PWZVW52 Apple iPadWall Street

JournalNews-Business &

Finance News Safari Browser 2015 04 28 09 45 05 732 2015 04 28 09 55 05 732 4 40 600272 NY

RK2FQ9PWZVW52 Apple iPadWall Street

JournalNews-Business &

Finance News Safari Browser 2015 04 28 17 03 14 204 2015 04 28 17 23 14 204 5 35 801714 NY

RK2FQ9PWZVW52

Apple iPad Facebook

Social Media & Networking-Social

Networking -2015 04 28 18 19 56 459

2015 04 28 18 23 21 459

318 5041054 Boston

RK2FQ9PWZVW52

Apple iPad Netflix

Media & Entertainment-Online

Video Ne&lixApp2015 04 28 21 23 25 876

2015 04 28 23 23 24 325

6 2330 295121789 Boston

Compute subscriber-level metrics and aggregates

…enrich with information about content (websites or apps) and categorization, devices, and locations

Aggregation and Enrichment

Insights: Marketing to Prepaid Users

•  With data science, operators can infer gender and approximate age from subscriber activity

•  Classify according to segmentation schemes (e.g., who does unknown subscriber resemble from their activity)

We can offer advertisers anonymized subscriber info mapped to standard marketing/advertising categories (e.g., IAB) based on activity

Marketing Questions We Can Answer with Analytics

• How will subscribers respond to changes in pricing?

• How do we market to anonymous pre-paid subscribers?

• Who’s likely to respond to an offer?

• Which OTT apps threaten our own branded apps?

• Which groups should we target with advertising?

Pivotal and Hortonworks are partnering to help

companies use their data for better

customer outcomes

Learn more

•  Videos: bit.ly/MADlibvideos •  Project: madlib.incubator.apache.org •  Downloads: bit.ly/getMADlib

•  Videos: bit.ly/HDBvideos •  Project: hawq.incubator.apache.org •  Commercial: pivotal.io/pivotal-hdb •  Downloads: bit.ly/getHDB

Let’s build something MEANINGFUL