Contexti / Oracle - Big Data : From Pilot to Production

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Big Data: From Pilot to Production

Vicky Falconer - Oracle

Grant Priestley - Contexti

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |my.oracle.com/go/bigdata

Program Agenda

1

2

3

4

Big Data Project Challenges

Typical Big Data Journey

Common Operating Models

Technology Considerations


Big Data Project Challenges


Experience with a regional Telco

• You don’t know what you don’t know…

• Build it and they will come

• Technology versus capability

• Clear definition of skill requirements

• Moving from Pilot to Production


Additional challenges

• Who owns data?

• What to do in house and what to externalise

– Analytics

– Admin

– Development

– Engineering

• Operationalising insight


Key Lessons

• Where to start?

• Culture

• Scope

• Building capabilities

• Technology

Right Questions

Right Use Cases

Right Business

Case


Typical Big Data Journey


Big Data Developments

Increasing use of infrastructure as a

service

Better understanding of the possibilities

offered by unstructured

data

Moving from historical batch computing to

real-time analytics

Wider awareness and a

more defined understanding of

Big Data

Wider variety of vendors offering

Big Data solutions

Less hype, more real use cases of

companies exploiting Big

Data

Maturity of Big Data tools

bringing them into the

mainstream

What changes we have noticed over the past 12 months with respects to Big Data that are most likely to impact on your organisation or on the market in general:

Increase in requests for platform as a

service

AdvancedLess Advanced 2015?


Where Value Lies for Most OrganisationsThe proliferation of Big Data Analytics applications and solutions has given rise to the need for a Big Data Platform that enable these initiatives to occur and support all use cases including Advanced Analytics, Internet of Things (IoT) and the Digital Enterprise. The Big Data PaaS accelerates organisation's projects by provisioning the initial platform and development environment, eliminating the need for hard-to-find Big Data skills and ultimately allows the enterprise to focus on strategic initiatives and IP creation rather than platform operations.


Big Data Use Cases

1

2

3

Customer Insights / Behavior

Data Warehouse Augmentation

Risk Analysis & Fraud Detection


Customer Insight / Behavior

Challenges

Understand customer behavior through predictive analytics

Solution

Leverage Hadoop and ML techniques to build “population-based behavioral” clusters enabling personalised content to be served up in certain real-time sequences

Business Outcomes

• Increase in sales conversion• Online engagement is personalised, as it is in store


Data Consumer(s)Data Source(s) Contexti Big Data Platform

Customer Insight / Behavior

Semi-StructuredData

StructuredData

Pre-computed Web Content &

Deals

Raw / EnrichedData Sets

(HDFS / MFS)

StreamingData

Acquisition

File-BasedData

Acquisition

RDBMS-Based Data

Acquisition

Data Ingestion

Deep Analytics &

Machine Learning

Streaming Capability Serving Capability

Batch Capability

Online Store

Customers



Challenges

Reduce latency between data generation and availability

Solution

Offload ETL processing to Hadoop platform and support the ingestion of multi-structured data sets

Business Outcomes

• Access of data reduced from T+1, T+2 to real-time / intra-day• Reduce cost of ETL processing• More time now spent on analysing data than data wrangling



Data Consumer(s)Data Source(s) Contexti Big Data Platform

Semi-StructuredData

UnstructuredData


(HDFS / MFS)

StreamingData

Acquisition

File-BasedData

Acquisition

RDBMS-Based Data

Acquisition

Data Ingestion Extract Load

Transform (ELT) and Data

Preparation Processes


Batch Capability

Users

StructuredData

Reporting, Search &

Query

RDBMS & MPP

Platforms

Pre-computed Views



Challenges

Reduce incidents of fraud through more sophisticated detection and monitoring

Solution

Ingested structured and raw law data from multiple applications and combined data filtering from Pig/Hive with statistical modeling by R,while executing CEP on streams of data

Business Outcomes

• Implemention of real-time trigger based analytics that provides early detection of fraud

• “Schema on read” provided greater flexibility for analysis


Contexti Big Data Platform


Data Consumer(s)Data Source(s)

Semi-StructuredData

UnstructuredData


(HDFS / MFS)

StreamingData

Acquisition

File-BasedData

Acquisition

RDBMS-Based Data

Acquisition

Data Ingestion


Batch Capability

Online Store

StructuredData

Data Access Provisioning

API

RDBMS & AnalyticsPlatforms

Raw Data(In-Memory)

CEP / Stream Analytics

Pre-computed Views

Real-Time Incremental

ViewsFraud Systems

Deep Analytics &

Machine Learning





Decentralised Centralised Federated

In a decentralized services model, each business or function has its own analytics group, which enables and encourages rapid decision-making and execution.

Pros:• Analytics needs aligned to business

functions• Close to business and customers needs

Cons:• Limited strategic view• Duplication, redundancies and inability

to standardise or leverage scale

The centralised shared-services model exists outside organizational divisions or functions, in some cases external to the organisation itself.

Pros:• Standardised processes and methods• Independent viewpoints shared across

the organisation

Cons:• Perception that group lacks functional

expertise• Ownership of IP when outsourced

The federated shared-services model is a centralized model that rolls under an existing function or business unit and serves the entire organization.

Pros:• Speed in execution & decision making• Pre-existing shared service processes

and structure

Cons:• Less transparent resource allocation• Focus on business function priorities


Building a Best Practice Analytics Capability

Analytical Capability Techniques Questions

BasicProvide static, historical view of business performance drawn on basic scorecard and static reports

Query and drill down Where is the problem?

Ad hoc reporting How many? How often? Where?

Standard reporting What happened?

AnticipatoryCreates transparency into past and future drivers, using systems and processes to perform a range of descriptive analytics

Segmentation Analysis What are the unique drivers?

Statistical Analysis Why is this happening?

Sensitivity Analysis What if conditions change?

PredictiveRequires high-quality integrated data and complex mathematical capabilities and offers dynamic forward looking insights

Optimisation What is the best that can happen?

Simulation What would happen if …?

Predictive Modelling What could happen next?


Challenges to FaceHurdles between Pilot and Production


Pilot vs Production Characteristics

Pilot Characteristics Production Characteristics

Funding Project-Based Project and BAU Funding

Number of Use Cases 2–3 use cases > 5 use cases

Insights Demonstrated Actionable / Operational

Service Level No / Loose SLA (project-based) Enforced SLAs, OLA

Big Data Capability • Batch• Serve

• Batch• Serve• Stream (advanced)

Resiliency / DR No Mandatory

Security Enabled Optional Mandatory

Scale 1 Rack, <5 data sources Multiple Racks, >10 data sources

Timing 3-6 months 6-9 months*


Is Your Big Data Pilot Ready for Production?

Culture

• Allows for trial and error, ability to fail

• Understanding that data is an enterprise asset, benefit of being “data informed”

Structure & Skills

• Governance of data and its use

• Decision on what skills to acquire/develop/buy (analyst, dev, data scientist, ops, engineering)

• Funding Model (How will users/customers be charged?)

Integration

• Technologies in place to connect internal/external data, including unstructured data

• Integration of “actionable insights” into operational processes





• Business Strategy drives IT Strategy

– Information Architecture

• Future State Infrastructure

– Scale out and up

– Adding Big Data to existing infrastructure can be complex

• Analytics

– Embed in operational systems

• Integration insight into existing systems and processes


Actionable

Events

Streaming Engine Data Reservoir Enterprise Data & Reporting

Discovery Lab

Actionable

Information

Actionable

Data Sets

InputEvents

Execution

Innovation

Discovery Output

Data

Conceptual View

StructuredEnterprise Data


Oracle Big Data Strategy

BY INDUSTRY & LINE OF BUSINESS

BIG

DA

TA

AP

PLI

CA

TIO

NS

DISCOVERY

BU

SIN

ESS

AN

ALY

TIC

S

BUSINESS ANALYTICS

DATA RESERVOIR

BIG

DA

TAM

AN

AG

EMEN

T

DATA WAREHOUSE

SOU

RC

ES


Oracle Big Data Management System

SOU

RC

ESDATA RESERVOIR DATA WAREHOUSE

Oracle Database

Oracle IndustryModels

Oracle Advanced Analytics

Oracle Spatial & Graph

Big Data Appliance

Apache Flume

OracleGoldenGate

Oracle Event Processing

Cloudera Hadoop

Oracle NoSQL

Oracle R Advanced Analytics for Hadoop

Oracle R Distribution

Oracle Database

In-Memory, Multi-tenant

Oracle Industry Models

Oracle Advanced Analytics

Oracle Spatial & Graph

Exadata

OracleGoldenGate

Oracle EventProcessing

Oracle DataIntegrator

Oracle Big DataConnectors

Oracle DataIntegrator

ORACLE BIG DATA SQL


Oracle R Enterprise Approach

Data and statistical analysis are stored and run in-database

Same R user experience & same R clients

Embed in operational systems

Complements Oracle Data Mining

ROpen Source


Closing Remarks


Summary

• Where to start?

• Culture

• Scope

– Data Ownership

– Data Governance

– IM Strategy

• Building capabilities

• Technology

Right Questions

Right Use Cases

Right Business

Case