Big data voor Sociale Secretariaten

FEBRUARY 10, 2014 | SLIDE 1

www.realdolmen.com

BIG DATAINSPIRATION SESSION

7/2/2014


AGENDA

Setting the scene.

Defined.

Avoiding pitfalls.

Practical approach.

Open discussion. Use cases & reality.



BIG DATA

interaction datatransaction data

data integrationdata processingdata analytics

?


BIG DATA



Social Media


BIG DATA



Call Detail Records

ClickStream Data

Scientific/Genome

Machine/Device


BIG DATA



Online Transaction Processing

Online Analytical Processing & Datawarehouse Appliances


BIG DATA




BIG DATA




BIG DATA




BIG DATA & YOUR MARKET REACH


BIG DATA & YOUR MARKET REACH

Marketing Mix

Direct Retail ATL Online

+Big Data = Better insight & Value


USE CASE #1 – SOCIAL TV ANALYTICS










USE CASE# 2 – RETAIL CUSTOMER ANALYTICS





Brand preferencePolitical learningsReading habitsCharitable givingNumber of cars






USE CASE# 3 – PERSONAL LIFE PREDICTION

Data patterns

Interactions

People



Relationship StatusChanges

Data patterns



Data patterns

Interactions

People

Type of relationshipFriendship historyLove history

Friends’ network

Relationship status changesWall-writing patternsrelationship strenghtFlirtation index

Photo tagsCheck-in

Page & photo viewsMessaging history

33% accuracy


Big Datadefined





Transactions

Interactions

2011 2,7ZB 2015 8ZB 2020 35ZB

Source: An IDC White Paper - sponsored by EMC. As the Economy Contracts, the Digital Universe Expands.

.

VOLUME


VARIETY


VELOCITY



Big Data Integration enables an

Organization to:

Do things they could not do before

Do things that they have been doing much more cost effectively

1Business Expects Big Benefits

2Traditional Paradigms

cannot support all of Big Data Challenges

3A New Big Data

processing Platform

emerged to support Big Data requirements

4Complimentary to other

technologies

Variety Complexity

Velocity Volume

BIG DATA OPPORTUNITIES


Volume

Source: IDC

LatencyYears Sub-Second

Data Volume

Across Time Scales

Bu

sin

ess V

alu

e

DEFINING BIG DATA

EXPLOSIVE GROWTH OF DATA – VOLUME, VARIETY, VELOCITY

Velocity

Variety

http://www.workday.com/index.php

http://www.workday.com/index.php

http://www.google.com/imgres?imgurl=http://thecloudtutorial.com/elephant.gif&imgrefurl=http://thecloudtutorial.com/hadoop-tutorial.html&usg=__gMB0dzbV8scian9XqoQcLzF-LkE=&h=473&w=2000&sz=147&hl=en&start=1&sig2=zb1VhqPYhtx0cLN1-dgAgg&zoom=1&itbs=1&tbnid=yH80Nlay6IFDeM:&tbnh=35&tbnw=150&prev=/images?q=hadoop&hl=en&gbv=2&tbs=isch:1&ei=Z9ZFTdDhFI2isAP1vbSKCg

http://www.google.com/imgres?imgurl=http://thecloudtutorial.com/elephant.gif&imgrefurl=http://thecloudtutorial.com/hadoop-tutorial.html&usg=__gMB0dzbV8scian9XqoQcLzF-LkE=&h=473&w=2000&sz=147&hl=en&start=1&sig2=zb1VhqPYhtx0cLN1-dgAgg&zoom=1&itbs=1&tbnid=yH80Nlay6IFDeM:&tbnh=35&tbnw=150&prev=/images?q=hadoop&hl=en&gbv=2&tbs=isch:1&ei=Z9ZFTdDhFI2isAP1vbSKCg

http://www.google.com/imgres?imgurl=http://3.bp.blogspot.com/_aN1WwtpRi5c/TJeCAiUwz9I/AAAAAAAAAFE/CXmGPn0KOQA/s1600/NetezzaFullLogo.jpg&imgrefurl=http://selmark.blogspot.com/2010/09/ibm-to-acquire-netezza.html&usg=__GYYOaZU9yUgJ0cI5s1mHpA6FXGc=&h=356&w=1506&sz=60&hl=en&start=1&sig2=6u8-CxZ1v-tm_EgFabHOBQ&zoom=1&itbs=1&tbnid=Iw5k7cbxZGDkcM:&tbnh=35&tbnw=150&prev=/images?q=netezza&hl=en&gbv=2&tbs=isch:1&ei=pNZFTc3PM4S8sAOIsISHCg

http://www.google.com/imgres?imgurl=http://3.bp.blogspot.com/_aN1WwtpRi5c/TJeCAiUwz9I/AAAAAAAAAFE/CXmGPn0KOQA/s1600/NetezzaFullLogo.jpg&imgrefurl=http://selmark.blogspot.com/2010/09/ibm-to-acquire-netezza.html&usg=__GYYOaZU9yUgJ0cI5s1mHpA6FXGc=&h=356&w=1506&sz=60&hl=en&start=1&sig2=6u8-CxZ1v-tm_EgFabHOBQ&zoom=1&itbs=1&tbnid=Iw5k7cbxZGDkcM:&tbnh=35&tbnw=150&prev=/images?q=netezza&hl=en&gbv=2&tbs=isch:1&ei=pNZFTc3PM4S8sAOIsISHCg

http://www.greenplum.com/

http://www.greenplum.com/

http://www.hyperion.de/

http://www.hyperion.de/

http://www.peoplesoft.com/corp/en/public_index.asp

http://www.peoplesoft.com/corp/en/public_index.asp

http://aws.amazon.com/

http://aws.amazon.com/



Based on traditional

relational database

technologies

PAST GENERATION TECHNOLOGIES WERE NEVER DESIGNED FOR BIG

DATA VOLUMES AND TYPES

What happened?

Why did it happen?

• Data mining

• Data analysis

• Business Intelligence

Data warehouse

What will happen?

Social Media, Web Logs

Machine & Device Data

Documents and Emails

Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


GROWING ADOPTION IN HADOOP

• Highly scalable: Any data volume/size

• Flexible: Any Data Type (Structure and

Unstructured)

• Cost effective: Leverages commodity

hardware




Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


INVESTMENTS IN DATA ANALYTICS, DATA MINING,

DATA VISUALIZATION TECHNOLOGIES

• Improve User Experience vs. previous generation offerings

• Flexible Delivery (On-premise and Cloud)

• Pre-packaged Analytics and Rules


Real-time Data

Visualization

Sentiment Analysis

Data Mining, Predictive

Analytics

Next Gen. Data

Warehouse

BIG DATA TECHNOLOGIES AT WORK

Process, Score,

Analyze All Data

Consume Results




Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


More Data Volumes

Requires Scalable

Data Integration

Capabilities


INTEGRATING DATA WITH HADOOP REQUIRES DEEP

PROGRAMMING EXPERTISE

• Hadoop Programming language is complex to integrate data

• Requires significant knowledge and expertise

• Development costs can significantly lower expected business value


Real-time Data

Visualization

Sentiment

Analysis

Data Mining,

Predictive

Analytics

Data Warehouse

BIG DATA MANAGEMENT

Process, Score,

Analyze All Data

Consume Results

Big

Da

ta I

nte

gra

tio

n &

Ma

sk

ing

Business User




Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


More Data Sources

and Types Increases

Risk of Data Quality

Errors


Real-time Data

Visualization

Sentiment

Analysis

Data Mining,

Predictive

Analytics

Data Warehouse

BIG DATA MANAGEMENT

Process, Score,

Analyze All Data

Consume Results

Big

Da

ta I

nte

gra

tio

n &

Ma

sk

ing

Business User

Data Quality & Governance

Data Analyst Data OwnerData Steward Developer




Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


Managing More Data

Increases the Risk of

an Unwanted Data

Breach


Real-time Data

Visualization

Sentiment

Analysis

Data Mining,

Predictive

Analytics

Data Warehouse

BIG DATA MANAGEMENT

Process, Score,

Analyze All Data

Consume Results

Big

Da

ta I

nte

gra

tio

n &

Ma

sk

ing

Business User


Da

ta P

riva

cy E

nfo

rce

me

nt




Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


Dealing with Big Data

Requires an Effective and

Efficient Archiving

Solution


Real-time Data

Visualization

Sentiment

Analysis

Data Mining,

Predictive

Analytics

Data Warehouse

BIG DATA MANAGEMENT

Process, Score,

Analyze All Data

Consume Results

Big

Da

ta I

nte

gra

tio

n &

Ma

sk

ing

Business User


Big Data Archiving and Retention

Da

ta P

riva

cy E

nfo

rce

me

nt




Mainframes, Apps

Payments, Trade

Customer

Entities

Reference Data

Other

Securities


BIG DATA SUPPLY CHAIN – HOW TO MAKE IT WORK


CHALLENGES


BIG DATA ARCHITECTURE

The Power of Hadoop Store huge amounts of data

Scaling & cost advantage

Complex data analytics & Extensible

Online

Transaction

Processing

(OLTP)

Online Analytical

Processing

(OLAP) &

DW Appliances

Social

Media Data

Other

Interaction Data

Scientific, genomic

Machine/Devic

e

BIG TRANSACTION DATA BIG INTERACTION DATA

BIG DATA PROCESSING

Call detail

records, image,

click stream data

Big Data Processing• Extract & Load Data from & into Hadoop (HDFS/Hive)

• Parse Complex files in Hadoop framework (Map & Reduce )

BIG DATA INTEGRATION

Access Discover Cleanse Integrate Deliver

Product &

Service

Offerings

Social Media Customer

Service Logs &

Surveys

Marketing

Campaigns

Account

Transactions

Sales &

Marketing

Data mart

Customer

Service

Portal

Operational

MDM

BIG DATA PROCESSINGBIG DATA ANALYTICS


SOLUTION FLOW FOR BIG DATA

REFERENCE ARCHITECTURE

EDWDimensions

FactsBI Reports and Dashboards

Analytic

Applications

and Portals

Sources

DW

Operational

MDM

Push-down

Transformation

AccessVirtualization

Transformation

Data

Marts

Proactive Alerts

Low Latency Update

& Distribution


APPROACH


10 WAYS BIG DATA IS USED TODAY

Understanding and Targeting Customers

Understanding and Optimising Business Processes

Personal Quantification and Performance

Optimisation

Improving Healthcare and Public Health

Improving Sports Performance

Improving Science and Research

Optimising Machine and Device Performance

Improving Security and Law Enforcement

Improving and Optimising Cities and Countries

Financial Trading




BIG DATA OPPORTUNITIES

BROAD SPECTRUM OF OPPORTUNITY

Enterprise Functions Use Cases Benefits

Marketing • Brand Monitoring & PR

• Gain Customer Insight

• Campaigns & Events

• Competitive analysis

• Increased campaign effectiveness

• Better Brand recall

Sales • Gain Sales Insight

• Referrals Mining

• Lead Capture

• Lead Generation

• Increased Lead conversion rates

• Increased revenue from “Native Social”

leads

Service & Support • Gain Support Insight

• Rapid Response

• Peer-to-Peer Support armies

• Better control of online “epidemics”

• Lower support costs due to community

Knowledge base

Innovation & Collaboration • Customer-led innovation

• “Exterprise” collaboration

• Effective product launches

• Collaborative R&D with 3rd parties

• Employee and partner productivity

Loyalty & Customer

Experience

• VIP Experience

• Loyalty & Incentives

• Increased loyalty

• Unpaid Brand Evangelism

Finance • Risk Management

• Fraud Detection

• Additional channels for risk

forecasting/modeling

Human Resources • Recruitment

• Background Verification

• Cost-effective recruitment

• Better fitting candidates


Q&A

Dimitri Maesfranckx

Division Manager Business Insights

+32 479 730131

[email protected]

Luc Delanglez

Solutions Manager Business Insights, MDM

+32 474 309053

[email protected]


REALDOLMEN BUSINESS INSIGHTS –

TURN DATA INTO MEANINGFUL INFORMATION

BUSINESS

INFORMATION

Technology

Big data voor Sociale Secretariaten