Upload
capgemini
View
2.193
Download
4
Tags:
Embed Size (px)
DESCRIPTION
What is the impact of Big Data on Analytics from a Data Science perspective. Presented at the Big Data and Analytics Summit 2014, Nasscom by Mamatha Upadhyaya.
Citation preview
Impact of Big Data on Analytics
Mamatha Upadhyaya
2
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Big Data and Analytics Summit 2014
The terms Big Data
and Analytics are
used simultaneously
3
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
However, analytics, predictive modeling, advanced analytics, data science is not new!
4
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
So what does this mean for Analytics?
Yes, the amount of data that is available to us is exploding
And Big Data Platforms and Commodity Hardware and bringing in additional capabilities
So what does this
mean for Analytics?
Media is rife with Big Data and Analytics
AND The Data Scientist makes it from Nerd to the most cool person!!
…makes it to on top of CIO agenda
Big Data and analytics is touted as the panacea for all problems
5
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Data Science perspective
– A Data Science perspective
Big Data and Analytics Impact of
6
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
A brief history of Data Science
Pre 1800s 1800-1900 1900-1940 1940-1960 1960 1970 1980 1990 2000 2010
Text/ string search
1974 Peter Naur “Concise Survey of Computer
Methods”, Data Science, Datalogy
Knuth – Art of Computer Programming.
1976 – SAS Institute
1977 The International Association for
Statistical Computing (IASC).
Computer Science
Data Technology
Visualization
Mathematics/ OR
Statistics
Probability
Correlation
Bayes Theorem.
Regression, Least
Squares
Time Series.
Theoretical Foundations of Modern Stats
Hypothesis, DOE
Mathematical Statistics.
Bayesian Methods
Time Series Methods (Box Cox,
Survival, etc.)
Stochastic Methods.
Simulation, Markov
Computational Statistics.
Decision Science
Pattern recognition
Machine learning.
Liebniz – Binary Logic. Babbage, Lovelace
Boolean Algebra
Punch cards.
Turing machines
Information Theory
Weiner & Cybernetics
Von Neumann Architecture.
Calculus
Logarithms
Newton-Raphson.
1989 First KDD Workshop
Gregory Piatetsky-Shapiro.
Sort & Search Algorithms –
Dijkstra, Kruskal, Shell Sort, …
Heuristics – Simulated Annealing, …
Graph Algorithms
Multigrid methods
Tree based methods.
Database Marketing
Data Mining, Knowledge Discovery
“Data science, classification, and related methods.”
William Cleveland: Data Science
Leo Breimann: Statistical Modeling: 2 Cultures.
Optimization Methods
Fourier and other transforms
Matrix & Generalizations
Non-euclidean geometries.
Applications to Military,
manufacturing,
Communications.
1962 John W. Tukey, Future of
Data Analysis
Networks
Assignment Problems
Automation
Scheduling.
First IBM
Computers
DBMS.
Removable Disk drives
Relational DBMS. Desktop, floppy
SQL, OOP
High level languages. William Playfair
Charles Minard
Florence Nightingale.
Catrography
Astronomical Charts.
John Tukey
Jacques Bertin. Edward Tufte. Grammar of Graphics
Word Cloud, Tag Cloud.
7
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Drivers of change
Data
Availability
Technology
Ability to
Handle
Structured and
unstructured
data
Platform Cost Agility
Business
Expectation
Digital
Experience
Strategic
Initiatives
New Business Models
8
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
TABLE OF CONTENTS
Data drivers Technology drivers
So what does all of this increased activity
mean to Talent!
9
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
8 TB data everyday
10 TB data everyday
152 million blogs on the internet
Data availability
Possible
Technically but
very expensive Not efficient enough to
handle the amount &
type of data generated
by newer internet-scale
technologies
Big Data
Internal data
M&A New Tech
adoption
Need to access
Unstructured data
External data
2 billion internet users
* Hortonworks CEO Rob Bearden
Digital Customer
IOT
Legacy data management system are not designed to handle heavy
demand of data consumption “85% of that data is coming from net-new data
sources.” – mobile, social media, and web- and
machine-generated data*… and this will increase.
RFID
10
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Improving traditional analytics
Understand Act Measure
Change in Market Share
Campaign Effectiveness
Revenue/ profit metrics/
reports
Customer Churn/
retention reports
Customer Next
Best Offer
Cross Sell/ Up
Sell Opportunities
CRM (Customer
Info, billing, etc.)
Subscription and
Usage Summary
External data
(Demography, etc.)
Customer Profiling
Demographic Segments
Behavior (usage/ profit/
satisfaction/ etc.) based
profiles.
Product Association and Product Mix
Customer Profitability/ Life Time Value
Product Purchase
Propensity Score
Targeted
Retentions
Strategies
Existing Customer Analytics Insights
CDR, IPDR data
Customer Service
Data
Network Data
Usage Based
Profiling Customer Links/
Network Analysis
Drivers of
Satisfaction Network
Performance and
Service Levels
New Analytics from Big Data
Social Media Data and Analysis
Social Media and
Web Data
Sentiment
Analysis Social Media
Influence Analysis Drivers of
Sentiment
Churn Scores
11
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
…but it is all about business value
KS Statistic
Accuracy Ratio
ROC Curve
Gini Coefficient
Implemented Cut-Off
Cut-off neighborhood
Shift
Baseline Population
Current Population
New
Applic
ants
PSI:
Distributional Shift
Scorecard Score
Strategic KPIs
Reduce Costs
Regulations
Compliance
Increase
Revenue
12
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Engineering analytics: Makes it a reality with Big Data + Data Science on top of traditional mathematical models
Various Data
Usage Pattern, Health Monitoring
Alarms & Control, Benchmarking
Predictive Rules for optimal recommendation from connected
Assets
Failure Prediction & Root Cause Analysis
Resource Scheduling.
Design for Reliability
Benchmark data/content Mining
Reduced Order Modeling for high volume Simulation
and Testing
Supplier Risk Modeling
Weight & Cost analytics.
Design, Testing & Production Operation Service
Manufacturer User Service Provider
Analytics
Engines
Mfg & other Guideline
Specification &
Performance
Benchmark Reports
System Topology
Financial
Sensors/Telemetry –
usage, operations
setting, events
/alarms logs, etc.
Failure/ Warranty
Claims
Field/Technical
Inspection Notes
Contract/Service
History
Social Media and
Third Party
Reliability
Testing/Simulation
Supplier/OEM
transactions
Value Chain
First Time Right Product Design Connected Assets, Operations Control &
Predictive Maintenance
Supplier
Medium Volume, Low Speed,
Domain Specific
High Speed, High Volume and
Domain Neutral
Data
Behaviors
13
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
As more data becomes available to the Data Scientist, so does the complexity…
Product Affinities
Private Label Analysis
Customer Purchase Patterns
Campaign Response Effectiveness
Shrinkage & Productivity Analysis
Category Scorecard & Contribution
to GM Promotion Decomposition
Promotion Mix Optimization
Product Price Optimization
Pricing with Consumer
Perception Analysis
Assortments Optimization for
Market Basket Analysis
Shopper Trip Mission Analysis
Shopper Market Basket
Shopper Brand Sentiment Analysis
Product Behavior Scan
Adjacencies Analysis
Out of Shelf Analytics
Scoring of Stores/ Retail
Chains
Cross-Channel Order
Management
Inventory Optimization
at DCs (SCM)
Demand/ Volume
Forecasting
Social Impact on
Category/ Brand
Consumption
Promotion Halo/
Cannibalization
Pricing Elasticity Analysis
Shopper Segmentation
Shopper Demographics
Shopper Loyalty Base
RFM Analysis
Product/ Brand
Switching
Trial & Repeat
Category
Uniqueness,
Popularity Indices
Category Leakage
Tree
Store Clustering
Category/ Brand
Offer Conversion
Cross-Sell
Up-Sell
Shopper
Assortment
Price
Promotion
Product Competition Category
Tactics
Data
Need
s
+
Other
Consumer +
Survey+
Social Data
+
Household
Panel +
Loyalty/ CRM
Data
+
Syndicated +
Promotions
Data
(IRi/ Nielsen)
+
POS Data
+ Campaign +
Shipments +
Public* Data
Public* Data includes
Weather, Census,
Topography,
Ordinance etc
Maturity Stages
14
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Availability of data is changing the way we address some traditional business problem
Pharmaceutical
Companies have used
physical surveys to
identify KOL. Big data
and analytics is
pioneering the way to
use a data driven
objective approach to
identifying and
monitoring KOL
Selection of right KOLs can help in better utilization of these marketing funds
A key success factor for these marketing spends is the correct methodology to
identify KOLs
Managing brand perception for the key Opinion Leaders is crucial for Brand
Management.
15
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
TABLE OF CONTENTS
Data drivers
Technology drivers So what does all of this increased activity
mean to Talent!
16
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Technology
New Technologies are allowing us to
manage this at a fraction of the cost &
faster than ever before.
Tra
ditio
nal
Data
Ware
house
BIG DATA
1/30 of the cost
Data does not have to be isolated in
repressive silos
17
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Big Data technologies are enabling a new approach
Response time
Vo
lum
e
Hadoop
Data warehouses
PB
TB
GB
Hour Min Sec SubSec
In-memory
databases
Event
processing
tools
Real-time
Applications
18
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Data and technology implications…
Model development
Reduced timelines
• Access all data from a „data lake‟
• Data discovery and visualization tools to
reduce EDA timelines
• In-memory/ in-database/ high performance
analytics and parallelized algorithms
Increased analytical capability
• Implement techniques like Graph/ Network
analysis, ensemble methods, matrix
algorithms at scale
• Analyze structured and unstructured data on
one platform
Improved accuracy
• Analyze much larger data sets
• Ability to personalize for a segment of one, for
e.g. targeting).
Model deployment
Seamless deployment (In-database, PMML)
• Decreases error in deployment
Big data deployment
• Analytics on exabytes, scoring in MB/ sec
Real time deployment
• Response (alert/ recommendations) in
milliseconds or less
Adaptive, machine learning algorithms
• “learn” and respond to recent events
Availability and velocity of data leads to
change in analytical approach
• for e.g. Can move from „complex algorithms
for precision prediction of failure modes‟ to
„real time monitoring, alerts and control
processes‟.
19
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Decreased
response time
Customer experience
Information is becoming the
new battleground
Business expectation
20
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Analytics is playing an ever important role
Increased Focus on identifying the
customer across all channels
Segmentation to Micro segmentation
to the individual
Personalized Messaging and offers –
Increased Individual Customer Centricity
Gradual evolution of Customer Analytics
Past
Customer segments who are
most likely to respond to
targeted campaigns for new
products offers
Can tailor offers to specific to
each customer segment
Mostly delivered through mass
mail campaigns and in store
promotions.
Now
Micro segmentation
Analyze customer behavior
and buying patterns across
channels
Delivery through email, web,
mass mail campaigns.
Moving toward
Historical individual customer behavior
and buying patterns across channels
Individual customer consumption
pattern
In-store basket analytics
Additional dimensions Location & time
Targeted Strategies to pre-empt
customers from visiting competition
Instantaneous Delivery in store or a
proactive delivery via mobile to bring
the customer to store.
Segment to Individual to Individual @ time, place and behavior
You have purchased Cheese,
here are the offers on Bagels
You are within 2 KMs of a store offering 50% off garden furniture
Do you need coffee?
21
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Analytics is only as good as the implementation…
Analytics has long excelled in silos …as
the amount of data and business
expectation increases, this will no longer
be feasible
IT will move from a facilitator role to an
enabler role
Decreased response time will mean end to end integrations – enterprise architecture
teams will need to be involved…
The Data Science team will have to work along with technology teams to effectively
serve the end customer
22
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
much of which is outside the
organization
Increased availability of data
Analytics as a Service and Data
Monetization
New service models
Decreasing Time value of data!
23
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
TABLE OF CONTENTS
Data drivers
Technology drivers
So what does all of this
increased activity mean to
Talent !
24
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
Scalability and industrialization to address skill shortage
Technical skills (Coding, Statistics,
Math)
+ Perseverance
+Creativity
+ Intuition
+Presentation Skills
+Business Savvy
= Great Data Scientist!
Key to a Great Data Scientist
Identified four Data Scientist clusters based
on how data scientists think about
themselves and their work, not
• Years of experience,
• Academic degrees, favorite tools
• Titles, pay scales, org charts.
Most successful data scientists are
those with substantial, deep expertise
in at least one aspect of data science,
be it statistics, big data, or business
communication
T-Shaped Skills.
25
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
…so can analytics solve all our problems
Help us acquire customers Product Recommendation
engine
Solve World Hunger Crop Sciences
Keep us fit!
Catch the bad guys Numbers
Win FIFA “German national football team uses real
time analytics for a competitive edge”
Get you married! Dating sites, Matchmaking
Analytics in Healthcare
26
Big Data & Analytics
Copyright © 2014 Capgemini. All rights reserved.
Impact of Big Data on Analytics | Mamatha Upadhyaya
So what is it Big Data and Analytics cannot do!!!
The information contained in this presentation is proprietary.
Copyright © 2014 Capgemini. All rights reserved.
Rightshore® is a trademark belonging to Capgemini.
www.capgemini.com/bim
About Capgemini
With more than 130,000 people in over 40 countries, Capgemini
is one of the world's foremost providers of consulting, technology
and outsourcing services. The Group reported 2013 global
revenues of EUR 10.1 billion.
Together with its clients, Capgemini creates and delivers
business and technology solutions that fit their needs and drive
the results they want. A deeply multicultural organization,
Capgemini has developed its own way of working, the
Collaborative Business Experience™, and draws on Rightshore®,
its worldwide delivery model.