The Big Data Story

Cloud IT Better 2

1

2

3

4

5

6

Big Data is getting

Bigger and Bigger !

Why is Cloud Big

Data’s Best Friend ?

Identifying Big Data

Industry Use cases

Figuring Out the

Big Data Life Cycle

How AWS Building Blocks

can Help Tame Big Data!

Cloudlytics – A Big

Data Use Case

So What is Big Data ?

Simply put, Big Data is

data which cannot be

processed by the current

tools or technologies. Big

Data is too Big, too Fast,

too Varied & Too

Unpredictable!

Cloud IT Better 3

* Twitter & Flickr Visualizations in North America

Cloud IT Better 4

The 4Vs - Volume

VOLUME

Cloud IT Better 5

2.5 quintillion

bytes of Data is generated everyday!

40 Zeta bytes of Data Will be created

by 2020!

With 2.4 Trillion GBs

of data created everyday!

Most Companies in the U.S. Have

100TBs of

Data Stored

The 4Vs - Volume

Cloud IT Better 6

The 4Vs - Variety

Cloud IT Better 7

The 4Vs - Velocity

VELOCITY

Cloud IT Better 8

The NY Stock Exchange

Captures 1TB of

Trade Information Every Day

Modern Cars have close to

100 Sensors measuring Fuel Level to Tire Pressure

The 4Vs - Velocity

Cloud IT Better 9

The 4Vs - Veracity

VERACITY

Cloud IT Better 10

27% Respondents

in a Survey were unsure of How much of their Data was inaccurate.

Poor Data Quality

Costs the U.S. $3.1 Trillion per Year!

The 4Vs - Veracity

Big Data is Getting Bigger and BIGGER!

Cloud IT Better 11

“ More data crosses the internet EVERY SECOND than were stored in the entire internet just 20 years ago!“

“ Zuckerberg noted that 1 billion pieces of content are shared via Facebook’s Open Graph DAILY ! “

“ It is estimated that Walmart collects more than 2.5 petabytes of data EVERY HOUR from its customer transactions ”

IMPACT of Big Data Play In Your Industry?

This McKinsey Report says How Big Data Will Impact Different Industries, But Whatever be Your Industry, you can’t survive Without Big Data after the Next 10-15 Years! So Be the Early Bird!

12 Cloud IT Better

Ring Ring? Anyone There?

The Telecom Industry

You have Your Networks Spread over 1000 Cities, some with Over a Million

Connections, Challenges such as:

• Customer Churning & Retention • Understanding Payment Details for new Schemes • Providing Customized Payment & Service models to the Right Customers • Understand Competitor Pricing Models & faster innovation • Checking Which Offers & Schemes are popular in which Geographies

13 Cloud IT Better

Telecom - Opportunities Using Big Data:

Innovative Business Models

Operational Efficiency: (10-15%

Open Reduction)

Precise Business Models

Real Time Analysis & Decision

Making: (Revenue Potential Inc. 5-10%)

Create Data Driven API models for improved customer service.

Creating World Class customer care, by tracking in depth subscriber activity, tracking issues and reducing call center iterations & time.

Optimizing Offers based on Subscriber Network usage patterns & Traffic to come up with newer offers which is critical on driving value added service adoption.

Controlling RAN(Radio Access Network) Congestion by dividing Subscribers to Individual Sub Cell Levels & by assimilating data of past geographic positions & real time data on current locations can provide priority to certain subscribers over others.

Using Payment Data from Retail Chains & Outlets to create Coupons & Offers, Combining them with NFC(Near Field Communication) to increase Buying frequency of Customer

Anticipating & Implementing Network Planning even before the demand & predict Network stress points & Under utilized Network areas.

Help Service providers understand what behaviors will trigger churn events & what actions will prevent churning, by dynamic offers created by complaint triggers in real time. Reducing Churn Rates by 8-12%

Cyber Cop Initiatives where pattern matches of subscriber activities can be used to detect malicious activities and determine traffic changing abnormal consumptions to predict Fraud activities.

14 Cloud IT Better

What are You Buying Today?

The Retail Industry Imagine Your Retail Outlets over a 100s of Cities, with more than 10 outlets per city. Even if you have a 10000 Customers per month buying at a frequency of 5. You will end up With 5,00,00,000 unique records! This Does not even take into Consideration the Age, Gender, Geographic Trend, Whether, Time of the month & more!

Challenges:

The Number Jumble :

• Buying Patterns • Shopping Offers

• Cross Selling Success

• Loyalty & Retention

• Effective Marketing Campaigns • Predictive Demands

• Dynamic Price Optimization

15 Cloud IT Better

Retail - Opportunities Using Big Data: Personalized Recommendations

Dynamic Pricing In Store Experience Micro segmentation & inventory management

Data Collected based on previous online & offline purchases, even online clicks, likes & wish lists are recorded to generate recommendations in real time.

Online Shoppers are given reduced prices based on data on time of the day, Festive offers validity period, loyalty of customers and more.

Geo-Fencing which allows retailers to provide real time offers to customers on their cell phones(based on their previous shopping sprees) as they enter a geo fenced area.

Segmenting Customers have been taken to the next level, with social media interactions, marketing campaign results, wish lists. Targeted offers are now made to granular customer segments with promo codes & coupons!

Online Shoppers are given recommendations at reduced prices based on their previous purchase trends. Amazon.com has increased their sales volumes by 25% on this.

Offline shoppers use these data driven approaches to map shopping patterns and based on proximity of customers allow price variations with RFID price tags. Check out Our Blogs for MORE

Optimized Product Placement is done Scientifically where algorithms check buyer tendencies & have products placed in the right geography, this becomes very important for bigger players like Wal-Mart & Macy's.

With Big Data Retailers can get predictive analytics on prices as they fluctuate through the supply chain. This allows them to set prices, and also react proactively to demand spikes to avoid over stock-outs.

16 Cloud IT Better

Will My Policy cover this Accident ?

Image Courtesy: blairingle

Challenges - Insurance Industry

$80 Billion LOSS in U.S. per Year due to Frauds!

15% of premium costs in South Africa are Frauds!

Claims on Automobiles have a 25-33% Fraud !

Are Your

Claims

Fraud?

• What Policies are best Suited for Customers?

• How to tackle Increasing Diseases & ailments?

• How to reduce customer Hassles?

17 Cloud IT Better

Insurance - Opportunities Using Big Data:

Fraud Detection Turning the Claim Centric Approach Person Centric. Using Cohort analytics to track Social activities of beneficiary & associated parties. Integration of these Data streams of Information to detect fraud patterns for Future (predictive analysis)

Delighting the Customer

Variety of Customer records can be stored in No SQL databases, and real time integration to multiple sources to optimize the process of the insurance validation & reimbursements. Customer Call Logs & interaction with staff can be checked for Sentiment analysis, to optimize insurance processes & reduce iterations base on negative customer calls.

Predictive Analysis Understand Customer Lifestyle by integrating feeds of social networks to determine Disease Patterns so that insurance schemes 10-15 years into the future insurance companies can identify these degenerating lifestyles & offer schemes at higher premiums.

Improving Product Opportunities

Checking out the Success of Insurance Schemes & Which are most popular, to drive similar scheme models & understand why other schemes are not popular. This can be done by mapping the successful customer base lifestyle trends.

18 Cloud IT Better

I Want My Car to Drive on its OWN!

Connected Cars is No Longer a Concept!

Number of Internet Capable Vehicles

in Europe 48Mil by 2016!

There are more than 74 Sensors in Ford’s Connected Cars!

These Hybrid Cars can generate

25GBs/Hr of Data!!

19 Cloud IT Better

Automotive - Opportunities Using Big Data Vehicle Insurance

Using "Telematics", driver's driving patterns can be analyzed. These can be used by insurance companies to give out alerts, warnings in real time & even give personalized pricing.

Integrating with Geo Fencing & Social Media

GPS trackers can provide customized alerts as vehicles pass a particular location. These alerts are real time & based on your social media likes & shared combined with discount coupons & offers!

Self Repair & Maintenance

Your Car's intelligence system will keep a track of all parts & liquids to be changed or repaired for periodic maintenance, giving you real time alerts as you pass repair shops, which will also bid for discounted pricing!

Learning from The mistakes

Using The Black Box mechanism similar to airplanes, product engineers can understand if any vehicle part was the cause & how parts can be improved in design to reduce future accidents.

20 Cloud IT Better

Don’t Watch that Movie! It’s Pathetic!

Challenges – The Media Industry

• What Content is popular?

Did You Know? You Tube Users Upload 48 Hrs of Video Every Minute!

• Where are my viewers coming from?

• What are my viewers opinions about my content?

• How Do I monetize my Content?

21 Cloud IT Better

Media - Opportunities Using Big Data Predictive analysis

Using Big Data Tools to analyze current content viewed, the storylines, characters etc. to determine which type of movies and or soaps are going to be a success in the future.

Log Analysis Analyzing Viewer demographics, the popular content, devices used to view and download data, detecting spams, browsers and OS used to generate actionable reports driving business decisions.

Sentiment Analysis

Tracking user comments, likes, shares, tweets & other user interactions with media content on social networks to track popularity & promote content similar to the hits.

Website Optimization

Based on Navigational pattern analysis that are popular among users, website builders can optimize the ease of reach of the content.

Ad Targeting & Scope to Monetize

Ad servers based on Visitor Cookie analysis & bid values generated sometimes even in real time, deliver ads to visitors in real time & continuously update for successful clicks & failures. Learn more on Ad serving with AWS Cloud.

22 Cloud IT Better

When will my Loan Get Sanctioned ?

23 Cloud IT Better

Challenges – The Banking Industry

• Detecting Frauds

• Which Schemes for which customers?

• When is my Customer Not Happy?

• How Do I Segment my customer base?

Banking - Opportunities Using Big Data

Creating Customer Segmentation

Banks are now pooling in all types of customer buying patterns, lifestyle habits & interests to create segmentations. With this 360 degree view from Big Data Analytics integration, banks can now customize product offerings & re distribute spending from non profitable to profitable customers.

Fraud Detection

Financial & Banking Institutes are using credit/debit card purchases to understand spending habits and detect suspicious patterns of buying to detect frauds.

Customer Sentiment Analysis

Banks and Financial Institutions can now track the the success or failure of their product or schemes as they integrate social sentiments of their products and track user complaints.

Sales And Marketing Campaigns

Using 360 Degree Customer Insights banks are generating smarter marketing & sales campaigns integrating them with offers & schemes that are more successful to the different customer segments.

Analyzing Voice Sentiments

Many Banks are now using highly unstructured data, such as customer voices, and using complex data analysis to track customer complaints . They are also trying to integrate the information with the transactional data warehouse to reduce attrition, drive up sales & even detect frauds.

24 Cloud IT Better

Let us Figure out the Big Data Life Cycle

In order to make the entire process of Big Data more tangible, it is divided

into 4 stages:

Generation

Collection

& Store

Analyze &

Computation

Data

Collaboration

& Sharing

25 Cloud IT Better

Generation

Generating the Data

Structured Data – Employee Records Semi Structured Data – End User Logs Unstructured Data – Social User Profile images

Data Mining

Log file analysis

Machine learning

Web indexing

Financial

analysis

Scientific

simulations

Data

warehousing Bioinformatics

research

Web based APIs can be used to access this data and Store it.

Cloud IT Better 26

Fitting AWS Cloud Components

AWS Direct Connect

AWS Storage Gateway

AWS Import/Export

Establish a dedicated network connection from your premises to AWS

Secure Integration between an On-premises IT & AWS’s storage infrastructure

Move large amounts of data into and out of AWS using portable storage devices for transport

Cloud IT Better 27

Transferring Your Data to AWS Cloud

AWS Direct Connect

AWS Storage Gateway

AWS Import/Export

Establish a dedicated network connection from your premises to AWS

Secure Integration between an On-premises IT & AWS’s storage infrastructure

Move large amounts of data into and out of AWS using portable storage

devices for transport

Cloud IT Better 28

Collecting & Storing Data on AWS Cloud

Write, read, and delete objects containing from 1 byte to 5 terabytes

of data each.

A full featured relational databases giving you access to capabilities of a MySQL,

Oracle, SQL Server, or PostgreSQL databases engines.

Relational Database Service (RDS)

A fast, fully managed NoSQL database service making it simple & cost-effective to store & retrieve

any amount of data, and serve any level of request traffic.

AWS DynamoDB

Cloud IT Better 29

Simple Storage Service (S3)

Data Analysis, Retrieval & Automation

Amazon Elastic Map Reduce (EMR)

A managed Hadoop distribution by Amazon Web Services using

customized Apache Hadoop framework. It integrates with

AWS S3 & EC2.

Cloud IT Better 30

Amazon Redshift is a fast, fully managed, petabyte-scale data

warehouse service making it simple & cost-effective to efficiently analyze

all your data using your existing business intelligence tools.

AWS Redshift

This allows users to define a dependent chain of data

sources and destinations with an option to create data

processing activities called pipeline.

AWS Data Pipelines

ANALYSIS RETRIEVAL AUTOMATION

AWS Kinesis (Big Data in Real Time)

Amazon Kinesis is a fully managed service for real-time processing of streaming data at

massive scale. Amazon Kinesis can collect and process hundreds of TBs of data/hr from hundreds of thousands of sources.

• Real Time Processing allowing you to answer questions about the current state

of your data.

• Amazon Kinesis automatically provisions & manages the storage required to reliably & durably collect your data stream. • Your Kinesis Streams are connected to your Kinesis

App from which you can use DynamoDB or Redshift to process complex queries at real Time.

Image courtesy: https://static.gosquared.com/images/liquidicity/kinesis/

Cloud IT Better 31

The Big Data Life cycle - Compiled

AWS S3 AWS RDS AWS DynamoDB

AWS EMR

AWS Data Pipeline

Generation

Collection

& Store

Analyze &

Computation

Data

Collaboration

& Sharing

AWS S3 AWS RDS

AWS DynamoDB AWS Redshift

AWS Data Pipeline

AWS Data Pipeline

Cloud IT Better 32

Use Case - Cloudlytics

Cloudlytics is a Pay-as-you-Go, SaaS based Log Analytics Tool powered by AWS. It Takes the Big Data Approach using AWS Components such as EMR & Redshift.

Customer Log Files Stored in S3

Processing Processed Data

Customer Reports

Cloud IT Better 33

Let’s Discuss Your Specific Business Use Case. Contact Us

www.blazeclan.com

Follow Us On :

Our Blog : http://blog.blazeclan.com/

Contact us : info@blazeclan.com

Cloud IT Better 34