Big data may 2012

Big Dataopportunities forMarket Research

Q. How big

is Big Data ?

Byte B 100 1Kilobyte KB 103 1,000Megabyte MB 106 1,000,000Gigabyte GB 109 1,000,000,000Terabyte TB 1012 1,000,000,000,000Petabyte PB 1015 1,000,000,000,000,000Exabyte EB 1018 1,000,000,000,000,000,000

Computer Science 101

A. Bigger than Shakespeare?

t1B

x 1000 =

1 KB

x 1000 =

1 MB

x 5 =

5 MB

A. Bigger than your pocket?

x 100 =

500 MB

x 2 =

1 GB

x 60 =

60 GB

5 MB

A. Bigger than the known universe?

60GB

1 TB

x 140 =

140TB

x 20 =

A. Bigger than a day at Google?

2.5PB

x 5-10 =

140TB

x 11 =

13PB/year

1.5PB

x 1.5 =

20PB/day

or

A. Bigger than the sum of human knowledge?

x 250 =

All words ever uttered by the human race since the beginning of time

5EB

20PB/day

A. Bigger than the Internet?

x 100 =

All words ever uttered by the human race since the beginning of time

5EB

500EB

All data to flow across the Internet this year

Pause to think…

These were the biggest data sets I could find statistics for

and both would be good raw material for Market Research

if we could find a big enough table to put them in

There is a simpler answer

Q. How big is Big Data?

A. Bigger than we can easily handle

(and usually unstructured)

Why now?

More activities are digital, creating “data exhaust”

More sensor devices creating digital data: “chips with everything”

More connectivity: data can be networked

Storage is cheap and getting cheaper

Big Data means different things

Scientists: new frontiers of knowledge

IT industry: projects > 1 PB

Investors: opportunity for growth

Commerce: efficiency, decision-making

Google: business as usual

Market leaders in commercial Big Data

Data ownership

Data Analytics

Data Storage

Commercial applications for Big Data

Micro-segmentation / mass customisation

Predictive propensity modelling

Digital marketingPricing optimisation

Operational performance improvement

Forecasting

Product improvement / development

The Big Data hypothesis for Market Research

“The availability of large quantities of consumer data

will allow us to generate new and/or lower cost

consumer insights through analysis of that data”

Big Data sets for Consumer Insight

Social media

Web traffic

Transactional

Geodemographic& geolocation

And let’s not forget qual and ethnography

Social media

Blogs

A change in research process and mindset

Controllable sample

Extendable conclusions

Data on real world outcomes

Statistics

Analytics

Actionable insights

Hypothesis-led / inductive

Fact-led /Deductive

Transferable Research skills

Understanding client needs

Asking/framing the right questions

Knowing what to look for

Interpretation

Synthesising insights

And researchers have a grasp of statistical techniques used in data analysis

Pattern recognition

Trend analysis

Classification

Cluster analysis

Regression analysis

Big Data firms want a piece of our action

Google Consumer Surveys

Facebook research

Dunnhumby entered the Honomichl 100

Nectar are launching an online panel

Big Data tells us what, but

not why

How can Researchers respond?

1) Find a friendly data scientist

2) Get involved: understand available data sets

3) Talk to clients: what data? what needs?

4) Get creative: how could the data meet client needs?

5) Experiment (with your friendly data scientist)

6) Complement data with traditional research

Success for Market Research in Big Data =

ResearcherData

scientistTechnology

+ x

No-one is doing this well yet:

there is an open goal for whoever gets it right