8

Click here to load reader

An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

Embed Size (px)

Citation preview

Page 1: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

An inside look at the eMarketer Methodology

®

eMarketer Estimates & Projections

The power of eMarketer’s research derives from

our uniquely comprehensive approach—we look at

and analyze all the relevant data on a given topic.

eMarketer clients do not rely on information and

intelligence from only one or two sources; they get

the widest possible perspective by seeing the best

from over 2,000.

THE FIRST PLACE TO LOOK

Page 2: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

eMarketer’s Unique Approach

There is no such thing as a perfect research

study. No single source can ever have all the

latest available data on a given topic. No research

company can consistently achieve accuracy in

all areas all the time.

This is why eMarketer’s research methodology is founded on a

simple philosophy: an accurate and reliable statistical picture can

only be painted by aggregating and analyzing data from as many

different sources as possible.

Moreover, history and statistical theory confirm that a careful eval-

uation and weighting of multiple sources inevitably yields a more

accurate analysis than a single source ever could.

Unlike many other market research providers, eMarketer has

no data-gathering technique to defend nor do we rely on a single

survey population or respondent base.We look at all of the

relevant data on a given topic and provide a perspective that is

based on the collective wisdom of multiple and diverse research

sources, as well as on our objective expertise and experience.

eMarketer provides clients with a wide variety of critical market

estimates and projections, including, but not limited to, the

following:

■ Number and penetration of Internet users, by region and

country

■ Broadband users, households and penetration, by region

and country

■ Mobile Internet users, by region and country

■ Online advertising spending, including totals and individual

components such as search, display ads, rich media, video, etc.

■ Spending on mobile marketing and advertising

■ B2C e-commerce sales, by region and country

■ Number and penetration of online shoppers and buyers

■ PVR,VOD, HDTV and IPTV households and penetration

■ Digital TV, by region and country

The Business Need for Numbers

Quantitative data are important in business for

a variety of obvious (and some not-so-obvious)

reasons. Business executives commonly use

statistics to:

Comparative Estimates: US Online AdvertisingSpending, 2005 (in billions)

Forrester Research*, August 2005

$14.7

JMP Securities, December 2005

$13.2

eMarketer, January 2006

$12.9

Piper Jaffray, April 2005

$12.8

Deutsche Bank, April 2005

$12.6

Veronis Suhler Stevenson, August 2005

$12.6

J.P. Morgan, January 2005

$12.5

SG Cowen, December 2005

$12.5

Merrill Lynch, June 2005

$12.4

Morgan Stanley, September 2005

$12.2

JupiterResearch, August 2005

$11.9

PricewaterhouseCoopers, June 2005

$11.5

Myers Report, September 2005

$11.1

The Kelsey Group, February 2005

$10.1

TNS Media Intelligence**, June 2005

$8.0

Universal McCann**, December 2005

$7.9

Note: eMarketer benchmarks its US Internet ad spending projectionsagainst the Interactive Advertising Bureau (IAB)/PricewaterhouseCoopers(PwC) data, for which the last full year measured was 2004; *includese-mail marketing along with media spending; **does not include paidsearch spendingSource: eMarketer, January 2006; various, as noted, 2005

069370 ©2006 eMarketer, Inc. www.eMarketer.com

2 www.emarketer.com

Page 3: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

■ make better, more informed business decisions;

■ answer the questions raised during strategic planning;

■ quantify markets and industries and their potential;

■ identify important trends that can benefit their businesses;

■ measure critical performance benchmarks;

■ understand the potential impact of new or evolving markets;

■ identify potential competitive threats and

■ support key business proposals and recommendations with

hard data ie to make the business case.

With these uses in mind, eMarketer aggregates market research

data from as many sources as possible on every topic we cover.

Then our analyst team provides objective analysis around the

numbers and trends, as well as generating their own market

estimates and projections.

With eMarketer’s aggregation approach business professionals

can fill in data gaps and obtain a unique picture of the present and

future to support business decision-making.

The Problem of Conflicting Numbers

Research statistics, particularly those focused on

Internet markets and trends, are often confusing.

The estimates and projections offered by research

and analyst firms can vary significantly, even when

they supposedly relate to the same variable.

The reasons for these disparities fall into three broad areas:

A. Researchers use different definitions of the variables reported

B. Researchers use different methodologies when they develop

studies, surveys and projections

C. Researchers are inevitably influenced by subjective factors

when they generate estimates and projections

A. Different DefinitionsA good deal of the variance between researchers’ estimates and

projections is explained by the use of different definitions. Even

when the same name or term is used, the actual variable that is

being measured can be significantly different.

For example, while most research firms and banking analysts

include online paid search in their estimates for online advertising

growth, at least two well-known research outfits do not:TNS Media

Intelligence and Universal McCann.With search being the largest

and fastest-growing segment, it is no wonder that the estimates

from these two companies are the lowest among 16 sources.

As another example of how differing definitions can affect results,

consider online consumer commerce.The estimates from

numerous research firms can vary by billions of dollars because

they either include or exclude various components such as travel,

auction sales or automobile sales.

Comparative Estimates: US Online AdvertisingSpending Growth, 2005 (as a % increase vs. prior year)

JMP Securities, December 2005

37.5%

eMarketer, January 2006

33.7%

Veronis Suhler Stevenson, August 2005

31.2%

Deutsche Bank, April 2005

30.0%

Myers Report, September 2005

30.0%

SG Cowen, December 2005

29.5%

The Kelsey Group, February 2005

29.5%

JupiterResearch, August 2005

29.3%

Piper Jaffray, April 2005

28.8%

Merrill Lynch, June 2005

27.8%

Morgan Stanley, September 2005

26.8%

J.P. Morgan, January 2005

22.9%

Forrester Research*, August 2005

22.6%

PricewaterhouseCoopers, June 2005

20.3%

Universal McCann**, December 2005

15.0%

TNS Media Intelligence**, June 2005

7.0%

Note: Growth figures were calculated by eMarketer based on the dollarfigures provided by each research firm; eMarketer benchmarks its USInternet ad spending projections against the Interactive Advertising Bureau(IAB)/PricewaterhouseCoopers (PwC) data, for which the last full yearmeasured was 2004; *includes e-mail marketing along with mediaspending; **does not include paid search spendingSource: eMarketer, January 2006; various, as noted, 2005

069357 ©2006 eMarketer, Inc. www.eMarketer.com

THE FIRST PLACE TO LOOK 3

Page 4: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

B. Different MethodologiesNot all research is created equal. Research firms employing

different methodologies and measurement techniques will

usually arrive at different results.

For example, a nationally conducted survey using random

digit-dialing to reach 10,000 respondents via telephone will

typically yield more accurate results than a 30-person survey

conducted among a self-selected group of people online.

eMarketer evaluates all the components of a study or survey,

including:

■ Sample size;

■ Other characteristics of the population being studied;

■ Method of respondent recruitment and any incentives used;

■ Statistical sampling methods employed;

■ Timing of survey or study and

■ Any sponsorship or other vested interest that might

influence results

As an illustration of how different methodologies can produce

dramatically different results consider estimates and projections

of the number of podcast listeners in the US.The Diffusion Group

put the total number of US podcast listeners at 4.5 million in 2005

and expected the total to hit nearly 57 million by 2010.

The likely accuracy of these numbers was confirmed in November

2005, when Bridge Ratings published a forecast for the US podcast

audience, putting the total number of listeners at 5 million in 2005,

growing to 63 million by 2010.

These figures, however, do not completely meet the needs of

eMarketer’s clients. It is the active audience that will potentially

respond to advertising and marketing.The totals given above refer

to those who have ever listened to a podcast. By incorporating

data on those who listen to podcasts at least once a week,

eMarketer showed that the active podcast audience was

significantly smaller than implied by the earlier estimates.

C. Guesswork and SubjectivityResearchers attempting to measure any aspect of Internet

activity or e-business will inevitably have to supplement their

raw data with qualitative judgments.These involve varying levels

of interpretation, extrapolation, explanation or prediction.This

process inherently entails a degree of subjectivity, introducing

the possibility of bias and even potential conflicts of interest.

The effect of subjective factors is amplified when researchers are

trying to predict the future.The further out the forecast horizon,

the more assumptions that have to be made about interrelated

future events. Predicting the number of wireless Internet users

next year is hard enough; trying to pick a number for 2010 can be

akin to a shot in the dark. Such projections are typically based on a

set of related assumptions about future events. However, without

a clear explanation of those assumptions and their relevance to

the forecast, their use to business decision-makers is still limited.

US Podcast Audience, 2006, 2008 & 2010 (millions)

Total podcast audience*

10.0

25.0

50.0

Active podcast audience**

3.0

7.5

15.0

2006 2008 2010

Note: *individuals who have ever downloaded a podcast; **individualswho download an average of one or more podcast(s) per weekSource: eMarketer, February 2006

070756 ©2006 eMarketer, Inc. www.eMarketer.com

US Podcast Audience, 2004-2010 (millions)

2004 0.8

2005 4.5

2006 11.4

2007 21.7

2008 32.9

2009 44.1

2010 56.8

Note: CAGR=101%Source: The Diffusion Group, May 2005

065839 ©2006 eMarketer, Inc. www.eMarketer.com

4 www.emarketer.com

Page 5: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

How eMarketer Makes Its Own Projections

eMarketer employs a four-step forecasting

process.This takes into account all of the avail-

able research data on a given market or trend.

Aggregate > Filter > Organize > AnalyzeThe process consists of aggregating data from as many sources

as possible, then normalizing, weighting and evaluating the data

and, finally, creating an estimate or projection that has the best fit

with all of the information available.

Step 1:AggregateeMarketer aggregates data from over 2,000 sources, including

research firms, consultancies, government agencies, non-profit

organizations and investment banking firms.This breadth and

depth of data yields a richer, far more complete picture of the

market than any single source can provide.

The Importance of Looking at All the Available Data:

■ There is no such thing as a perfect research study or survey

■ No single source can provide all the answers

■ A careful evaluation and weighting of multiple sources always

leads to a more accurate picture of reality

■ Critical business decisions should be based on complete,

up-to-date and reliable information

Step 2: NormalizeMerely lumping together raw numbers from various research

firms would inevitably result in apples-to-oranges comparisons.

eMarketer seeks to “normalize” the data by first understanding

and then adjusting for the different definitions used by the

research firms.

For example, when estimating online consumer retail spending,

it is important to note that some research firms include travel,

while others, including the US Department of Commerce, do not.

Similarly, while most researchers include paid search in their

estimates for online ad spending, at least two widely-quoted

research firms do not.This makes a significant difference because

paid search accounts for about 40% of spending on online

advertising and it is the fastest growing segment within it.

Step 3:Weight (the Data)eMarketer analysts carefully weight the methodologies and

results of different research firms based on a number of factors,

including (but not limited to) the following:

■ General reputation and credibility of each research organization

■ Historical accuracy of research sources, based on a comparison

between projected and actual results (utilizing eMarketer’s

eStat Database)

■ The sophistication and sample size of the study or survey,

as well as the overall research design and respondent

recruitment technique

In the weighting process, eMarketer will also look at the degree of

convergence between different estimates. In some cases numbers

from different sources line up very closely.This means the reader

can have a higher degree of confidence in the data. For example,

leading research firms consistently estimate that the online travel

market in the US was approximately $60 billion in 2005.

Comparative Estimates: US Online Travel Sales,2003-2010 (in billions)

comScore, January 2005 (1)

Forrester Research, May 2005 (2)

JupiterResearch, November 2005

Merrill Lynch, August 2005 (3)

PhoCusWright, October 2005 (4)

2003

$40.4

-

-

-

$40.3

2004

$50.9

$52.4

$57.0

-

$52.1

2005

-

$62.8

$68.0

$60.2

$65.4

2006

-

-

-

-

$78.8

2007

-

-

-

-

$93.6

2010

-

$119.0

$104.0

-

-

Note: (1) consumer unmanaged online travel purchases, the majority ofwhich are leisure; (2) leisure travel only; (3) this figure combines salesestimates for online travel agencies and direct suppliers; (4)leisure/unmanaged business travelSource: various, as noted, 2004 & 2005

067838 ©2005 eMarketer, Inc. www.eMarketer.com

THE FIRST PLACE TO LOOK 5

Page 6: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

Step 4: Evaluate (the Bigger Picture)Focusing only on the estimates from other researchers, however,

does not necessarily take into account the latest trends or

developments in the marketplace. Before creating an estimate

or projection, eMarketer also looks at the broader economic,

technological, market and cultural trends that affect the area

being measured.

In many cases, eMarketer will also look at individual company

data. For instance, when measuring broadband households,

eMarketer tabulates subscriber figures from the leading

broadband providers in each market to create a bottom-up

“reality check” for total number of broadband households.

Similarly, when estimating the online paid search market,

eMarketer analysts examine the revenue figures from leading

players such as Yahoo, Google, MSN and AOL.

Step 5: Estimate All of the above is factored into our forecasts. In many cases it is

incorporated into an integrated statistical model, along with

data on other related trends. Our analysts then create estimates

and projections that represent the best fit with all the available

information on the market being measured.

Importantly, this process is a continuous and self-improving

feedback loop.With each new data point that becomes available,

eMarketer analysts compare the new information with the

existing body of evidence and reassess their statistics.

As a result of this five-step process, eMarketer’s estimates and

projections are frequently among the most accurate available.

Projections, Benchmarking & Accountability

There is not much point in making predictions

about the future unless you have a means of

measuring actual results.Without a standard of

measurement, any researcher could claim that

their projections are the most accurate.

eMarketer recognizes that certain government departments,

non-profit organizations and other respected, impartial sources

are able to provide reliable numbers that can be consistently

tracked over time. Most of these established sources, however,

only measure past results; typically they do not predict the future.

Wherever possible, eMarketer identifies a reputable third-party

source to serve as a “benchmark” for future projections.

For example, eMarketer has selected the International

Telecommunication Union (ITU) as the benchmark source for its

Internet User forecasts.The ITU has been measuring the Internet

sector in nearly every country in the world since 2000, and is

regarded as a trusted impartial source for international telecom-

munications statistics.The ITU does not make predictions, but

measures past levels of Internet penetration using information

gathered directly from national statistical agencies and local

research firms.

The use of internationally-accepted and independent benchmark

sources for historical data means our estimates and projections

can be measured against an accepted standard.As a result we

are more accountable for all of our data.

A Few of the Benchmark Sources eMarketer Uses for its Projections:

Internet users ITU (worldwide), Department of

Commerce (US)

Online advertising Interactive Advertising Bureau

spending (IAB) & PricewaterhouseCoopers

US total media spending Universal McCann

US B2C e-commerce sales US Department of Commerce

Mobile phone subscribers CTIA

6 www.emarketer.com

Page 7: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

Building Data Models

These principles and techniques are used to

construct the data models which yield

eMarketer’s estimates and projections.

Example: How Many Online Shoppers & Buyers in 2005, 2010?eMarketer has built a demand-side model for consumer online

buying in several countries, including the US and Canada. Each

country model relates total Internet users to those shopping

and buying online (as well as to average spending and total dollar

purchases).The models are based on a continuous supply of new

data compiled from dozens of research and government sources.

Each research source provides varying degrees of information on

the following interrelated metrics:

■ Country population (for US, data comes from the most recent US

Census figures)

■ Number online in the country

■ Number of online “shoppers”

■ Number of online “buyers”

■ Percentage of Internet users who shop online

■ Percentage of Internet users who buy online

■ Percentage of the entire population who shop and buy online

■ Average annual online spending per buyer

eMarketer analyzes this information, accounting for differences in

definitions and methodologies, to construct models for Internet

purchase behavior. Here is a sample grid of eMarketer’s online

buying model for the United States, covering the years 2003–2008:

US Consumer Online Buying and Shopping Grid,2003-2008 (in millions and % penetration)

Internet users*

Online shoppers

Online buyers

Retail e-commerce

Total retaile-commerce

US population ages14+ (US Census)

Total US Internetusers

Internet users ages14+

Online penetrationamong populationages 14+

Online shoppersages 14+

as a % of Internetusers ages 14+

Online buyers ages14+

as a % of Internetusers ages 14+

Retail e-commerce(excluding travel)

Online travelrevenues

Average annualretail purchase peronline buyer (not inmillions; excludingonline travel)

2003

233.8

164.3

142.3

60.9%

104.2

73.2%

83.0

58.3%

$55,731

$38,012

$93,743

$672

2004

236.5

170.1

147.3

62.3%

109.4

74.3%

89.9

61.0%

$69,238

$48,503

$117,741

$771

2005

239.3

175.4

151.7

63.4%

114.2

75.3%

96.2

63.4%

$86,603

$58,010

$144,613

$900

2006

241.9

180.1

155.2

64.2%

118.1

76.1%

101.7

65.5%

$105,158

$68,510

$173,668

$1,034

2007

244.4

184.6

158.4

64.8%

121.8

76.9%

106.8

67.4%

$126,400

$80,225

$206,625

$1,184

2008

246.8

188.5

161.2

65.3%

124.9

77.5%

111.4

69.1%

$150,417

$93,141

$243,558

$1,350

Note: *eMarketer's 2003 baseline is from the Department of Commerce'sOctober 2003 estimate of Internet users who had access to the Internet atthe time of the survey; **eMarketer benchmarks its retail e-commercerevenues figures against US Department of Commerce data, for which thelast period measured was Q3 2005; the travel component was formulatedbased on aggregated dataSource: eMarketer, December 2005

069272 ©2005 eMarketer, Inc. www.eMarketer.com

The Benefits of eMarketer’sAggregation Approach toMarket Research

As a result of eMarketer’s unique approach,

the information we provide to our clients is:

■ more comprehensive, objective and ultimately accurate

than any single research source can provide;

■ available all in one place, making it easy to locate, evaluate

and compare;

■ presented along with clear analysis, which makes sense

of all the disparate figures and interrelated trends, and,

therefore

■ a uniquely-powerful decision-support tool.

THE FIRST PLACE TO LOOK 7

Page 8: An inside look at the eMarketer Methodology eMarketer ...assets.csom.umn.edu/assets/110437.pdf · An inside look at the eMarketer Methodology ® eMarketer Estimates & Projections

®

©2006 eMarketer Inc.All rights reserved.

is The First Place to Look for market

research information related to the Internet,e-business,

online marketing and emerging technologies.

eMarketer aggregates and analyzes e-business research from

over 2,000 sources, and brings it together in analyst reports,

daily research articles and the eStat Database – the most

comprehensive database of e-business and online marketing

statistics in the world.

eMarketer’s Core Expertise eMarketer specializes in researching and sorting vast amounts

of publicly-available information, and objectively compiling and

analyzing this information into widely read reports, articles and

newsletters. Our information products help business executives

worldwide make smarter, faster decisions about online marketing,

emerging technologies and e-business.

Dedicated Team Our team of researchers and analysts comb through Web sites,

data repositories and government statistics, uniquely providing

a 360-degree overview of available data, combined with original

analysis that is quickly accessible, comprehensive, objective,

actionable, cost-effective and, most of all, intelligent.

A Trusted Resource eMarketer serves as a trusted, third-party resource, cutting

through the clutter and hype – helping businesses make sense

of the e-business numbers and trends. eMarketer's products and

services help companies make better, more informed business

decisions by:

■ Streamlining e-business research sources and reducing costs

■ Eliminating critical data gaps

■ Providing an objective, bird’s eye view of the entire e-business

landscape

■ Better deploying and sharing information across the company

■ Building solid business cases backed up by hard data

■ Reducing business risk

■ Saving valuable time

For more information about subscribing toeMarketer, contact:

eMarketer, Inc. Toll-Free: 800-405-0844

75 Broad Street Outside the US: 212-763-6010

32nd floor Fax: 212-763-6020

New York, NY 10004 [email protected]

About eMarketer