Click here to load reader
Upload
duonghanh
View
213
Download
1
Embed Size (px)
Citation preview
An inside look at the eMarketer Methodology
®
eMarketer Estimates & Projections
The power of eMarketer’s research derives from
our uniquely comprehensive approach—we look at
and analyze all the relevant data on a given topic.
eMarketer clients do not rely on information and
intelligence from only one or two sources; they get
the widest possible perspective by seeing the best
from over 2,000.
THE FIRST PLACE TO LOOK
eMarketer’s Unique Approach
There is no such thing as a perfect research
study. No single source can ever have all the
latest available data on a given topic. No research
company can consistently achieve accuracy in
all areas all the time.
This is why eMarketer’s research methodology is founded on a
simple philosophy: an accurate and reliable statistical picture can
only be painted by aggregating and analyzing data from as many
different sources as possible.
Moreover, history and statistical theory confirm that a careful eval-
uation and weighting of multiple sources inevitably yields a more
accurate analysis than a single source ever could.
Unlike many other market research providers, eMarketer has
no data-gathering technique to defend nor do we rely on a single
survey population or respondent base.We look at all of the
relevant data on a given topic and provide a perspective that is
based on the collective wisdom of multiple and diverse research
sources, as well as on our objective expertise and experience.
eMarketer provides clients with a wide variety of critical market
estimates and projections, including, but not limited to, the
following:
■ Number and penetration of Internet users, by region and
country
■ Broadband users, households and penetration, by region
and country
■ Mobile Internet users, by region and country
■ Online advertising spending, including totals and individual
components such as search, display ads, rich media, video, etc.
■ Spending on mobile marketing and advertising
■ B2C e-commerce sales, by region and country
■ Number and penetration of online shoppers and buyers
■ PVR,VOD, HDTV and IPTV households and penetration
■ Digital TV, by region and country
The Business Need for Numbers
Quantitative data are important in business for
a variety of obvious (and some not-so-obvious)
reasons. Business executives commonly use
statistics to:
Comparative Estimates: US Online AdvertisingSpending, 2005 (in billions)
Forrester Research*, August 2005
$14.7
JMP Securities, December 2005
$13.2
eMarketer, January 2006
$12.9
Piper Jaffray, April 2005
$12.8
Deutsche Bank, April 2005
$12.6
Veronis Suhler Stevenson, August 2005
$12.6
J.P. Morgan, January 2005
$12.5
SG Cowen, December 2005
$12.5
Merrill Lynch, June 2005
$12.4
Morgan Stanley, September 2005
$12.2
JupiterResearch, August 2005
$11.9
PricewaterhouseCoopers, June 2005
$11.5
Myers Report, September 2005
$11.1
The Kelsey Group, February 2005
$10.1
TNS Media Intelligence**, June 2005
$8.0
Universal McCann**, December 2005
$7.9
Note: eMarketer benchmarks its US Internet ad spending projectionsagainst the Interactive Advertising Bureau (IAB)/PricewaterhouseCoopers(PwC) data, for which the last full year measured was 2004; *includese-mail marketing along with media spending; **does not include paidsearch spendingSource: eMarketer, January 2006; various, as noted, 2005
069370 ©2006 eMarketer, Inc. www.eMarketer.com
2 www.emarketer.com
■ make better, more informed business decisions;
■ answer the questions raised during strategic planning;
■ quantify markets and industries and their potential;
■ identify important trends that can benefit their businesses;
■ measure critical performance benchmarks;
■ understand the potential impact of new or evolving markets;
■ identify potential competitive threats and
■ support key business proposals and recommendations with
hard data ie to make the business case.
With these uses in mind, eMarketer aggregates market research
data from as many sources as possible on every topic we cover.
Then our analyst team provides objective analysis around the
numbers and trends, as well as generating their own market
estimates and projections.
With eMarketer’s aggregation approach business professionals
can fill in data gaps and obtain a unique picture of the present and
future to support business decision-making.
The Problem of Conflicting Numbers
Research statistics, particularly those focused on
Internet markets and trends, are often confusing.
The estimates and projections offered by research
and analyst firms can vary significantly, even when
they supposedly relate to the same variable.
The reasons for these disparities fall into three broad areas:
A. Researchers use different definitions of the variables reported
B. Researchers use different methodologies when they develop
studies, surveys and projections
C. Researchers are inevitably influenced by subjective factors
when they generate estimates and projections
A. Different DefinitionsA good deal of the variance between researchers’ estimates and
projections is explained by the use of different definitions. Even
when the same name or term is used, the actual variable that is
being measured can be significantly different.
For example, while most research firms and banking analysts
include online paid search in their estimates for online advertising
growth, at least two well-known research outfits do not:TNS Media
Intelligence and Universal McCann.With search being the largest
and fastest-growing segment, it is no wonder that the estimates
from these two companies are the lowest among 16 sources.
As another example of how differing definitions can affect results,
consider online consumer commerce.The estimates from
numerous research firms can vary by billions of dollars because
they either include or exclude various components such as travel,
auction sales or automobile sales.
Comparative Estimates: US Online AdvertisingSpending Growth, 2005 (as a % increase vs. prior year)
JMP Securities, December 2005
37.5%
eMarketer, January 2006
33.7%
Veronis Suhler Stevenson, August 2005
31.2%
Deutsche Bank, April 2005
30.0%
Myers Report, September 2005
30.0%
SG Cowen, December 2005
29.5%
The Kelsey Group, February 2005
29.5%
JupiterResearch, August 2005
29.3%
Piper Jaffray, April 2005
28.8%
Merrill Lynch, June 2005
27.8%
Morgan Stanley, September 2005
26.8%
J.P. Morgan, January 2005
22.9%
Forrester Research*, August 2005
22.6%
PricewaterhouseCoopers, June 2005
20.3%
Universal McCann**, December 2005
15.0%
TNS Media Intelligence**, June 2005
7.0%
Note: Growth figures were calculated by eMarketer based on the dollarfigures provided by each research firm; eMarketer benchmarks its USInternet ad spending projections against the Interactive Advertising Bureau(IAB)/PricewaterhouseCoopers (PwC) data, for which the last full yearmeasured was 2004; *includes e-mail marketing along with mediaspending; **does not include paid search spendingSource: eMarketer, January 2006; various, as noted, 2005
069357 ©2006 eMarketer, Inc. www.eMarketer.com
THE FIRST PLACE TO LOOK 3
B. Different MethodologiesNot all research is created equal. Research firms employing
different methodologies and measurement techniques will
usually arrive at different results.
For example, a nationally conducted survey using random
digit-dialing to reach 10,000 respondents via telephone will
typically yield more accurate results than a 30-person survey
conducted among a self-selected group of people online.
eMarketer evaluates all the components of a study or survey,
including:
■ Sample size;
■ Other characteristics of the population being studied;
■ Method of respondent recruitment and any incentives used;
■ Statistical sampling methods employed;
■ Timing of survey or study and
■ Any sponsorship or other vested interest that might
influence results
As an illustration of how different methodologies can produce
dramatically different results consider estimates and projections
of the number of podcast listeners in the US.The Diffusion Group
put the total number of US podcast listeners at 4.5 million in 2005
and expected the total to hit nearly 57 million by 2010.
The likely accuracy of these numbers was confirmed in November
2005, when Bridge Ratings published a forecast for the US podcast
audience, putting the total number of listeners at 5 million in 2005,
growing to 63 million by 2010.
These figures, however, do not completely meet the needs of
eMarketer’s clients. It is the active audience that will potentially
respond to advertising and marketing.The totals given above refer
to those who have ever listened to a podcast. By incorporating
data on those who listen to podcasts at least once a week,
eMarketer showed that the active podcast audience was
significantly smaller than implied by the earlier estimates.
C. Guesswork and SubjectivityResearchers attempting to measure any aspect of Internet
activity or e-business will inevitably have to supplement their
raw data with qualitative judgments.These involve varying levels
of interpretation, extrapolation, explanation or prediction.This
process inherently entails a degree of subjectivity, introducing
the possibility of bias and even potential conflicts of interest.
The effect of subjective factors is amplified when researchers are
trying to predict the future.The further out the forecast horizon,
the more assumptions that have to be made about interrelated
future events. Predicting the number of wireless Internet users
next year is hard enough; trying to pick a number for 2010 can be
akin to a shot in the dark. Such projections are typically based on a
set of related assumptions about future events. However, without
a clear explanation of those assumptions and their relevance to
the forecast, their use to business decision-makers is still limited.
US Podcast Audience, 2006, 2008 & 2010 (millions)
Total podcast audience*
10.0
25.0
50.0
Active podcast audience**
3.0
7.5
15.0
2006 2008 2010
Note: *individuals who have ever downloaded a podcast; **individualswho download an average of one or more podcast(s) per weekSource: eMarketer, February 2006
070756 ©2006 eMarketer, Inc. www.eMarketer.com
US Podcast Audience, 2004-2010 (millions)
2004 0.8
2005 4.5
2006 11.4
2007 21.7
2008 32.9
2009 44.1
2010 56.8
Note: CAGR=101%Source: The Diffusion Group, May 2005
065839 ©2006 eMarketer, Inc. www.eMarketer.com
4 www.emarketer.com
How eMarketer Makes Its Own Projections
eMarketer employs a four-step forecasting
process.This takes into account all of the avail-
able research data on a given market or trend.
Aggregate > Filter > Organize > AnalyzeThe process consists of aggregating data from as many sources
as possible, then normalizing, weighting and evaluating the data
and, finally, creating an estimate or projection that has the best fit
with all of the information available.
Step 1:AggregateeMarketer aggregates data from over 2,000 sources, including
research firms, consultancies, government agencies, non-profit
organizations and investment banking firms.This breadth and
depth of data yields a richer, far more complete picture of the
market than any single source can provide.
The Importance of Looking at All the Available Data:
■ There is no such thing as a perfect research study or survey
■ No single source can provide all the answers
■ A careful evaluation and weighting of multiple sources always
leads to a more accurate picture of reality
■ Critical business decisions should be based on complete,
up-to-date and reliable information
Step 2: NormalizeMerely lumping together raw numbers from various research
firms would inevitably result in apples-to-oranges comparisons.
eMarketer seeks to “normalize” the data by first understanding
and then adjusting for the different definitions used by the
research firms.
For example, when estimating online consumer retail spending,
it is important to note that some research firms include travel,
while others, including the US Department of Commerce, do not.
Similarly, while most researchers include paid search in their
estimates for online ad spending, at least two widely-quoted
research firms do not.This makes a significant difference because
paid search accounts for about 40% of spending on online
advertising and it is the fastest growing segment within it.
Step 3:Weight (the Data)eMarketer analysts carefully weight the methodologies and
results of different research firms based on a number of factors,
including (but not limited to) the following:
■ General reputation and credibility of each research organization
■ Historical accuracy of research sources, based on a comparison
between projected and actual results (utilizing eMarketer’s
eStat Database)
■ The sophistication and sample size of the study or survey,
as well as the overall research design and respondent
recruitment technique
In the weighting process, eMarketer will also look at the degree of
convergence between different estimates. In some cases numbers
from different sources line up very closely.This means the reader
can have a higher degree of confidence in the data. For example,
leading research firms consistently estimate that the online travel
market in the US was approximately $60 billion in 2005.
Comparative Estimates: US Online Travel Sales,2003-2010 (in billions)
comScore, January 2005 (1)
Forrester Research, May 2005 (2)
JupiterResearch, November 2005
Merrill Lynch, August 2005 (3)
PhoCusWright, October 2005 (4)
2003
$40.4
-
-
-
$40.3
2004
$50.9
$52.4
$57.0
-
$52.1
2005
-
$62.8
$68.0
$60.2
$65.4
2006
-
-
-
-
$78.8
2007
-
-
-
-
$93.6
2010
-
$119.0
$104.0
-
-
Note: (1) consumer unmanaged online travel purchases, the majority ofwhich are leisure; (2) leisure travel only; (3) this figure combines salesestimates for online travel agencies and direct suppliers; (4)leisure/unmanaged business travelSource: various, as noted, 2004 & 2005
067838 ©2005 eMarketer, Inc. www.eMarketer.com
THE FIRST PLACE TO LOOK 5
Step 4: Evaluate (the Bigger Picture)Focusing only on the estimates from other researchers, however,
does not necessarily take into account the latest trends or
developments in the marketplace. Before creating an estimate
or projection, eMarketer also looks at the broader economic,
technological, market and cultural trends that affect the area
being measured.
In many cases, eMarketer will also look at individual company
data. For instance, when measuring broadband households,
eMarketer tabulates subscriber figures from the leading
broadband providers in each market to create a bottom-up
“reality check” for total number of broadband households.
Similarly, when estimating the online paid search market,
eMarketer analysts examine the revenue figures from leading
players such as Yahoo, Google, MSN and AOL.
Step 5: Estimate All of the above is factored into our forecasts. In many cases it is
incorporated into an integrated statistical model, along with
data on other related trends. Our analysts then create estimates
and projections that represent the best fit with all the available
information on the market being measured.
Importantly, this process is a continuous and self-improving
feedback loop.With each new data point that becomes available,
eMarketer analysts compare the new information with the
existing body of evidence and reassess their statistics.
As a result of this five-step process, eMarketer’s estimates and
projections are frequently among the most accurate available.
Projections, Benchmarking & Accountability
There is not much point in making predictions
about the future unless you have a means of
measuring actual results.Without a standard of
measurement, any researcher could claim that
their projections are the most accurate.
eMarketer recognizes that certain government departments,
non-profit organizations and other respected, impartial sources
are able to provide reliable numbers that can be consistently
tracked over time. Most of these established sources, however,
only measure past results; typically they do not predict the future.
Wherever possible, eMarketer identifies a reputable third-party
source to serve as a “benchmark” for future projections.
For example, eMarketer has selected the International
Telecommunication Union (ITU) as the benchmark source for its
Internet User forecasts.The ITU has been measuring the Internet
sector in nearly every country in the world since 2000, and is
regarded as a trusted impartial source for international telecom-
munications statistics.The ITU does not make predictions, but
measures past levels of Internet penetration using information
gathered directly from national statistical agencies and local
research firms.
The use of internationally-accepted and independent benchmark
sources for historical data means our estimates and projections
can be measured against an accepted standard.As a result we
are more accountable for all of our data.
A Few of the Benchmark Sources eMarketer Uses for its Projections:
Internet users ITU (worldwide), Department of
Commerce (US)
Online advertising Interactive Advertising Bureau
spending (IAB) & PricewaterhouseCoopers
US total media spending Universal McCann
US B2C e-commerce sales US Department of Commerce
Mobile phone subscribers CTIA
6 www.emarketer.com
Building Data Models
These principles and techniques are used to
construct the data models which yield
eMarketer’s estimates and projections.
Example: How Many Online Shoppers & Buyers in 2005, 2010?eMarketer has built a demand-side model for consumer online
buying in several countries, including the US and Canada. Each
country model relates total Internet users to those shopping
and buying online (as well as to average spending and total dollar
purchases).The models are based on a continuous supply of new
data compiled from dozens of research and government sources.
Each research source provides varying degrees of information on
the following interrelated metrics:
■ Country population (for US, data comes from the most recent US
Census figures)
■ Number online in the country
■ Number of online “shoppers”
■ Number of online “buyers”
■ Percentage of Internet users who shop online
■ Percentage of Internet users who buy online
■ Percentage of the entire population who shop and buy online
■ Average annual online spending per buyer
eMarketer analyzes this information, accounting for differences in
definitions and methodologies, to construct models for Internet
purchase behavior. Here is a sample grid of eMarketer’s online
buying model for the United States, covering the years 2003–2008:
US Consumer Online Buying and Shopping Grid,2003-2008 (in millions and % penetration)
Internet users*
Online shoppers
Online buyers
Retail e-commerce
Total retaile-commerce
US population ages14+ (US Census)
Total US Internetusers
Internet users ages14+
Online penetrationamong populationages 14+
Online shoppersages 14+
as a % of Internetusers ages 14+
Online buyers ages14+
as a % of Internetusers ages 14+
Retail e-commerce(excluding travel)
Online travelrevenues
Average annualretail purchase peronline buyer (not inmillions; excludingonline travel)
2003
233.8
164.3
142.3
60.9%
104.2
73.2%
83.0
58.3%
$55,731
$38,012
$93,743
$672
2004
236.5
170.1
147.3
62.3%
109.4
74.3%
89.9
61.0%
$69,238
$48,503
$117,741
$771
2005
239.3
175.4
151.7
63.4%
114.2
75.3%
96.2
63.4%
$86,603
$58,010
$144,613
$900
2006
241.9
180.1
155.2
64.2%
118.1
76.1%
101.7
65.5%
$105,158
$68,510
$173,668
$1,034
2007
244.4
184.6
158.4
64.8%
121.8
76.9%
106.8
67.4%
$126,400
$80,225
$206,625
$1,184
2008
246.8
188.5
161.2
65.3%
124.9
77.5%
111.4
69.1%
$150,417
$93,141
$243,558
$1,350
Note: *eMarketer's 2003 baseline is from the Department of Commerce'sOctober 2003 estimate of Internet users who had access to the Internet atthe time of the survey; **eMarketer benchmarks its retail e-commercerevenues figures against US Department of Commerce data, for which thelast period measured was Q3 2005; the travel component was formulatedbased on aggregated dataSource: eMarketer, December 2005
069272 ©2005 eMarketer, Inc. www.eMarketer.com
The Benefits of eMarketer’sAggregation Approach toMarket Research
As a result of eMarketer’s unique approach,
the information we provide to our clients is:
■ more comprehensive, objective and ultimately accurate
than any single research source can provide;
■ available all in one place, making it easy to locate, evaluate
and compare;
■ presented along with clear analysis, which makes sense
of all the disparate figures and interrelated trends, and,
therefore
■ a uniquely-powerful decision-support tool.
THE FIRST PLACE TO LOOK 7
®
©2006 eMarketer Inc.All rights reserved.
is The First Place to Look for market
research information related to the Internet,e-business,
online marketing and emerging technologies.
eMarketer aggregates and analyzes e-business research from
over 2,000 sources, and brings it together in analyst reports,
daily research articles and the eStat Database – the most
comprehensive database of e-business and online marketing
statistics in the world.
eMarketer’s Core Expertise eMarketer specializes in researching and sorting vast amounts
of publicly-available information, and objectively compiling and
analyzing this information into widely read reports, articles and
newsletters. Our information products help business executives
worldwide make smarter, faster decisions about online marketing,
emerging technologies and e-business.
Dedicated Team Our team of researchers and analysts comb through Web sites,
data repositories and government statistics, uniquely providing
a 360-degree overview of available data, combined with original
analysis that is quickly accessible, comprehensive, objective,
actionable, cost-effective and, most of all, intelligent.
A Trusted Resource eMarketer serves as a trusted, third-party resource, cutting
through the clutter and hype – helping businesses make sense
of the e-business numbers and trends. eMarketer's products and
services help companies make better, more informed business
decisions by:
■ Streamlining e-business research sources and reducing costs
■ Eliminating critical data gaps
■ Providing an objective, bird’s eye view of the entire e-business
landscape
■ Better deploying and sharing information across the company
■ Building solid business cases backed up by hard data
■ Reducing business risk
■ Saving valuable time
For more information about subscribing toeMarketer, contact:
eMarketer, Inc. Toll-Free: 800-405-0844
75 Broad Street Outside the US: 212-763-6010
32nd floor Fax: 212-763-6020
New York, NY 10004 [email protected]
About eMarketer