Full screen and Product testing

The Full ScreenA step often seen as a necessary

evil, yet very powerful and with long-lasting effects.

Forces pre-technical evaluation, and summarizes what must be done.

Methods range from simple checklists to complex mathematical models.

Most professional firms undertake, last low-risk evaluation

Purposes of the Full Screen

To decide whether technical resources should be devoted to the project.◦ Feasibility of technical accomplishment -- can we do it?◦ Feasibility of commercial accomplishment -- do we want

to do it?To help manage the process.

◦ Recycle and rework concepts◦ Rank order good concepts◦ Track appraisals of failed concepts

To encourage cross-functional communicationAvoid potholes

10-3

Screening Alternatives

Judgment/Managerial OpinionConcept Test followed by Sales

Forecast(if only issue is whether consumers

will like it)

Scoring Models

10-4

A Simple Scoring Model

ValuesFactors: 4 Points 3 Points 2 Points 1 PointDegree of FunNumber of PeopleAffordabilityCapability

MuchOver 5EasilyVery

Some4 to 5ProbablyGood

Little2 to 3MaybeSome

NoneUnder 2NoLittle

Student's Scores: Skiing Boating HikingFun 4 3 4People 4 4 2Affordability 2 4 4Capability 1 4 3 Totals 11 15 13

Answer: Go boating.

Figure 10.2

10-5

Source of Scoring Factor ModelsFigure 10.3

10-6

A Scoring Model for Full Screen

Note: this model only shows a few sample screening factors.

Factor Score (1-5) Weight Weighted Score

Technical Accomplishment:

Technical task difficulty

Research skills required

Rate of technological change

Design superiority assurance

Manufacturing equipment...

Commercial Accomplishment:

Market volatility

Probable market share

Sales force requirements

Competition to be faced

Degree of unmet need...

Figure 10.4

10-7

The Scorers

Scoring Team: Major Functions (marketing, technical, operations, finance)

New Products Managers

Staff Specialists (IT, distribution, procurement, PR, HR)

Problems with Scorers: May be always optimistic/pessimistic

May be "moody" (alternately optimistic and pessimistic)

May always score neutral

May be less reliable or accurate

May be easily swayed by the group

May be erratic

10-8

Industrial Research Institute Scoring ModelTechnical success

factors:

Proprietary Position

Competencies/Skills

Technical Complexity

Access to and Effective Use of External Technology

Manufacturing Capability

Commercial success factors:

Customer/Market Need

Market/Brand Recognition

Channels to MarketCustomer StrengthRaw

Materials/Components Supply

Safety, Health and Environmental Risks

Source: John Davis, Alan Fusfield, Eric Scriven, and Gary Tritle, “Determining a Project’s Probability of Success,” Research-Technology Management, May-June 2001, pp. 51-57.

Figure 10.5

10-9

Alternatives to the Full Screen

Profile SheetEmpirical ModelExpert SystemsAnalytic Hierarchy Process

10-10

A Profile Sheet Figure 10.6

10-11

Criteria Based on the NewProd Studies

Must-Meet Criteria (rated yes/no):◦Strategic alignment◦Existence of market need◦Likelihood of technical feasibility◦Product advantage◦Environmental health and safety

policies◦Return versus risk◦Show stoppers (“killer” variables)

10-12

Criteria Based on the NewProd Studies(continued)

Should-Meet Criteria (rated on scales):◦Strategic (alignment and

importance)◦Product advantage (unique benefits,

meets customer needs, provides value for money)

◦Market attractiveness (size, growth rate)

◦Synergies (marketing, distribution, technical, manufacturing expertise)

◦Technical feasibility (complexity, uncertainty)

◦Risk vs. return (NPV, IRR, ROI, payback)

10-13

Products 1, 2, 3, and 4

Goal: Select Best NPD Project

Market Fit Tech. Fit Dollar Risk Uncertainty

Product Line

Channel

Logistics

Tim ing

Price

Sal es Force

Desi gn

Materials

Suppl y

Mfg. Tech.

Mfg. Tim ing

DifferentialAdvantage

Payof fs

Losses

Unmit igated

Mi tigated

Product Line

Analytic Hierarchy Process (AHP)Figure 10.9

10-14

15

PRODUCT TESTINGProvides a critical measures of a new

product's market potentialExtremely important in FMCG Cos.Product testing for four purposes

◦ Against competition: which of the alternatives offered is preferred relative to competition

◦ Product improvement: whether an improved formula could replace the current product

◦ Cost saving: whether a less expensive product could replace the current one

◦ Concept fit: whether the product variant resembles the selling message

16

Product testing procedures◦ Blind Vs Branded test - a key issue◦ Blind test - Reactions to "pure" product◦ No brand name as yet◦ Branded test - Difficult to conceal◦ Measurement of effects of brand etc.

Four basic principles◦ Representative of the product that will be in

the market ultimately◦ Name, packaging should be similar ◦ If different formulas are used, size, shape,

colour to be identical◦ Avoid labels that bias (e.g. sequence of

letters etc.)

17

Procedures for product testsMonadic - designs where a consumer

evaluates one product, having no other product for comparison

Comparison - Consumer rates 2 or more products

Sequential monadic - rates one product and then is given a second product (rated) independently then compared

Protomonadic - rates one product, is given a second product and compares both

Paired comparison - directly compares two products

18

Repeat-paired comparison - consumer is given two or more sets of products to compare against each other at two different points of time

Round robin - tests where a series of products is tested against each other

Triangle designs - is given 2 samples of one product and one sample of another to identify the one that differs

Duo-trio - a standard product is given and asked to determine which of the other (two) products are similar

Difference - asked to determine if one product is different from the other

19

In - home testing Vs Central location

(expensive) 1. Unrealistic2. Opinions of other

family members ignoredPeriodicity - usually a week,

depends on the product, purchase cycle

Sales wave extended product test consumers encouraged to buy at intervals coinciding with normal product cycle

20

Advantages: Identification of novelty product wear outs Identification of problems Market share prediction Potential segments

Monadic Vs Paired test Monadic is realistic. Typically a consumer uses

a product at a time and decides Monadic tests are difficult to interpret. (e.g.

80% say "excellent") Comparison tests concentrate on product

differences In certain situations involving sensory

evaluations, comparison tests are impractical.

21

Questions asked - Preference, overall rating attribute rating, likes-dislikes, uniqueness, usage pattern etc.

Sampling ◦ non-probability◦ 100-200 for in-home◦ CLT around 20◦ Cost is a factor

Action standards◦ preference that is statistically significant ◦ Where claims of superiority are made

should have significant preference. Conventions may vary with MR agencies

22

Questions asked - Preference, overall rating attribute rating, likes-dislikes, uniqueness, usage pattern etc.

Sampling ◦ non-probability◦ 100-200 for in-home◦ CLT around 20◦ Cost is a factor

Action standards◦ preference that is statistically significant ◦ Where claims of superiority are made

should have significant preference. Conventions may vary with MR agencies

Product testing in industrial marketsBuyers cannot decide on the

merits and demerits of a new product quickly

Only a few product testers, distinct from/in contrast to from potential buyers

Testers need to adapt products to suit their needs

Buyers have expertise in the product

Purposes of beta testTo check product functioning in situTo confirm selection of features, both core

and optionalTo test accuracy and usefulness of

support materialTo assess level of training requiredTo evaluate perceived strengths and

weaknesses compared to those of competitors

To promote sales with site chosenTo use site as a demo for product benefits

25

A few important aspectsA systems approach needed : Methods

and procedures of product testing should constitute a standardized system for like products

Normative databases need to be built over time for better interpretation

Same research company Real environment testsRelevant variables from consumers'

perspective (particularly while using qualitative methods)

Conservative action while dealing with established products