Anew Model

7/29/2019 Anew Model

1/65

In partnership with:

A new Customer Experience MeasurementModel A Meta Analytical Review of Findingsover the period 2002 to 2009

Presented by Prof Adr Schreuder

MD of Consulta Research & Extra-ordinary Professor

of Marketing Research University of Pretoria,South Africa

19th Annual Frontiers in Service Conference 201010-13 June 2010 - Karlstad, Sweden


2/65

Index

Background & Rationale for Research Previous Research and Literature Review

Research Question & Objectives

Research Methodology & Data Analysis Research Results & Discussion

Dangers of Reporting Net Measures in Isolation

Satisfaction Measures as Predictors of NPS

Normality of Customer Experience Modelled Score Conclusion

Slide 2
http://www.consulta.co.za/


3/65

Background & Rationale for Research

Slide 3


4/65

Terminology Confusion

Slide 4

Source: Created by Adr Schreuder reference:

< http://www.wordle.net/show/wrdl/1954142/Customer_experience >
http://www.wordle.net/show/wrdl/1954142/Customer_experiencehttp://www.wordle.net/show/wrdl/1954142/Customer_experiencehttp://www.consulta.co.za/


5/65

CUSTOMER SATISFACTION

A Historic Overview

Slide 5

TQM of EdwardsDeming - ZeroDefect, Six Sigma

Relationship

Quality Era

(1995)

CRM

Customer

Experience Era

(2003)

CEM

Service Quality

Era (1984)

SERVQUAL

Product Quality

Era (1950s)

TQM

The Nordic approach (Grnroos 1984: Technical/FunctionalModel, Lethinen & Lethinen 1988 : Technical, Corporate,Interactive)

The North American Debate (PZB 1985: SERVQUAL (Gap-basedmeasure, Familiar five quality dimensions, Cronin & Taylor 1992:SERVPERF - Performance only measure, Brown Churchill & Peter

1993: Better/worse than expected scale, Teas 1993: EvaluatedPerformance Model = gap between perceived performance &ideal amount of feature)

Jagdish Sheth introduced Relationship Management in mid 90s

Growth of CRM-systems and popularity

NPS introduced by Reichheld in 2003 CEM era is born
http://www.consulta.co.za/http://www.consulta.co.za/


6/65

Customer satisfaction:

Contrasting academic and consumers interpretations

Satisfaction defined Derived from Latin satis = enough & facere (faction) = to do/to

make

Early interpretation and use of the word mostly focused on some sort

of release from wrong doing - later release from uncertainty

At least two basic approached in defining the concept: CS viewed as an outcome of a consumption activity

CS viewed as a process

Most widely adopted description = evaluation between what wasreceived and what was expected

Slide 6

Source: Parker, C & Mathews, B.P. Marketing Intelligence & Planning, 19/1 2001 (pp 38-44)


7/65


CS viewed as an outcome - Focus on the nature (not cause) of

satisfaction: Emotion - satisfaction is the surprise element of product

acquisition and/or consumption experiences, or an affectiveresponse to a specific consumption experience

Fulfilment - motivation theories state that either people are

driven by the desire to satisfy their needs or achieving specificgoals.

State - Olivers (1989) framework of four satisfaction states,where satisfaction is related to reinforcement and arousal. Low arousal = satisfaction-as-contentment

High arousal = satisfaction as surprise (positive / delight ornegative / shock)

Positive reinforcement = satisfaction-as-pleasure

Negative reinforcement = satisfaction-as-relief

Slide 7



8/65


CS viewed as a process Concentrate on the antecedents to satisfaction rather than

satisfaction itself. (Origins in discrepancy theory - (Porter, 1961)and Contrast Theory (Cardozo, 1965);

Most common interpretation = a feeling which results from aprocess of evaluating what was received against that expected, thepurchase decision itself and/or the fulfillment of needs/wants.

Most well-known descendent of the discrepancy theories is the

expectation disconfirmation paradigm (Oliver, 1977, 1981).

Slide 8



9/65

Customer Experience the new CustomerSatisfaction?

Yet despite the recognition of the importance of customerexperience by practitioners, the academic marketing literature

investigating this topic has been limited.

Publications on customer experience are mainly found in

practitioner-oriented journals or management books tend to

focus more on managerial actions and outcomes The literature in marketing, retailing and service management

historically has NOT considered customer experience as aseparate construct. Instead researchers have focused onmeasuring customer satisfaction and service quality.

Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A.Schlesinger (2009), Customer Experience Creation: Determinants, Dynamics and Management Strategies, Journal of

Retailing, 85 (1), 3141.

Slide 9


10/65

Customer Experience the new CustomerSatisfaction?

One reason for the apparently weak observed link betweensatisfaction and future behaviour may lie in the role ofemotions

Previously studies emphasised cognitive aspects of satisfaction growing body of evidence that affective measures of satisfaction(which incorporate emotions) may be a better predictor of

behaviour As a cognitive measure, satisfaction is more likely to be distorted

over time than a measure that incorporates an affectivecomponent (emotions are more deep-seated & more stable overtime)

Satisfaction should thus include a combination of an evaluative(cognitive) and emotion-based (affective) response to a serviceencounter

Source: Koenig-Lewis, N. and Palmer, A. "Experiential values over time a comparison of measures of satisfactionand emotion," Journal of Marketing Management (24:1-2), 2008, pp. 69-85.

Slide 10


11/65

Construct definition of Customer Experience

The customer experience construct is holistic in nature andinvolves the customerscognitive, affective, emotional, social and

physical responses to the retailer.

This experience is created by: controllable elements - service interface, retail atmosphere,

assortment, price,

uncontrollable elements - influence of others, purpose of shopping

Customer experience encompasses the total experience, includingthe search, purchase, consumption, and after-sale phases of theexperience, and may involve multiple retail channels.

Three major focus areas: cognitive evaluations (i.e., functional values)

affective (emotional) responses

social and physical components

Slide 11

Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A.Schlesinger (2009), Customer Experience Creation: Determinants, Dynamics and Management Strategies,Journal of

Retailing, 85 (1), 3141.


12/65

Putting Customer Experience into Perspective

The term Customer Experience Management

is used within the broader context ofCustomer Relationship Management (CRM) clearly seen in the view of Kirkby, Wecksell& Janowski (2003) when they say: CEM is

part of customer relationship management

(CRM) and the natural extension of buildingbrand awareness.

Where brand gives the promise, CEM is the

physical delivery of that promise and is vital

in an economy where a brand is increasinglybuilt on value delivered rather than product

features.

Slide 12

Illustration Copyright Consulta 2010


13/65

Putting Customer Experience in Perspective

Slide 13


14/65

Index



Research Methodology & Data Analysis Research Results & Discussion Dangers of Reporting Net Measures in Isolation



Slide 14


15/65

Previous Research & Literature Review

Collection of previous research and literature regardingCustomer Experience measurement are presented anddiscussed under the following topics: Multi-attribute measures such as:

SERVQUAL,

ASCI &

Others Effort Score & ERIC

Net Measures such as: The Net Promoter Score from Fred Reichheld & Bain Company

Secure Customer Index from Burke

Slide 15


16/65

Customer satisfaction and companyprofitability: The Service-Profit Chain

Slide 16

External

ServiceValue

Profitability

Internal

ServiceQuality

Employee

Satisfaction

EmployeeRetention

EmployeeProductivity

Customer

Satisfaction

RevenueGrowth

Customer

Loyalty

3Rs (>Market Share) Retention,

Repeat Business Referrals

Service designed & delivered to meet targeted customers

needs

Service Concept:Results for Customer

Workplace Design

Job Design

Employee Selection & Development (skills& empowerment drives good feelings

towards the firm)

Employee Rewards & Recognition

Tools for Serving Customers

Operating Strategy &

Service Delivery System

Adapted from: Heskett, Jones, Loveman, Sasser & Schlesinger (HBR 1994, HBR July/Aug 2008, p.120)


17/65

The GAP never mentioned

Slide 17

CEM

TheMissingGap

Expectations

Perceptions

Delivery

Interface

Managementunderstanding of

expectations

Marketing &

Communication

ExperienceStandards

Gap 1

Gap 2

Gap 3

Gap 4

Gap 5

CEM = delivering what ourcustomers expect us to and

a little bit more ,

making them feel great atevery moment of truth,

Adapted from original Gaps-

Model of Parasuraman,

Zeithaml & BerryIllustration Copyright Consulta 2010


18/65

CEMS

tra

tegy

Conceptual Model of CustomerExperience Creation

Slide 18

Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A. Schlesinger (2009),Customer Experience Creation: Determinants, Dynamics and Management Strategies, Journal of Retailing, 85 (1), 3141.

Social Environment:Reference group, tribes, co-destruction, service staff

Service Interface:Service person, technology, co-creation/customisation

Retail Atmosphere:Design, scents, temperature, ambient noise, music

Assortment:

Variety, uniqueness, quality

Price:Loyalty programs, promotions, rewards

Customer experiences in alternativechannels

Retail Brand

CUSTOMER EXPERIENCE (t 1)

Situational

Moderators:Type of store, location,

culture, economic climate,season, competition

ConsumerModerators:

Goals: experientialTask orientation, socio-

demographics, consumerattitudes (price sensitivity,

involvement)

Customer Experience(t):

Cognitive, affective, social,physical


19/65

Effort Score worth the effort?

Slide 19

Pred

ictive

Pow

er*of

Repurchase

High

Low

Low HighPredictive Power* for Increased

Spend

Power* - Linear regression coefficients regressed against Likelihood

to Repurchase & Increase Spend

Research conducted by Customer Contact Council of theCorporate Executive Board

NPSCouncil ConclusionInadequate measure in theservice channel:

Question inherently positive(only likelihood to recommend not criticize)

Captures company-levelsentiment (incl brand, product,pricing)

EffortCouncil Conclusion

Better suited for servicechannel. Better financialpredictor & best indicator ofloyalty

CSAT Council ConclusionPopular, widely used BUT notsufficient in predicting

financial outcomes de-emphasize its use in strategicdecisions

Comments: Directly contrasting scientific

proof ofACSI (American), SCSI(Sweden)

No scientific foundation Irresponsible to recommend

members against Effort-score purely developed in

Contact centre environment No published proof of scientific

reliability & validity

Scale is reverse scored SouthAfrican research shows low

reliability & poor predictiveproperties to the contrary


20/65

ERIC Empathy Rating Index

Slide 20

Source: Lywood, J., Stone, M. and Ekinci, Y. "Customer experience and profitability: An application of the

empathy rating index (ERIC) in UK call centres," Journal of Database Marketing & Customer Strategy

Management (16), 2009, pp. 207-214. & Lywood, J., Stone, M. and Hackett, D. Eric Methodology Whitepaper

2005 < http://www.empathy.co.uk/ >

The ERIC instrument consists of 29 empathy questions measured

on a 10-point rating scale and 11 call process questions that arerelated to how the calls are processed

The trained researchers (mystery callers) then make 40

unscripted(?) calls over three weeks to each company and

complete an online questionnaire

The study sample was limited to 28 companies in which ROCE andERIC ratings were both available.
http://www.empathy.co.uk/http://www.empathy.co.uk/http://www.consulta.co.za/http://www.consulta.co.za/


21/65

ERIC Testing the claims

Slide 21

Comments: No proven scientific grounding

Non rated Journal, 6 rated referencesused Questionable statistics & sample No longitudinal data or reference to

time Methodology basically mystery caller Psychometric properties of scale no

scientific grounding Mixed construct in scale (15 constructs

across 33 statements Of 5 attributes only one (Empathy) is

an interval scale, all other Yes/no or

numerical (number of calls) Claimed at 2008 CS Conference = False

claim

Source: Lywood, J., Stone, M. and Ekinci, Y. "Customer experience and profitability: An application of theempathy rating index (ERIC) in UK call centres,"Journal of Database Marketing & Customer StrategyManagement (16), 2009, pp. 207-214. & Lywood, J., Stone, M. and Hackett, D. Eric Methodology Whitepaper2005 < http://www.empathy.co.uk/ >

Claimed at 2008 CS Conference:At Last a proven link between a service

related measure and profitability
http://www.empathy.co.uk/http://www.empathy.co.uk/http://www.consulta.co.za/http://www.consulta.co.za/


22/65

Net Promoter Score single net measure

A simple recommend question measured on 0 to 10 scale oflikelihood to recommend

How likely is it that you would recommend (brand or company X)

to a friend or colleague?

Net Promoter score is calculated by taking the percentage ofpromoters (9-10 rating; extremely likely) and the percentage ofdetractors (0-6 rating; extremely unlikely)

NPS = % of Promoters minus % of Detractors

Companies with scores above 75% have world-class loyalty andword-of-mouth, which will correlate with a firms growth1

Slide 22

1Reichheld, F. (2003). The One Number You Need to Grow. Harvard Business Review, Dec 2003


23/65

Net Promoter Score single net measure

Slide 23

NPS adopted by executives: Swift to survey Simple to understand and

communicate Top-of-house dashboard metric

Reichheld (2003): NPS is a moreaccurate predictor of sales growththan the elaborate AmericanConsumer Satisfaction Index1

General Electrics CEO: This is thebest customer satisfaction metric

Ive seen

Positive Negative

1Reichheld, F. (2003). The One Number You Need to Grow. Harvard Business Review, Dec 20032Keiningham, T. et al. (2007). The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet,

Managing Service Quality 17(4), 361-384.3Morgan, N. & Rego, L. (2006). The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance. Marketing Science 25(5), Sep Oct.

Little scientific research linkingrecommend intentions to actualintentions2

Morgan and Rego (2006) assessedsix different metrics over a sevenyear period and found: recentprescriptions to focus customer

feedback systems & metrics solely

on customers recommendation

intentions and behaviours aremisguided3


24/65

Testing the Net Promoter Scoreclaims Contrary to Reichhelds assertions, the results indicate

that recommend intention alone will not suffice as asingle predictor of customers future loyaltybehaviour.

Use of a multiple indicator instead of a single

predictor model performs better in predictingcustomer recommendations and retention.

Thus far, however, there have been no peer-reviewed,scientific investigations examining the relationshipbetween recommend intention and customerbehaviours (outside of customer referral/complainingbehavior).

Slide 24

Source: Keiningham, T., Cooil, B., Aksoy, L., Andreassen, T. and Weiner, J. "The value of different customer satisfaction andloyalty metrics in predicting customer retention, recommendation, and share-of-wallet," Managing Service Quality (17:4),2007, pp. 361-384


25/65

Testing the Net Promoter Score claims

FINDING: The assertion that recommend intention alone willsuffice as a predictor of customers future loyaltybehavior

(Reichheld NPS), however, is not supported. We reach this

conclusion based upon three primary findings.

First, bivariate correlations of all the attitudinal variables

and customer behaviours investigatedtended to be modest. Second, when examining the three primary behaviours

associated with customer loyalty (retention, share of

wallet, and recommendations) recommend intention was

generallynot the best predictorfor each of these variables.

Third, multivariate models universallyoutperformedmodelsthat use only recommend intention

Slide 25

Source: Keiningham, T., Cooil, B., Aksoy, L., Andreassen, T. and Weiner, J. "The value of different customersatisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet," ManagingService Quality (17:4), 2007, pp. 361-384


26/65

Secure Customer Index as Net measure

The Secure Customer Index probes three attributes1: the secure customers were very satisfied,

had a likelihood to definitely continue using the service,

and had a likelihood of definitely recommending the service toothers

Customers grouped into subgroups or loyalty segments

Direct linkage to financial & market performance was

calculated

Slide 26

1Brandt, D. (1996). Customer Satisfaction Indexing, Conference Paper presented at American Marketing Association, USA

Secure Favourable Vulnerable At Risk


27/65

Secure Customer Index (SCI) as net measure

Today the new improved SCI is Burke Incorporateds proprietarymodelling approach

Five dimensions to assist validity and predictions of future share ofwallet:

Burke has studied data which directly links and also projects a

correlation between customer satisfaction, loyalty, and value tofinancial performance

Through projection and direct linkage, they can calculate whichpart of the marketing mix will bring the largest ROI

Slide 27

EarnedLoyalty

Likelihoodto

Recommend

Likelihoodto

Repurchase

OverallSatisfaction

PreferredCompany


28/65

Slide 28

Customer Experience A deep ecological paradigm shift(Fritjof Capra The Web of Life, 1996)


29/65

Slide 29

Key Drivers of Loyalty


30/65

Slide 30

Outcomes of Improved Customer Experience

Outcomes of Customer

Experience

Customer-RelatedOutcomes

Efficiency-RelatedOutcomes

Employee-RelatedOutcomes

Overall Performance-Related Outcomes

Behavioral

Intentions

CustomerBehaviours

CustomerCommitment

RepurchaseIntentions

Price Perceptions &Willingness to pay

Customer Loyalty &Repurchase Behaviour

Word-of-Mouth &Complaining Behaviour

FinancialPerformance

NonfinancialPerformance

Source: Luo, X & Homburg, C. April 2007 Neglected Outcomes ofCustomer Satisfaction. Journal of Marketing, Vol 71, Apr 2007 (0 133-149)

Behavioral Intentions are determined byhow the drivers of Customer Satisfactionare managed

this is the essence of CustomerExperience Management

CustomerDefection


31/65

Index



Research Methodology & Data Analysis Research Results & Discussion Dangers of Reporting Net Measures in Isolation



Slide 31


32/65

Research Question

The popularity of the Net Promoter Score has

highlighted the use of net measures in customerexperience measurement

Considering the preceding literature review anddiscussion regarding different net measures, it is

obvious that no single measure can be usedsuccessfully in measuring the complex constructsof customer experience, customer satisfactionand customer loyalty

This presentation will explore a quantitativemodel that integrates the best-of-both-worldsthrough a combined metrics of net measures anda multi-attribute measure of customerexperience

Slide 32


33/65

Research Objectives

Slide 33

Explore the use and application of Net Measures in themeasurement of Customer Experience

Compare Net measures in terms of reliability, validity,predictive ability and practical application

Position Net Measures within the body of knowledge ofmulti-attribute Customer Experience Measurementtheory and practise

The purpose of this study is toinvestigate the following threeobjectives:


34/65

Research Design & Data Collection

Meta-analysis on data collected over a time frame of more than 5

years, covering more than 1.5 million customer interviews acrossSouth Africa

Survey results have been consolidated from enterprise wideproprietary customer satisfaction surveys across a range of clients

For the purpose of this presentation (and reliability) the data is

limited to results from surveys in the financial services industry inSouthern Africa

Respondent selection for each of the surveys under considerationwas quota-based from client contact lists on proportional stratifiedsample designs

At the time of the interview, the respondent was a currentcustomer of the financial service provider being evaluated, andfilter-controlled for having a recent interaction at a specificchannel (enterprise-wide metrics across channels across segments)

Slide 34


35/65

Research Design & Data Collection

Survey data was collected via telephonic, web-based and face-to-face interviews

To ensure minimal non-sampling error, all interviews were subjectto strict quality assurance processes, and advanced technology wasused to capture data

No ethical issues are relevant to the study since most of thefindings will be reported at meta-data levels without identifyingany specific sponsoring company (to protect confidentiality andproprietary measures)

A strict ESOMAR code-of-conduct was followed in all data

collection. The respondents were made aware of the institutionssponsoring the survey and for what purposes the information wouldbe used

Slide 35


36/65

Index



Research Methodology & Data Analysis

Research Results & Discussion Dangers of Reporting Net Measures in Isolation



Slide 36


37/65

Research Methodology & Instruments

Prof Adr Schreuder developed a conceptual cause-and-effect model illustrated as an integratedcustomer experience measurement

Developed through years of academic researchcombined with extensive experience regarding

Customer Satisfaction measurement across multipleindustries

Basis for measurement is a structural model ofcustomer satisfaction that incorporates theimportant constructs of satisfaction that willidentify underlying service or product deficiencies(or strengths) and a proprietary algorithm forintegrating net measures into this multi-attributemodel

Slide 37


38/65

Slide 38

The CONSULTA Integrated Customer ExperienceMeasurement Model

FAILUREFAILURE DELIGHTDELIGHT

FAILUREFAILURE DELIGHTDELIGHT

FAILUREF AI LU RE D EL IG HTDELIGHT


39/65

Slide 39

The Conceptual Model Flow

Copyright Consulta Research - 2010


40/65

Slide 40

Principle Calculation of Modeled Scores

FAILUREFAILUREDELIGHTDELIGHT


41/65

Slide 41

Instrument Development Process


42/65

Slide 42

Model Development Process


43/65

Use an Enterprise-wide Model A Retail Bankingexample

Slide 43


44/65

Present CE Metrics in Dashboards

Slide 44Slide 44


45/65


Slide 45

For this reason the customer experience index score is notreported in isolation as a single number, but merely as the net

result of multiple items, each of which contains detail results and

offers valuable strategic information into the management ofcustomer delight, loyalty, propensity to shift, service recovery,

corrective improvement measures and consequence management

It is important to be able to delve deeper into the results toenable the receiver to delve deeper than satisfaction

The integrated customer experience measurement, although

resulting in a final index score, acknowledges the fact that a singlevalue for an index might hide more that it reveals


46/65


Research Instruments:

Same basic layout including sections corresponding to thecomponents contained in the conceptual model for customersatisfaction measurement

First section measures specific channels value proposition with

a range of custom designed service attributes - incorporates

both customer perception and customer expectation by usingconfirmation-disconfirmation scale

Specific questions on product quality, service quality,relationship quality & pricing as contributingfactors/components of customer satisfaction

Slide 46

0 1 2 4 53 9 10876

Much worse than expected Much better than expected


47/65

Meta-data and Analysis

For each of the surveys the statistical analysis (using the statistical

software package STATISTICA) included: reliability and factor analysis;

structural equation modelling;

multiple regression analysis

The result, for each of the surveys, was a unique structural (cause-

and-effect) model of customer satisfaction that considers all theimportant drivers of satisfaction

Final data set used for meta-analysis contained each of thecomponents defined on next slide

Included 704 separate customer satisfaction studies forming part

of the enterprise wide measurement of customer experience, foreach of the financial institutions - each with a sample of at least100 respondents and more

Slide 47


48/65


Slide 48

Metric Description

Weighted serviceattribute average score A weighted average of the (unique channel) serviceattributes measured in terms of customer expectation

Service problems % Proportion of respondents who indicated that theyexperienced a service problem within a certain time period.This is different from the proportion of respondents

complaining (formally or informally) as measured in ACSI

Problem recovery % Proportion of respondents who indicated that their serviceproblem was recovered to their satisfaction

Overall delight % Proportion of respondents who gave a 9 or 10 rating out of

10 for overall satisfaction. This is much more strict than thetypical Top 2 Box metric calculated on a 5 point verbal scale

or the equivalent top four boxes on the ten-point ACSI

scale


49/65


Slide 49

Metric Description

Overall failure % Proportion of respondents who gave a 0 or 1 rating out of10 for overall satisfaction

Average score (overallsatisfaction)

A simple average of overall satisfaction rated on a scalefrom 0 to 10

Customer satisfaction

index score

Index score (out of 100) is a function of the following key

elements: Underlying structural model Basic calculation principle of being rewarded for

positive ratings and being penalised for negativeratings corresponding to the concept of a net measure

Net Promoter Score Calculated according to the original definition of Reichheld(2003) the Net Promoter Score equals the % of promotersminus the % of detractors


50/65

Index




Research Results & Discussion Dangers of Reporting Net Measures in Isolation



Slide 50

h


51/65

The Dangers of Reporting Net Measures inIsolation

Danger/weakness inreporting any netmeasure (in isolation):two measurementshaving exactly thesame value for the netmeasure can in facthave a range ofdifferent valuesassigned to thecomponents of the net

measure

Slide 51

h D f R i i


52/65

The Dangers of Reporting Net Measures inIsolation

Recommendation not onlyapplicable to netmeasures, but to othersimple statistical

measures (e.g. the samplemean) as well

A variety of differentrespondent values canalso yield the same resultfor the specific statisticalmeasure and typicaldistribution detailand/or graphs providemore insight into theresults

Slide 52


53/65

Satisfaction Measures as Predictors of the NPS

As is to be expected, service problems and failure ratings show anegative correlation with customer satisfaction and NPS, whiledelight ratings show a positive correlation. Service problemrecovery shows a very low, but positive, correlation with the NPS NOTE poor R2

Slide 53 Sample Base: 1.5million respondents


54/65


Slide 54


55/65


Individually, as independent variables in modelling the Customer

Loyalty, the graphs and correlation coefficients clearly show thatthe integrated index score with an R2 of 0.73 seems to be the bestpredictor of the Net Promoter Score

Slide 55 Sample Base: 1.5million respondents


56/65

Satisfaction Measures as Predictor

However, we do notrecommend either the NPS orcustomer satisfaction index score in isolation as thebest and sufficient measurement to evaluate businessperformance, but agree with Schneider et al. that

using a variety of measures rather than simply onemeasure would better capture the complexityunderlying customer satisfaction and customer

behaviours

Slide 56Schneider, D.; Berent, M.; Thomas, R. & Krosnick, J. (2008). Measuring Customer Satisfaction and Loyalty: Improving the Net-PromoterScore. Poster presented at the Annual Meeting of the American Association for Public Opinion Research, New Orleans, Louisiana


57/65

Integrated Satisfaction Measure as Predictor

The net measure(s) in itselfcan provide a top line measurement to trackperformanceor even be effectively used as a top-of-house executive

indicator

Analysing the detail of all the different metrics constituting the customersatisfaction index score and NPS will assist greatly in the need for rootcause analyses and strategic/tactical direction

The quantitative data analysis of these measures is further enriched byqualitative questions similar to the whys asked by GE, including

verbatim descriptions of service problems that were experienced,suggestions on improving service delivery, etc.

Slide 57

Normality of Customer Experience


58/65

Normality of Customer ExperienceModelled Score

Due to more complex nature of its calculation, efforts to examinestatistical properties of net measures using a mathematicalapproach can be tedious and difficult

Computer-intensive simulation methods such as the bootstrapprovide a solution

The bootstrap method was applied to replicate 1 000 bootstrapsamples for each of four different studies each bootstrap sampleconsisted of 380 respondents chosen randomly (with replacement)from the survey data

This provided 1 000 simulated index scores, which can be plottedas histograms and normal probability plots

The accuracy of the simulations increase as the number of bootstrap replications

increase; 500 or more simulations are sufficient to reduce variability and provide

accurate results

Slide 58


59/65

Normality of Customer Experience Modelled Score

Slide 59

Variable: VoC1, Distribution: Normal

Chi-Square test = 8.67399, df = 9 (adjusted) , p = 0.46790

50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69

Category (upper limi ts)

0

2

4

6

8

10

12

14

16

18

20

RelativeFrequency(%)



40 41 42 43 44 45 46 47 48 49 50 51 52 53 54

Category (upper limi ts)

0

5

10

15

20

25

RelativeFre

quency(%)

Normal Probabili ty Plot of Vo C1 (4 VoCs for normality g raphs 4v*1000c)

50 52 54 56 58 60 62 64 66 68

Observed Val ue

-5

-4

-3

-2

-1

0

1

2

3

4

ExpectedNormalValue

VoC1: SW-W = 0.998084051, p = 0.3196

Normal Probabili ty Plot of Vo C2 (4 VoCs for normality graphs 4v*1000c)

40 42 44 46 48 50 52 54

Observed Val ue

-4

-3

-2

-1

0

1

2

3

4

ExpectedNormalValue

VoC2: SW-W = 0.998708772 = 0.6945


60/65


Slide 60



28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

Category (upper limits)

0

5

10

15

20

25

RelativeFrequency(%)



32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

Category (upper limits)

0

5

10

15

20

25

RelativeFreq

uency(%)

. , .

Normal Probabili ty Plot of VoC3 (4 VoCs for normality graphs 4v*1000c)

28 30 32 34 36 38 40 42 44

Observed Val ue

-4

-3

-2

-1

0

1

2

3

4

ExpectedNormalValue

VoC3: SW-W = 0.998033823, p = 0.2971

Normal Probabili ty Plot of VoC4 (4 VoCs for normality graphs 4v*1000c)

32 34 36 38 40 42 44 46

Observed Val ue

-4

-3

-2

-1

0

1

2

3

4

ExpectedN

ormalValue

VoC4: SW-W = 0.998200047, p = 0.3767


61/65


For all four studies, both the chi-square test and Shapiro-Wilk testdid NOT reject normality of the customer satisfaction index score,which holds the benefit of statistical inference of the index score(e.g. calculating confidence intervals and performing hypothesistesting)

Although these results are based on only four studies, representinga small portion of the wide range of underlying models used todescribe the results of the various studies, we believe that withadditional research we will be able to establish similar results forthe whole range of studies under consideration, and consequentlyestablish normality for the customer satisfaction index score in

general

Slide 61


62/65

Index




Research Results & Discussion Reporting Net Measures in Isolation



Slide 62


63/65

Conclusion

Without denying the fact that net measures has a role to play, theuse of net measures as standalone questions has been shown tohave some disadvantages

Reporting net measures in context, supported by the multipleitems it contains, provides the opportunity to analyse the detail ofall the different metrics constituting the net measure

This assist in the need for root cause analyses andstrategic/tactical direction, while the net measure in itself canprovide a top line measurement to track performance or even beeffectively used as a top-of-house executive indicator

The quantitative data analysis of these measures can further beenriched by qualitative questions, including verbatim descriptionsof service problems that were experienced, suggestions onimproving service delivery, etc.

Slide 63


64/65

Conclusion

Using longitudinal meta-data analysis of more than 1.5million customer satisfaction measurement interviews,we have presented reliable correlations between theNet Promoter Score and an Integrated CustomerSatisfaction Index score, as well as establishingstatistical properties of these measures

The Customer Satisfaction Index score can be classifiedas a combined multi-attribute and net measureapproach, since it incorporates the net effect of

failure and delight ratings, as well as serviceproblems and the recovery thereof

Slide 64


65/65

Conclusion

Understanding that customers, as human beings, arecomplex by nature and accepting that the

measurement of customer satisfaction involves the

measurement of a complex construct, the use of anintegrated measure of multiple-item & net measureshas the advantage of providing insight into

underlying drivers of customer satisfaction, whilealso offering a simple top-of-house dashboard

metric that is simple to communicate.

Documents

Anew Model