8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
1/33
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
2/33
2
Structure:
MUSA Method
Research objectives
Data mining approach
The experiment
Application of results
Conclusions
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
3/33
3
MUSA MethodThe main objective of the method is the aggregation of individual judgementsinto a collective value functionassuming that clients global satisfaction
depends on a set of ncriteria representing service characteristic dimensions.
Customers Global Satisfaction
Satisfactionaccording to the
1-st criterion
The MUSA method assesses global and partial satisfaction functions Y* and X*I
respectively, given customers judgements Y and Xi.
1b
XbY
n
1i
i
n
1i
*
ii
*
where the value functions Y* and X*Iare normalised in the
interval [0,100], and biis the weight of the i-th criterion
Satisfactionaccording to the2-nd criterion
Satisfactionaccording to the
n-th criterion
MUSA Method (1)
Grigoroudis and Siskos (2002)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
4/33
4
MUSA Method
MUSA Method (2)
CRITERIA GLOBALPREFERENCE
disaggregation
aggregation
AggregationModel
Aggregation
Model?
MUSA uses a preference disaggregation model. In the traditional aggregation
approach, the criteria aggregation model is known a priori, while the global
preference is unknown. On the contrary, the philosophy of disaggregation
involves the inference of preference models from given global preferences.
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
5/33
5
MUSA Method
MUSA Method (3)
Customer's global satisfaction
y1 y2 ym y
y*2
y*m
Y*
Yy*1
.
.
.
.
.
.
y*
... ...
Global Added Value Function
Satisfaction according to the 1st criterion
x1
1 x1
2 x1
k x11
x1*2
x1
*m
X1
*
X1
x1*i
x1
*1
.
.
.
.
.
.
... ...
Satisfaction Function for
the 1st Criterion
Satisfaction according to the 2nd criterion
xi1 x
i2 x
ik x
ii
xi*2
xi*m
Xi*
Xi
xi*i
xi*1
.
.
.
.
.
.
... ...
Satisfaction Function for
the 2nd Criterion
Satisfaction according to the n-th criterion
xn
1 xn
2 xn
k xnn
xn*2
xn
*m
Xn
*
Xn
xn*n
xn
*1
.
.
.
.
.
.
... ...
Satisfaction Function for
the n-th Criterion
...
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
6/33
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
7/33
7
Research ObjectivesLets say that we have two equal divided, farraginous
groups of customers in our sample. The first isconsisted of demanding customers and the second onehas non-demanding customers. MUSA will produce aresult describing neutral customers.
Research objectives (2)
Demanding Non-demanding Neutral
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
8/33
8
Research ObjectivesIn the case of different importance of the criteria (criteria
wights) given by farraginous groups of customers willlead us to similar problems.
Research objectives (3)
1st Group of Customers
15.0% 15.0%
35.0% 35.0%
0%
10%
20%
30%
40%
50%
Criterion 1 Criterion 2 Criterion 3 Criterion 4
2nd Group of Customers
35.0% 35.0%
15.0% 15.0%
0%
10%
20%
30%
40%
50%
Criterion 1 Criterion 2 Criterion 3 Criterion 4
MUSA results
25.0% 25.0% 25.0% 25.0%
0%
10%
20%
30%
40%
50%
Criterion 1 Criterion 2 Criterion 3 Criterion 4
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
9/33
9
Research ObjectivesMUSA gives as internal measures evaluating the quality of
its results. The reliability evaluation of the results ismainly related to the following quantitative measures:
the fitting level to the customer satisfaction data(Average Fitting IndexAFIand Overall PredictionLevel-OPL)
the stability of the near-optimality analysis results(Average Stability IndexASI).
Research objectives (4)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
10/33
10
Research ObjectivesThe Overall Prediction Level (OPL) is based onthe sum of the main diagonal cells of theprediction table, and it represents thepercentage of correctly classified customers:
: the number of customers that havedeclared to belong to global satisfaction levelm1, while the model classifies them to levelm2
: the percentage of customers ofactual global satisfaction level m1, that themodel classifies to level m
2
: the percentage of customers ofestimated global satisfaction level m1, thathave declared to belong to level m2
Research objectives (5)
1~y 2~y y~
1y
2y
y
Nij R
ij
Cij
N11
R11
C11
N12
R12
C12
...
jy~
N1j
R1j
C1j
...N
1 R
1
C1
N21
R21
C21
N22
R22
C22
...N
2j R
2j
C2j
...N
2 R
2
C2
.
.
.
Na1
Ra1
Ca1
Na2
Ra2
Ca2
...
Naj
Raj
Caj
...
Na
Ra
Ca
... ...N
i R
i
Ci
Ni1 R
i1
Ci1
Ni2 R
i2
Ci2
iy
.
.
.
.
.
.
.
.
.
Predicted global satisfaction level
Actualglobals
atisfactionlevel
1 2m m
N
1 2m mR
1 2m mC
1 1 1 2
1 1 21 1 1
m m m m
m m m
OPL N N
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
11/33
11
Data Mining ApproachSurveys data will be processed following a clustering
(unsupervised learning or segmentation) approach.
Data mining approach (1)
Data from
questionnaires
......
Preprocessed
Data
MUSAfor eachcluster
Clusters(2,, n)
DataSelection:
Valid Answers
DataPreprocessing:
Transformations
DataMining
Labeling: Basedon demographic
data
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
12/33
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
13/33
13
Transformations
Data mining approach (3)
For different Demand, DCr(i) nominal:
If Cr(i)(T+thr) Then DCr(i)=INT(Cr(i)-(T+thr))
If(T-thr)Cr(i)(T+thr) Then DCr(i)=0
where T is the declared total satisfaction of the customer, Cr(i)is thesatisfaction regarding his/her satisfaction on i criterion and thris athreshold.
For different Criteria Weights, W1Cr(i), W2Cr(i), numeric:
W1Cr(i)=ABS(Cr(i)-T)and
W2Cr(i)= W1Cr(i)*(crc(a)-(ABS([(a-1)/2-T]/(a-1)/2))
where T is the declared total satisfaction of the customer, Cr(i)is thesatisfaction regarding satisfaction on i criterion, crc(a)is a correction
parameter and a is the number of global satisfaction levels
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
14/33
14
EM algorithm
Data mining approach (4)
The EMalgorithm can be seen as a generalizedversion of K-means clustering
A hard membership is adopted in the K-meansalgorithm, (i.e., a data pattern is assigned to onecluster only).
This is not the case with the EMalgorithm, wherea soft membership is adopted, (i.e., themembership of each data pattern can bedistributed over multiple clusters)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
15/33
15
EM algorithm
Data mining approach (5)
Similarly to K-means, first select the cluster parameters (A, A,P(A)) or guess the classes of the instances, then iterate
Each cluster A is defined by a mean (A)and a standarddeviation (A)
Samples are taken from each cluster A with a specifiedprobability of sampling P(A)
Adjustment needed: we know cluster probabilities, not actualclusters for each instance. So, we use these probabilities asweights
For cluster A:
Stop when the difference between two successive iterationbecomes negligible (i.e. there is no improvement of clusteringquality).
We measure that by:
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
16/33
16
EM algorithm
Data mining approach (6)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
17/33
17
The experiment
The experiment (1)
For the development of the transformation procedureand for the evaluation of our research results wedesigned and we implemented an experiment.
Steps:
1. Generation of synthetic dataDataSet Generator
2. Evaluate clusters generationWEKA DM tool
3. Evaluate MUSA results on new segments
4. Select the most appropriate transformations
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
18/33
18
Generation of synthetic data
The experiment (2)
A dataset generator, developed by our team for MUSA software evaluation,was used for the production of different data sets. The generator is ableto produce data (answers to surveys) that have specific characteristics.
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
19/33
19
Generation of synthetic data
The experiment (3)
1stData Set Produce two segments regarding different customers demand:Criterion 1 Criterion 2 Criterion 3 Criterion 4
Weights 25% 25% 25% 25%Sets (500) Non Demanding Non Demanding Non Demanding Non Demanding (500) Demanding Demanding Demanding DemandingSatisfaction LevelsA (Global) 5a(i) (per criterion) 5 5 5 5
2nd
Data Set
Produce two segments regarding different criteria weights:Criterion 1 Criterion 2 Criterion 3 Criterion 4Demand Neutral Neutral Neutral NeutralSets (500) 15% 15% 35% 35% (500) 35% 35% 15% 15%Satisfaction LevelsA (Global) 5a(i) (per criterion) 5 5 5 5
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
20/33
20
Evaluate clusters generation
The experiment (4)
WEKA, a Java Data Mining Tool developed in University of Waikato, wasused for classes to clusters evaluation.
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
21/33
21
Evaluate clusters generation
The experiment (5)
Evaluation of 1stdata set using DCr(i) transformation:
Assigned to Cluster
Initial Classes 0 1
I 164 336
II 446 54
Cluster 0
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
22/33
22
Evaluate MUSA results
The experiment (6)
Evaluation of 1stdata set using DCr(i) transformation:
MUSA's Internal Quality Measures
Samples - Data Sets Customers AFI ASI OPL
Generator Data Set I 500 97.17% 96.59% 95.00%
Generator Data Set II 500 96.29% 95.68% 95.40%
Generator Data Set I + II 1000 91.29% 88.77% 56.50%
Cluster 1 --> I 390 95.21% 96.13% 86.15%
Cluster 0 --> II 610 94.75% 94.63% 92.95%
Criteria Weights Demanding Indices
Samples - Data Sets Cr 1 Cr 2 Cr 3 Cr 4 Global Cr 1 Cr 2 Cr 3 Cr 4Generator Data Set I 25.04% 25.92% 24.61% 24.44% -55.25% -60.43% -57.28% -16.13% -61.86%Generator Data Set II 26.28% 25.43% 24.16% 24.13% 41.82% 59.962 65.41% 30.55% 12.99%Generator Data Set I + II 25.34% 27.69% 23.61% 23.36% -27.96% -19.57% -27.40% -5.16% -54.35%Cluster 1 --> I 25.24%
26.23%
24.89%
23.64%
-52.34%
-57.61%
-57.92%
-20.09%
-65.90%
Cluster 0 --> II 25.54% 25.01% 24.86% 24.59% 40.26% 54.48% 62.63% 35.28% 19.14%
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
23/33
23
Evaluate MUSA results
The experiment (7)
Evaluation of 2nddata set using W1Cr(i), W2Cr(i) transformations:
MUSA's Internal Quality Measures
Samples - Data Sets Customers AFI ASI OPL
Generator Data Set I 500 96.45% 97.64% 89.60%
Generator Data Set II 500 96.51% 96.63% 91.00%
Generator Data Set I + II 1000 92.52% 91.97% 60.30%
Cluster 0 --> I 498 94.35% 94.85% 80.92%
Cluster 1 --> II 502 95.32% 94.65% 93.43%
Criteria Weights Demanding Indices
Samples - Data Sets Cr 1 Cr 2 Cr 3 Cr 4 Global Cr 1 Cr 2 Cr 3 Cr 4Generator Data Set I 16.45% 16.34% 33.67% 33.53% 28.92% 26.97% 51.05% 33.58% 36.29%Generator Data Set II 34.31% 34.23% 16.50% 14.96% 29.80% -36.15% 32.27% -49.20% 46.51%Generator Data Set I + II 25.31% 25.08% 24.35% 25.27% 27.06% -7.87% 37.47% 23.23% 31.45%Cluster 0 --> I 19.78%
17.65%
35.35%
27.22%
14.56%
24.30%
-20.66%
22.10%
3.70%
Cluster 1 --> II 33.91% 34.14% 16.11% 15.83% -6.48% -17.50% -20.52% 15.42% 25.44%
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
24/33
24
Application of results
Application of results (1)
The clustering procedure was applied on two realworld surveys in order to be further evaluated.The measure of success would be the
improvement of MUSAs internal quality measuresthrough the proper segmentation of the initialsample.
Survey 1: Policemen Satisfaction in Greece(sample: 1508, criteria: 8)
Survey 2: Tourists Satisfaction in Skopelos Island
(sample: 599, criteria: 5)
li i f l (2)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
25/33
25
Survey 1
Application of results (2)
Evaluation of segments production using DCr(i) transformation:
MUSA's Internal Quality Measures
Samples - Data Sets Customers AFI ASI OPL
Initial Sample 1508 93.02% 78.02% 56.63%
Cluster 0 537 92.76% 82.65% 77.09%
Cluster 1 971 95.16% 74.46% 72.81%
Cluster 0 526 94.98% 66.42% 82.32%
Cluster 1 835 96.07% 83.14% 88.38%
Cluster 2 147 85.03% 81.96% 52.38%
Cluster 0 720 96.87% 80.19% 90.83%
Cluster 1 105 82.10% 79.15% 40.95%
Cluster 2 494 96.03% 85.06% 90.28%
Cluster 3 189 93.51% 67.67% 82.01%
A li ti f lt (3)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
26/33
26
Survey 1
Application of results (3)
Evaluation of segments production using W1Cr(i), W2Cr(i) transformations:
MUSA's Internal Quality Measures
Samples - Data Sets Customers AFI ASI OPL
Initial Sample 1508 93.02% 78.02% 56.63%
Cluster 0 579 90.39% 79.65% 34.72%
Cluster 1 929 95.60% 75.63% 77.40%
Cluster 0 533 94.59% 84.22% 54.60%
Cluster 1 440 90.29% 80.52% 47.50%
Cluster 2 535 96.30% 76.01% 90.84%
Cluster 0 374 93.99% 82.40% 42.51%
Cluster 1 188 85.98% 83.47% 27.13%
Cluster 2 547 94.96% 74.41% 73.13%
Cluster 3 399 97.04% 78.24% 91.73%
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
27/33
A li ti f lt (5)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
28/33
28
Survey 2
Application of results (5)
Evaluation of segments production using DCr(i) transformation:
` MUSA's Internal Quality Measures
Samples - Data Sets Customers AFI ASI OPL
Initial Sample 599 93.18% 62.64% 53.59%
Cluster 0 214 89.48% 60.51% 57.48%
Cluster 1 385 96.25% 90.69% 77.66%
Cluster 0 186 89.04% 59.65% 56.45%
Cluster 1 342 96.69% 74.61% 84.21%
Cluster 2 71 92.31% 93.86% 39.44%
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
29/33
Application of results (7)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
30/33
30
Survey 2
Application of results (7)
Evaluation of segments production using DCr(i) transformation:
Demanding Indices
Clusters Global Cr 1 Cr 2 Cr 3 Cr 4 Cr 5
Initial Sample -60.37% -49.56% -55.99% -38.34% -12.14% -75.29%
Cluster 0 -64.50% -54.00% -54.00% -54.00% -54.00% -54.00%
Cluster 1 -55.10% -40.09% -56.97% -29.22% -10.32% -76.54%
Criteria Weights
Clusters Cr 1 Cr 2 Cr 3 Cr 4 Cr 5
Initial Sample 17.84% 20.45% 14.60% 11.24% 36.87%
Cluster 0 20.00% 20.00% 20.00% 20.00% 20.00%
Cluster 1 15.36% 21.38% 13.00% 10.26% 40.01%
Labelling: Tourists staying in hotels turn to belong into Cluster 1 while on the contrary the
ones chose to stay in rooms to let seem to belong in cluster 0.
Conclusions (1)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
31/33
31
Conclusions
Conclusions (1)
Data Mining Clustering procedure led to morehomogeneous segments of customers both insynthetic datasets and in real world surveys results.
DCr(i) transformation seems to work better thanW1Cr(i), W2Cr(i) transformations.
The labelling of the produced clusterssegments is
not always obviousMaybe more attention shouldbe paid during the designing of the survey to includemore demographical information.
Conclusions (2)
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
32/33
32
Future research
Conclusions (2)
Some improvements regarding the data mining procedure mayinclude:
Further experiments using the dataset generator evaluating theresults should be undertaken. Real world surveys should beused as well.
Other MUSA internal quality measures, recently proposed,should be also considered.
The transformations regarding the different criteria weightsshould be improved, if it is possible.
Other or new similarity metrics should be studied.
The labelling procedure should be thoroughly examined.
8/13/2019 EURO XXII Presentation Improving the Quality of Customer Satisfaction Measurements of MUSA Method Using Clustering Data Mining Techniques
33/33
33
Thank you