Upload
nguyendan
View
215
Download
2
Embed Size (px)
Citation preview
Multivariate Analysis of Sensory Data
Phucan Le
Kongens Lyngby, 2008
ii |
| iii
Technical University of Denmark Informatics and Mathematical Modelling Building 321, DK-2800 Kongens Lyngby, Denmark Phone +45 45253351, Fax +45 45882673 [email protected] www.imm.dtu.dk
iv |
Summary
In this thesis a sensory profiling data set describing the quality of fish on 17 different sensory attributes
as evaluated by ten assessors is analyzed with main focus on the differences between products (fish
given different feed and stored on ice a different amount of days. Especially the difference resulting
from different feed is of interest. The analysis will be done by a number of multivariate methods.
Comparison of the performance of the methods on the given data set is also an objective of the thesis,
but only secondary.
The data analytical strategy involves a descriptive statistical analysis to obtain an overview of the
distribution and standard deviations of the scores for each sensory attribute along with the correlations
between pairs of attributes. Especially the presence of multicollinearity is of interest as the performance
of the multivariate methods depends on this.
A Principal Component Analysis (PCA) is employed in order to visualize the main tendencies of variation.
3-way and 4-way univariate mixed models are analysed in order to model multivariate test statistics. 3-
way and 4-way mixed model MANOVA are done together with Canonical Variate Analysis (CVA) in order
to visualize the main tendencies in the same way as done with the PCA, but taking the error structure
into account and with p-values for difference in products.
A 50-50 MANOVA is done using the principles of both PCA and MANOVA, hereby obtaining test statistics
on dimension reduced data.
The statistical reliability and predictive validity of the product differences are obtained by (M)ANOVA
and cross validation.
Similar data structures are observed in the various multivariate (and univariate) methods with slight
differences. Odor, flavor and texture attributes differentiated the fish samples and the different type of
feed had an effect on the sensory evaluation of the fish.
| v
vi |
Resume
Et sensorisk profilerings data sæt der beskriver kvaliteten af fisk, givet forskelligt foder og islagret
forskellige antal af dage, evalueret af 10 dommere på 17 forskellige sensoriske attributter analyseres
med henblik på at undersøge for forskelle mellem produkter. Specielt forskelle forsaget af de forskellige
typer af foder er af interesse. Analysen udføres ved hjælp af forskellige multivariate metoder.
Sammenligninger af metoder ud fra deres brugbarhed til analyse af dette data sæt vil også blive lagt
vægt på dog kun sekundært.
Data explorative analyse udføres for at skabe overblik over distributioner og standardafvigelser af de
sensoriske attributer. Korrelationer mellem attributterne og specielt tilstedeværelsen af
multicollinearitet vil blive undersøgt da dette har stor betydning for brugbarheden af de forskellige
multivariate metoder.
En Principal Component Analyse (PCA) udføres for at visualisere hovedtendenser af variation i data.
3-vejs og 4-vejs univariate mixede modeler undersøges med henblik på senere modellering af test
statistikker for mixede multivariate modeller.
3-vejs og 4-vejs mixed model MANOVAer udføres på dette grundlag sammen med Canonisk Variate
Analyse (CVA) for at visualisere tendenser I data på same made som ved PCA, bare med fejlstrukturen
taget højde for og med p-værdier for test af produkt forskel.
50-50 MANOVA udføres som en kombination af metoderne PCA og MANOVA, hvorved p-værdier for
test på dimensionsnedsat data opnås.
Den statistiske validitet af modellerne opnås (M)ANOVA og kryds validering.
Lignende resultater opnås ved brug af de forskellige multivariate metoder med små afvigelser. Det kan
konkluderes at de anvendte lugt, smag og tekstur attributter beskriver forskellene mellem produkter og
at typen af foder påvirker den sensoriske kvalitet af fisk.
| vii
viii |
Preface
This thesis was prepared at Informatics Mathematical Modelling, the Technical University of Denmark in
fulfillment of the requirements for acquiring the Master of Science degree in engineering.
This thesis deals with the analysis of a sensory profiling data set describing the quality of fish on a
number of sensory attributes as evaluated by a number of assessors. Analysis is done by various
multivariate methods.
The thesis is divided in two parts. In Part 1 a brief introduction of the models used in the analysis is
given. In Part 2 the analysis by the methods described in Part 1 is done.
Phucan Le
| ix
x |
Acknowledgements
First and foremost I would like to thank my mum and dad. I would never have done this without you.
I would also like to thank my brothers and sisters for doing cool stuff like complaining over not being
mentioned in my thesis.
I would also like to thank my advisor Per Bruun Brockhoff for excellent guidance.
Finally I would also like to thank Grethe Hyldig for the data.
| xi
xii |
Contents
Summary ................................................................................................................................................ iv
Resume .................................................................................................................................................. vi
Preface ................................................................................................................................................. viii
Acknowledgements ................................................................................................................................. x
Contents................................................................................................................................................ xii
Part 1 Theory ........................................................................................................................................... 1
Chapter 1 Sensory Profiling Data ............................................................................................................. 3
Chapter 2 Principal Component Analysis (PCA) ........................................................................................ 7
2.1 Data representation in PCA ................................................................................................................ 8
2.2 The principle of PCA: Geometric approach ......................................................................................... 8
2.2 Properties of PCA ............................................................................................................................. 10
2.4 Residual analysis .............................................................................................................................. 11
2.5 Validation by cross validation .......................................................................................................... 11
2.6 The principle of PCA: The algebraic approach .................................................................................. 12
2.7 calculating the principal components ............................................................................................... 13
Chapter 3 Multivariate Analysis of Variance (MANOVA)......................................................................... 15
3.1 One-way Models.............................................................................................................................. 16
3.1.1 Univariate one-way ANOVA ...................................................................................................... 16
Tests of significance ....................................................................................................................... 17
3.1.2 Multigroup one-way MANOVA .................................................................................................. 18
Tests of significance ....................................................................................................................... 18
Wilks’ Test Statistics....................................................................................................................... 19
| xiii
Roy’s test ....................................................................................................................................... 20
Pillai’s test ..................................................................................................................................... 21
Lawley-Hotellings tests .................................................................................................................. 21
3.2 Unbalanced one-way MANOVA ....................................................................................................... 21
Summary of the former tests ............................................................................................................. 22
Chapter 4 Mixed Model ANOVA ............................................................................................................ 25
Estimating G and R in mixed model .................................................................................................... 26
Estimating β and γ in the Mixed Model .......................................................................................... 26
Inferential tests ................................................................................................................................. 26
Chapter 5 Canonical Variate Analysis (CVA) ........................................................................................... 28
5.1 Principle of CVA ............................................................................................................................... 29
5.2 Calculationg the canonical variates .................................................................................................. 29
5.3 CVA and PCA .................................................................................................................................... 29
Chapter 6 50-50 MANOVA ..................................................................................................................... 31
Part 2 Analysis of Fish Data .................................................................................................................... 35
Chapter 7 Fish Data ............................................................................................................................... 37
Design of data ................................................................................................................................... 39
Missing values Imputation ................................................................................................................. 39
Chapter 8 Initial Explorative Analysis ..................................................................................................... 41
Panel assessment .............................................................................................................................. 43
Chapter 9 Multivariate Analysis by PCA ................................................................................................. 51
Outliers ............................................................................................................................................. 51
Scaling or no scaling .......................................................................................................................... 52
the final model .................................................................................................................................. 53
results ............................................................................................................................................... 54
Other PCA models on subsets of data ................................................................................................ 56
PCA on subset of data not containing Time=0 and Time=12 ............................................................... 56
Chapter 10 Univariate Analysis by Mixed Model ANOVA and ANCOVA .................................................. 59
10.1 3-way univariate model ................................................................................................................. 60
Test of fixed effects ........................................................................................................................... 61
Test of random effects parameters .................................................................................................... 61
xiv |
Post hoc analysis ............................................................................................................................... 61
Validation of model ........................................................................................................................... 62
Residuals ....................................................................................................................................... 62
Variance homogeneity ................................................................................................................... 62
Outliers .......................................................................................................................................... 64
Normality of random effects .......................................................................................................... 64
10.2 4-way univariate mixed model ANOVA .......................................................................................... 66
10.2.1 4-way ANOVA ............................................................................................................................. 68
Post hoc analysis ............................................................................................................................... 69
10.2.2 3-way mixed model ANCOVA ...................................................................................................... 70
10.2.3 Validation of 4-way ANOVA and 3-way ANCOVA ......................................................................... 71
10.3 Discussion of results ...................................................................................................................... 72
Chapter 11 Multivariate Analysis bu Mixed Model MANOVA with CVA .................................................. 73
11.1 3-way mixed model MANOVA ........................................................................................................ 74
Tests of fixed effects .......................................................................................................................... 74
Post hoc analysis ............................................................................................................................... 74
Validation of model ........................................................................................................................... 79
11.2 4-way mixed model MANOVA ........................................................................................................ 79
Test of fixed effects ........................................................................................................................... 79
Post hoc analysis ............................................................................................................................... 80
11.3 Comparison with PCA .................................................................................................................... 85
Chapter 12 Multivariate Analysis by 50-50 MANOVA ............................................................................. 87
Chapter 13 Conclusion and Discussion ................................................................................................... 89
Appendix A ............................................................................................................................................ 91
Appendix B ............................................................................................................................................ 92
Appendix C ............................................................................................................................................ 94
Appendix D ............................................................................................................................................ 97
Least square mean estimates ................................................................................................................ 97
Pair wise comparisons using tukey adjustment ...................................................................................... 98
Appendix E .......................................................................................................................................... 113
Mixed model ANOVA and ANCOVA from Chapter 10 ........................................................................... 114
3-way mixed model ANOVA ............................................................................................................. 114
| xv
4-way mixed model ANOVA ............................................................................................................. 114
3-way mixed model ANCOVA ........................................................................................................... 114
3-way mixed model MANOVA with CVA .......................................................................................... 115
4-way mixed model MANOVA with CVA .......................................................................................... 116
Part 1
Part 1 Theory
2 |
| 3
Chapter 1
Sensory Profiling Data
In a sensory profiling dataset different products are assessed on a number (often large) of attributes by
a trained panel consisting of a number of assessors. If more than one assessment is done these make up
the replicates. In conventional sense the data is described by three design variables: Product, Assessor
and Replicate, and response variables given by the attributes.
In a complete design the data structure for the independent variables is as illustrated in Figure 1
Figure 1 Design variables: Product assessor replicate
In the balanced case the total number of samples is found by multiplying the number of products,
assessors and replicates.
Disregarding the replicates for a moment, the data including the attributes (response variables) can be
expressed as in Figure 2
4 |
Figure 2 the data represented as tables by 1 assessor (left) and by the entire panel (right)
For a single assessor the data can be summarized as in Figure 2 left. Here the products are given scores on
the attributes by a single assessor. The replicates could be envisioned by imagining as many tables of
this kind as there are replicates or simply as a product added to the list.
The data for the entire panel can be summarized as in Figure 2 right. Here there is a data table similar to
the one described in Figure 2 left for a single assessor for each assessor.
The main objective the sensory profiling dataset is to answer the question of difference in the products.
The methods used and the analysis should be made having this in mind.
The setup of the data leaves different reasonable ways of using the data. As discriminating amongst the
products is the overall objective, information regarding products should not be compromised.
Reasonable alternations should hence only concern averaging unfolding attributes and assessors
Two cases for attributes are relevant:
Univariate analysis by viewing only one attribute at a time
Multivariate Analysis in the case that all attributes are considered as a whole
Three ways of viewing the assessors are relevant:
| 5
One assessor at the time
The panel mean
The entire panel, all assessors
According to which case is being considered, different methods are applicable. These will be described
in the following section. In Table 1 they are listed according how they will be used in the analysis of the
data in this report:
1 assessor Panel mean Panel
Univariate
ANOVA Mixed model ANOVA Mixed model ANCOVA
Multivariate
PCA
Mixed model MANOVA Canonical Variate Analysis 50-50 MANOVA
Table 1 Statistical methods
A special feature in data of this kind is the presence of a panel. Even if the overall aim of the data is to
investigate for product differences or similarities means should be taken to ensure that the panel
delivers data that live up to certain criteria. No clear consensus exists on this issue but a number of
suggestions that are intuitively clear for an acceptable panel performance are:
Repeatability
Same products get same scores
Compare Replicates
Validity
For a single assessor to be in agreement with the panel (on a mean)
Ability to score products the same on average with other panel members
For the panel not to include too many assessors who disagree
Panel homogeneity (but assessors ensure consensus rather than
Discrimination
Ability to give different products different scores
6 |
| 7
Chapter 2
Principal Component Analysis (PCA)
PCA is a multivariate technique applied to a single set of variables. The method seeks to maximize the
variance of a linear combination of the variables. It provides the most compact representation of all the
variation in a data table. This is done by summarizing the original variables into much fever and
informative variables called scores. These new variables are linearly weighted combinations of the
original X-variables. The loadings contain the weights used for each X-variable and thus reveal the
influence of individual X-variables.
It can be used as a dimension reducing method, representing multivariate data as a low dimensional
plane. It was first described by Pearson as finding “lines and planes of closest fit to systems in space”
[Jackson, 2003]. Statistically PCA finds lines, planes and hyper planes in K-dimensional space that
approximate the data as well as possible in a least squares sense (this is equivalent to maximizing the
variances).
In section 2.1 it will be presented from a geometric point of view as a rotation of coordinates. In sections
2.6 and 2.7 an algebraic approach will be given. Understanding the link between these two approaches
is important for full understanding of the link with Canonical Variate Analysis in Chapter 5.
The concept of cross validation is described in section 2.5.
8 |
2.1 Data representation in PCA With n observations and p random variables,
pxxx ...,,, 21 the data is represented by the np data
matrix X
pnp
n
xx
xx
1
111
X
Without loss of generality it will be assumed throughout this chapter that X is centered by variable. The
variables can then be viewed as having zero mean.
The sample covariance matrix, S, of X is given as
ppp
p
ss
ss
1
111
S
Where the components of S, jks ,
n
i
kikjijnjk xxxxs1
11
represents covariances between the variable components jx and kx . The components iis are the
variance of component ix . If two components jx and kx of the data are uncorrelated, their covariance
is zero ( jks = kjs = 0). S is by definition always symmetric.
It may be noted that PCA can be done on the correlation matrix R as well.
2.2 The principle of PCA: Geometric approach In the following the method of PCA is described as a coordinate rotation. It might be useful reviewing
Appendix A on characteristic roots and vectors before proceeding.
The data table, the X matrix, with p rows and n columns can be represented as a swarm of points (n
points) in a p-dimensional space. The centering of the variables assumed in the former section,
resembles transforming the origin of the original axes to the point x .
In the following the indexing of the columns of X and the elements of X is given only by the descriptive
index. That is the n observation vectors 1x , 2x ,…, nx and the p coordinates 1x , 2x ,…, px for each of these
is given so:
| 9
nxxX 1
and
px
x
1
1x.
The p axes can be rotated by multiplying each ix by an orthogonal pp matrix A:
i
T
i xAz
Finding the orthogonal matrix A that rotates the axes, so that the new variables (the principal
conmponents) pzz ,...,1 in XAZT are uncorrelated, can be done by letting the covariance matrix of Z,
zS , be diagonal. That is
2
2
2
0
0
00
2
1
pz
z
z
z
s
s
s
S
The sample covariance matrix of xAzT is given by
SAAST
z (2.1)
By the property that a symmetric matrix S is diagonalized by an orthogonal matrix of its eigenvectors
and the resulting diagonal matrix contains eigenvalues of S (see appendix A)
p
T
0
0
00
2
1
DSCC , (2.2)
Where pii ,...1, are the eigenvalues of S, C is the orthogonal matrix whose columns are normalized
eigenvectors, ia , of S
naaC 1
From (2.1) and (2.2) it is seen that the matrix that diagonalizes yS is the transpose of the matrix C:
10 |
T
p
T
T
a
a
CA 1
The principal components are the transformed variables xaxaT
pp
T zz ...,,11 in XAZT . The
eigenvalues p ,...,1 of S are the sample variances of the principal components xaT
iiz . That is
1
2 izs
xaTz 11 has the largest sample variance and T
ppz a the smallest.
2.2 Properties of PCA Because of the way the solution is constructed the method of PCA has a number of useful properties.
1. ii
T
i ,1aa
2. Any two principal components xaT
iiz and xa
T
jjz are orthogonal for ji .
The first two properties follow from the property that A is orthonormal.
3. The principal components are uncorrelated in sample, that is the covariance of iz and jz is zero:
jis j
T
izz ,021
Saa
4. The principal components are not scale invariant. If variables are standardized before calculating
eigenvalues and eigenroots (finding principal from the correlation matrix R) then these principal
components are scale invariant.
5. Because i ’s are variances of the PC’s it makes sense to talk of “the proportion of variance
explained by the first k components:
p
j
jj
p
p
k
s
riancevaofproportion
1
1
1
1...
...
...
since )(1
Strp
j
i
.
6. Inversion of the PC model
The equation XAZT may be inverted so that the original variables may be stated as a
function of the principal components because A is orthonormal and hence A-1=A. That is
AZX
| 11
2.4 Residual analysis The last two properties of section 2.3 lead to a principal result in PCA. In the last property x will be
determined exactly only if all the principal components are used. It is possible to get an estimate of x if
only k<p principal components are used, explaining the proportion of variance corresponding to the first
k λi’s given in property 5.
The model for the first k principal components is given by:
EAZX (2.3)
Where E is a nk matrix of residuals forming
the part of X not explained by the model forms the nk matrix E of residuals. Geometrically the
residuals is given by the distance between each point in k space and its point in the plane. As before X is
a nk matrix of variables, A is the kk matrix of the first k eigenvalues and Z is the nk matrix of
transformed variables the principal components.
2.5 Validation by cross validation The residuals in the residual matrix E in (2.3) are a measure of how much of the variance the principal
component model describes with a given number of principal components. It is not however an
indication of neither how well the model will perform on a new set of data, nor of the stability of the
model. In order to assess these questions validation of the model is required. The most correct way to
do this is to test the model on a new data set. That is a calibration set, Xcal to make the model and a
validation set, Xval to test it on. In this case the calibration and validation residuals are given by
n
XXe
calcal
cal
2)ˆ(
n
XXe
valval
val
2)ˆ( (2.4)
In most cases a validation set is not available. In data with few samples a sound validation option is full
cross validation also known as Leave One Out (LOO) validation with Jack-Knifing. This will be described in
this section.
In LOO validation as many sub models as there are samples are tested, each time leaving only one
sample out and using this as the validation set. With a data set with n samples, n sub models each
12 |
containing n-1 samples are tested. The squared difference between the predicted and the X value for
each omitted sample is summed and averaged in the same sense as in (2.4)
This validation residual or equivalently the validation explained variance by the model is a good measure
of how good the model will perform on other sets of data.
The concept of studying the variation the full model and the different sub models computed with Aopt
principal components during cross validation is a modification of the established Jack-knife technique
[Martens, 2001]. The deviations between the full model and the individual local submodels are called
partial perturbations. These are a good indication of the stability of the model and can be used in order
to find possible outliers.
As was mentioned above the primary goal of validation is
Estimating the predictive ability of the model
Assessing parameter stability
Furthermore validation can be used to
Optimize the model by determining the number of optimal principal components to use
Defining limits for outlier warnings
For details on other validation methods e.g. Test set validation suited for larger data sets, segmented
cross validation for intermediate sized data sets and leverage corrected validation see [Esbensen, 2006].
2.6 The principle of PCA: The algebraic approach In this section PCA will be (re)presented as an iterative method, where eigenvalues and vectors are
obtained sequentially starting with the largest eigenvalue and its associated eigenvector, then the
second largest eigenvalue and its associated vector and so on. This is done so subsequent vectors are
uncorrelated with existing ones.
This alternative representation may seem redundant but is done in keeping with understanding later
chapter on Canonical Variate Analysis.
Given the linear combination
PCA seeks a vector a1 that maximizes the variance of z. Since the variance of z1 is has no
maximum if a1 is unrestricted the function to maximize is given by
Subject to the constraint
xazT
11
11 SaaT
11
111
aa
SaaT
T
| 13
Subsequent vectors are found
Subject to the constraint
And the additional constraints
2.7 calculating the principal components The eigenvalues and eigenvectors can be calculated by means of eigenvalue decomposition. The
maximum value of the eigenvalue is given by the characteristic equation
(2.5)
The eigenvectors are then found by the expression
(2.6)
It is worth noticing that there is no inverse of S involved before obtaining eigenvectors for the principal
components. Therefore S can be singular, in which some of the eigenvalues are zero and can be ignored.
A singular S would arise for example when n<p. The tolerance for a singular S is a very important aspect
for the use of PCA.
111 aaT
i
T
i
i
T
ii
aa
Saa
1i
T
i aa
jij
T
i ,0Saa
0 IS
0aIS
14 |
| 15
Chapter 3
Multivariate Analysis of Variance (MANOVA)
In this section univariate anova is extended to MANOVA in which more than one variable is measured on
each experimental unit. It is not the purpose of this section to present the model of the MANOVA in its
full detail, but rather to outline the basic principles regarding hypothesis testing which may serve as a
framework for later sections.
16 |
3.1 One-way Models In this section univariate ANOVA is reviewed before covering the multivariate MANOVA.
3.1.1 Univariate one-way ANOVA In the balanced one-way ANOVA we have a random sample often referred to as group of n observations,
each of g normal populations with equal variances 2 .
g groups with n observations
Sample 1 from
),( 2
1 N
Sample g from
),( 2gN
1ky 1gy
kny gny
Total 1y gy
Mean 1y gy
Variance 2
1s 2
gs
For each group total, iy , and mean , iy , are calculated
n
j
iji yy1
n
j
ij
in
yy
1
With the overall total, y , and mean , y , calculated as
gn
ij
ijyy,
1,1
gn
ij
ij
gn
yy
,
1,1
The k samples are assumed independent. This along with the assumption of common variance is
necessary to obtain an F-test.
The model for each observation is
ijiijy
| 17
iji , gi ,...,1 and nj ,...,1
Where ii is the mean of the ith population.
Tests of significance
The statistical significance test of interest is the hypothesis of no group difference (in group means):
gH 10 : against the alternative jia jiH :,:
If the hypothesis is true all ijy are from the same population with the distribution ),( 2N and two
estimates of 2 can be obtained.
One based on sample variances si , gi ,...,1 pooled within-sample estimate of 2
11 ,
22
1
22
ng
nyys
gs
ji i iijg
i
iwithin (3.1)
and the other based on sample means, gyy ,,1
1
2
22
g
yynSs i i
ybetween
1
22
g
gnynyi i
(3.2)
When sampling from a normal distribution2
withins , a pooled estimate based on the g values of si, is
independent of 2
betweens which is based on the iy ’s.
Since 2
withins and 2
betweens are independent and both 2 , their ratio form an F-statistic
)1(
122
22
2
2
ngnyy
ggnyny
s
sF
ij i iij
i i
within
between
within
between
within
between
MS
MS
ngSS
gSS
)1(
1 (3.3)
Where i ibetween gnynySS 22 is the between sample sum of squares,
ij i iijwithin nyySS 22 the within sample sum of squares,
18 |
1 gSSMS betweenbetween and )1( ngSSMS withinwithin the corresponding sample mean
squares.
The F-statistic is distributed )1(,1 nggF when H0 is true. H0 is rejected if FF , where is the
significance level.
3.1.2 Multigroup one-way MANOVA Assume g independent random samples of size n, obtained from p-variate normal populations with
equal covariance matrices
Sample 1 from
),(2
1pN
Sample g from
),(2
g
N
11y
1gy
n1y gny
Total 1y gy
Mean 1y gy
The model for each observation vector is expressed
ijiij y
iji , ijevar
, gi ,...,1 and nj ,...,1
Tests of significance
Comparison of the mean vectors of the g samples for significant differences, hypothesis of no group
difference at all
gH
10 : against the alternative jia jiH :,:
As in the univariate case the between and within sample sum of squares withinSS and betweenSS are
given by (3.1), (3.2) and (3.3).
In the multivariate case the between sample and within sample sum of squares matrices are betweenSS
and withinSS defined as
| 19
i
T
iibetween yyn yySS
i j
T
iijiijwithin yyyySS
Assuming there are no linear dependencies in the p variables, the rank of the pp matrix betweenSS is
the smaller of p and degrees of freedom, dfbetween=g-1. That is, betweenSS can be singular.
The rank of the pp matrix withinSS is p unless dfwithin =g(n-1) is less than p.
The within group error covariance matrix is estimated by
withingng
SS
1ˆ
Wilks’ Test Statistics
The likelihood ratio test of g
H 10 : is given by
total
within
SS
SS
Wilks’ , can be expressed in terms of eigenvalues of betweenwithinSSSS1
as follows
s
i i1 1
1
,
Where betweendfps ,min and p is the number of variables (dimension).
An approximate F-test is given by
1
21
1
1
df
dfF
t
t
Where
betweendfpdf 1 ,
22
12 betweendfptdf ,
12
1 betweenbetweenwithin dfpdfdf and
20 |
5
422
22
between
between
dfp
dfpt
An approximate test is given by
ln1/
2
12
betweenwithin dfpdf
which has a 2 distribution with dfbetween degrees of freedom. 0H is rejected if 22
.
Roy’s test
In the union intersection we seek the linear combination ij
T
ijz ya that maximizes the spread of the
transformed means i
T
iz ya relative to the within sample spread of points. As SaaT
zs 2 , a is found
as the vector that maximizes
ggn
gF
within
T
between
T
aSSa
aSSa 1
This is maximized by 1a , the eigenvector corresponding to 1 , the largest eigenvalue of betweenwithinSSSS1
.
This gives
1
1
1max
g
ngF
a
Fa
max has no F distribution as it is maximized over all possible linear functions.
To test g
H 10 : based on i , Roy’s union intersection test also called Roy’s largest Root test
is used
1
1
1
(3.4)
The eigenvector 1a corresponding to 1 is used in the discriminated function yaTz 1 since this best
separates the transformed means
i
T
i yz 1a , gi ,,1
The coefficients of 1a can be examined for an indication of which variables contribute most to separating
the means
| 21
Pillai’s test
The Pillai’s statistic is given by
s
i i
ibetweenbetweenwithin
s trV1
1
1
SSSSSS
Pillai’s test statistic is an extension of Roy’s statistic given by (3.4). If the mean vectors do not lie in one
dimension, the information in the additional terms, ii 1 , si ,...,3,2 may be helpful in rejecting
0H .
An approximative F-statistic is given by
)(
)(
12
12s
s
Vssm
VsNF
Approximately distributed )12(),12( sNsmsF .
Lawley-Hotellings tests
The Lawley-Hotellings statistics is defined as
s
i
ibetweenwitin
s trU1
1)( SSSS ,
where pdfs between,min .
An approximate F-statistic is given by
12
122
sms
UsNF
s
Approximately distributed )1(2),12( sNsmsF .
3.2 Unbalanced one-way MANOVA
The balanced extended to the unbalanced case, in which there are in observation vectors in the ith
group.
The model becomes
ijiij y
iji , gi ,...,1 and nj ,...,1
22 |
The mean vectors
n
j i
ij
in1
yy
g
i
n
j
ij
N1
yy
, where
g
i
inN1
The betweenSS and the withinSS matrices are calculated as
i
T
iiibetween n yyyySS
(3.5)
i j
T
iijiijwithin yyyySS (3.6)
Wilk’s and other tests have same form as in specified above for the balanced one-way MANOVA using
betweenSS and the withinSS from (3.5) and (3.6).
In each test
1 kdfbetween and
k
i
iwithin kndf1
Summary of the former tests The measure of group differences with respect to df within group variability can all be expressed by the
eigenvalues s ...1 of the matrix
betweenwithinSSSS1
where withinSS is not singular.
The four test statistics can be summarized as follows
Pillai :
s
i i
isV1 1
Lawley-Hotelling :
s
i
i
sU1
)(
Wilk’s lambda :
s
i i1 1
1
| 23
Roy’s largest root : 1
1
1
Where 1,min,min gpdfps between .
Note that for all four tests pdfwithin assumed that the number of eigenvalues is equal to the minimum
of the dimensions of the variables space and the number of groups-1.
24 |
| 25
Chapter 4
Mixed Model ANOVA
The MANOVA models used will be based on univariate analogous. To describe the data mixed model
analysis of variance is deployed.
The general linear model ANOVA assumes independent and identically distributed errors with 0 mean
and 2 variance. The mixed model ANOVA extends the general linear model by allowing a more flexible
specification of the covariance matrix of . The mixed model ANOVA allows for both correlation and
heterogeneous variances, while still assuming normality.
Let the n x 1 vector y describe the n observations. Then the mixed model for can be written as
ZXy
The matrix notation for a mixed model uses two design matrices; One design matrix X to describe the
fixed effects in the model and one design matrix Z to describe the random effects in the model. If n
denotes the number of observations in the data, p the number of fixed effect parameters in the model,
and q is the number of random effects coefficients, then X has dimension n x p and Z has dimension n x
q. The p x 1 vector, , and q x 1 vector, , are the coefficients for the fixed and random design
matrices, respectively.
key assumption in the mixed model ANOVA is that and are normally distributed with
0
0
E
R0
0G
V
The variance of y is, therefore, V = ZGZ' + R.
26 |
Estimating G and R in mixed model Estimation in the mixed model cannot be done by least squares as in the generalized linear model as
there are three additional unknown parameters besides , namely , G and R.
In many situations, the best approach is to use likelihood based methods exploiting the assumption that
and are normally distributed [Hartley and Rao, 1967]. In the following the restricted (also known as
the residual) maximum likelihood method will be considered, because it accommodates data that are
missing at random [Little, 1995]. In the balanced cases, the random effect parameters are estimated
without bias, and for this reason the REML estimator is used in mixed models [Brockhoff, 2007].
The REML log likelihood function is:
2log2
'2
1'log
2
1log
2
1, 11 pn
REML
RVRXVXVRG (4.1)
where r = y - X(X'V-1X) - X'V-1y and p is the rank of X.
The estimates of G and R are denoted and , respectively in the following.
Estimating β and γ in the Mixed Model
To obtain estimates of and , the standard method is to solve the mixed model equations
[Henderson, 1984]:
yRZ
yRX
GZRXXRZ
ZRXXRX1
1
111
11
ˆ'
ˆ'
ˆ
ˆ
ˆˆ'ˆ'
ˆ'ˆ'
The solutions can also be written as
yVXXVX11 ˆ'ˆ'ˆ
ˆˆ'ˆˆ 1XyVZG
If G and R are known then ̂ is the best linear unbiased estimator (BLUE) of and ̂ is the best linear
unbiased predictor (BLUP) of gamma – here “best” means minimum mean squared error *Brockhoff,
2007].
Inferential tests The hypothesis for test of fixed effects are often expressed as the linear combination of the model
parameters
cH ':0 L against the alternative cH ':1 L (4.2)
| 27
Where L is a matrix, or a column vector with the same number of rows as there are elements in and c
is a constant.
If (4.2) is true and inserting the BLUE then
LXVXLL11'',0ˆ' Nc (4.3)
With this distribution the Wald test can be constructed as
)'ˆ'('')'ˆ'(111 ccW LLXVXLL (4.4)
Where W is approximately 2 distributed with 1df degrees of freedom. 1df equals the number of
parameters eliminated by 0H (rank of L). The F-test becomes
1df
WF (4.5)
Combining (4.5) with Satterthwaite’s approximation of the denominator degrees of freedom, 2df , the p-
value for 0H is given by [Brockhoff, 2007]
FFPP dfdfH 210 , .
The test for random effects can similarly be based on a Wald Z statistic, which is valid for large samples.
Another alternative is the likelihood ratio 2 . This statistic compares two covariance models, one a sub-
case of the other. With A as the full model, and B as the sub-model the test statistic is given by
A
REML
B
REMLBAG 22
Where A
REML and B
REML are the two negative restricted/ residual log likelihood values from equation
(4.1). Asymptotically BAG follows a )1(2 distribution, when B differs from A with one variance
parameter.
28 |
Chapter 5
Canonical Variate Analysis (CVA)
CVA is a widely used method for analyzing group structure in multivariate data. Krzanowski (2001)
summarizes that the objective of CVA is to, “provide a low-dimensional representation of the data that
highlights as accurately as possible the true differences existing between the m subsets of points in the
full configuration”. CVA finds a weighted sum of the variables whose between-groups variation is
maximized with respect to its within-groups variation.
| 29
5.1 Principle of CVA
With g groups CVA finds the 1p vector 1a maximizing the ratio
11
11
1
maxaSSa
aSSa
bwithin
T
between
T
subject to
1iwithin
T
i aSSa
Where the between-groups covariance matrix SSbetween and the within-groups covariance matrix SSwithin are given by
T
i
g
i
iibetween n ))((1
xxxxSS
,
g
i
n
j
T
iijiijwithin
i
1 1
))(( xxxxSS
where ix is a vector of sample means for the ith group, and x is the overall mean. The notation SSbetween
and SSwithin is chosen for compatibility with Chapter 3.
Successively find ia ,...,2i ),min( pgk , maximizing the ratio
iwithin
T
i
ibetween
T
i
i aSSa
aSSa
amax
Subject to
IASSA between
T
Where A is a kp matrix with columns ai.
5.2 Calculationg the canonical variates The matrix of canonical variates is obtained by finding the eigenvectors of
betweenwithinSSSS1
as it is done in for PCA in section 2.7. Note that the inverse of SSwithin is used in finding the eigenvalues.
Hence SSwithin cannot be singular.
5.3 CVA and PCA Another method for looking at the variance of the variables of a single data set is the PCA reviewed in
Chapter 2. It is intuitively perceivable that CVA resembles a PCA expressed on group level. Campbell ant
30 |
Atchley (1981) argue that one can view a CVA as a PCA performed on group means in the space
obtained by transforming the variables by the Mahalanobis transformation, that is
s, where Sxx=SSbetween+SSwithin. In this space Euclidian distance equals Mahalanobis distance, where
Mahalanobis distance between two group means i
and j
is defined as ji
T
ji , where
is a pp within-groups covariance matrix. Further in this space IS **xx. The principal components
of the group means in this transformed space correspond to the canonical variates.
| 31
Chapter 6
50-50 MANOVA
When nq number of responses exceed the number of observations the tests of (classical) MANOVA
collapse. When several of the responses are highly correlated the tests perform poorly.
50-50 MANOVA method, suggested by Langsrud (2001), is designed to handle these cases.
The concept of the 50-50 MANOVA is to handle the problem of collinearity by PCA of the response data.
The original response variables are then replaced by a few principal components on which an ordinary
MANOVA can be performed (can be understood as a reverse PCR on multivariate response).
Existing MANOVA tests are altered so the MANOVA on principal components have statistically correct
tests.
This is done in the Appendix B.
The singular value decomposition of Y is
s
i
T
iii
1
vuY
Where ),min( qns is the rank of Y. When the tests in (B2) is valid q<n and therefore s=q.
The p-value in (B2) for the test (B1) is invariant under the transformation
s
,,1Y
That is, the original responses Y nay be replaced by the principal components scores s
,,1 . When
q>n-m-p (n, m and p given as in Appendix B) the test (B1) cannot be performed, but a valid test can be
performed using the right number of principal components.
By letting
32 |
ppmn
pp
0
IX~
Tnm 1,...,~
zzY
s
i
T
iii
~
1
~vu
Where ),min(~ qmns is the rank of Y~
. The above property of principal components give
s
pvpv 1
,~~
,~
XYX
When there are too many responses for (B1), (B2) an alternative test may be based on the first k
principal components and the p/value is computed as
s
pvpv ~~,~~
,~
1XYX (6.1)
For simplicity assume that mnq and hence mns ~ . Let M denote the orthogonal matrix of the
n-m principal component scores
Tmn
~~1M
And the rows of XM~
denoted by mn ,...,1 . Then
Tmn XM~
and
kkmn
kk
k 0
IMYM 1
1
~
This transformation will not change the p-value given in (6.1). The p-value is hence
kkmn
kkT
mnpv0
I, (6.2)
Which is the reverse of that given in Appendix B (B2) regarding response and predictor variables. As the
function pv is symmetric regarding X and Y arguments. The two expressions are (reverse but) similar and
(6.2) can analogously to Appendix B (B2) be viewed as a test comparing the variability of k ,...,1 with
| 33
the variability of mnk ,...,1 . Similarly k ,...,1 are called the hypothesis observations and the
mnk ,...,1 the error observations.
The choice of k is handled by introducing a group of d buffer observations that will not be involved in the
expression SSerror.
The p-value will then be calculated as
kdkmn
kkT
mndkkpv0
I,1
Compared to (5.28) the number of responses has been reduced from q to k. The number of rows of the
matrices is reduced by d, which can be viewed as a reduction in the error degrees of freedom from n-m-
p to n-m-p-d. For the hypothesis the degrees of freedom, p, remain unchanged.
The rule for choosing k is given by
1. Choose k=1 if 90.0~
~
1
2
2
mn
i
i
i
otherwise choose the smallest k>1 so that 50.0
~
~
1
2
1
2
mn
i
i
k
i
i
2. Choose 23 kpmnd (truncated)
This is a simplified form of the 50-50 MANOVA. It can be modified to cases q<n-m where the rules
should be modified accordingly.
34 |
| 35
Part 2
Part 2 Analysis of Fish Data
36 |
| 37
Chapter 7
Fish Data
The sensory profiling data set to be analyzed represents sensory assessment of fish quality. The samples
consist of 38 different products evaluated by 10 different assessors twice (some products in four
replicates). There are 17 sensory variables whereof five are of the type odor, seven of flavor and five of
texture. The attributes are scored on a 15 centimeter ordinal scale. The scores are then measured and
reported with two decimals.
The replicates are not blocked in different sessions, however if replicate 1 is missing, replicate 2 is also
missing for all attributes by the same assessor and product. In total there are 128 samples that have not
been evaluated. This gives a total of 632 (38 x 10 x 2 – 128) samples.
Table 2(a) shows the structure of the design variables, product, assessor and replicate, set up as in Figure 1
There is one of these for each Product. That is 38 in all.
The response variables are described by Table 2(b). For each of the 17 attributes the number, type and
name are given.
38 |
Product Replicate Assessor
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
1 10
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
2 10
Table 2 (a) Design variable structure and (b) response variables
The variable product can furthermore be partitioned into two factors: feed and time. Here feed signifies
what type of fish feed the fish were given and time signifies how many days the fish were stored on ice
prior to consummation. There are of 7 different fish feeds and 5 different storage times. The make up of
every product according to feed and time is given in Table 3.
Number Name Feed Time Number Name Feed Time
1 Blå_0 Blå 0 20 gul_12b Gul 12
2 Blå_12a Blå 12 21 gul_3a Gul 3
3 Blå_12b Blå 12 22 gul_3b Gul 3
4 blå_3a Blå 3 23 gul_5 Gul 5
5 blå_3b Blå 3 24 gul_7 Gul 7
6 blå_5 Blå 5 25 hvid_0a Hvid 0
7 blå_7 Blå 7 26 hvid_0b Hvid 0
8 grøn_0 Grøn 0 27 hvid_12 Hvid 12
9 grøn_12 Grøn 12 28 hvid_5 Hvid 5
10 grøn_3 Grøn 3 29 hvid_7 Hvid 7
11 grøn_5 Grøn 5 30 jis_3 Jis 3
12 grøn_7 Grøn 7 31 rød_0 Rød 0
13 grå_0a Grå 0 32 rød_12 Rød 12
14 grå_0b Grå 0 33 rød_3 Rød 3
15 grå_12 Grå 12 34 rød_5 Rød 5
16 grå_5 Grå 5 35 rød_7 Rød 7
17 grå_7 Grå 7 36 grå_3 Grå 3
18 gul_0 gul 0 37 hvid_3 Hvid 3
19 gul_12a Gul 12 38 Sort Sort
Table 3 Products listed by number, feed and time
Number Type Name
1 Odor Earthy
2 Cooked potato
3 Sourish
4 Sour
5 Muddy
6 Flavor Earthy
7 Mushroom
8 Cooked potato
9 Sourish
10 Sweet
11 Green
12 Muddy
13 Texture Flaky
14 Firm
15 Juicy
16 Fibrousness
17 Oiliness
| 39
Design of data From inspection of Table 3 it is seen that for the feeds Blå, Grøn,Ggrå, Gul, Hvid and Rød there is a
product for each of the five different storage times. For the combinations Blå 12 and 3, Grå 0, Gul 12 and
3 and hvid 12 there are two products, the difference indicated by index a or b. For the Feeds Jis and Sort
only one product is available. Jis with time 3 and for Sort no storage time is indicated.
In Table 4 it can be seen how the 38 products are distributed in a more schematic form. Each product is
indicated by a X. There are only 37 Xs as Sort has no storage time specified.
Storage time 0 3 5 7 12
Feed
Blå X X X X X X X
Grøn X X X X X
Grå X X X X X X
Gul X X X X X X X
Hvid X X X X X X
Rød X X X X X
Jis X
Sort Table 4 The products indicated by their feed and time
For six feeds it is possible to view data in a full factorial design.
Only Jis and Sort do not have the property of being tested for all times. These are not intended to be
analyzed on the same terms as the other feeds and are furthermore expected to be taken out of the
analysis if proven to be outliers.
The goal of the analysis is to decide whether there is difference between the different types of feed. In
the case of design variables being Product, Assessor and Replicate this translates to difference in
Product.
Missing values Imputation As mentioned before there is a total of 128x17 missing values. Where possible analysis is carried out on
the raw data, but in cases where a full balanced data set is required this will be done on an imputed data
set.
The imputations are done by replacing the missing value with the product mean across all samples for
the attribute in question. This does not change the mean of the group, but it does reduce the within
group heterogeneity on the sample. In analysis of variances this way of imputation increases the
likelihood of Type 1 error [Tinsley, 2000].
Because of the nature of the missing values is that all 17 attribute values for a sample either present or
missing, the question of missing values in this data set simply translates to missing samples. By taking
the 128 samples in question out, we are left with a full data set, unbalanced but without missing values.
40 |
| 41
Chapter 8
Initial Explorative Analysis
Before the methods described in Part 1 is done initial analysis of the data is carried out.
The normality of the variables can be assessed by viewing histograms of the scores given. In Figure 3 the
histogram for the attribute earthy(o) is given along with the normal probability plot.
Figure 3 Histogram and normal probability plot for attribute earthy(o)
As is seen no unacceptable indication of non-normality is visible. The remaining 16 variables are
inspected by similar plots and all proved acceptable.
The attributes means, standard deviations, minimum values and maximum values are given in the
following table
Variable Mean Standard deviation Minimum value Maximum value
Earthy(o) 3.9059335 1.2750875 1.3500000 9.7500000
42 |
Cooked potato(o) 3.9059335 0.9737972 1.3500000 6.9000000
Sourish(o) 2.7469937 0.8037691 0.6000000 7.2000000
Sour(o) 1.6084652 0.8349841 0 5.2500000
Muddy(o) 1.5868671 0.9206049 0 6.0000000
Earthy(f) 5.1517405 1.3708611 1.0500000 8.7000000
Mushroom(f) 4.2498418 1.3042597 0.7500000 7.9500000
Cooked potato(f) 4.3383703 1.0549837 1.8000000 7.6500000
Sourish(f) 3.2862342 0.9657570 0 7.8000000
Sweet(f) 3.5043513 0.8869100 0 7.8000000
Green(f) 2.5424051 0.9203790 0 7.0500000
Muddy(f) 2.2179589 1.1079197 0 6.7500000
Flaky(t) 7.4420886 2.7592146 0 12.7500000
Firm(t) 6.2271361 2.4412599 0 12.3000000
Juicy(t) 7.1691456 2.4068933 0 12.9000000
Fibrousness(t) 3.9168513 1.8429409 0 10.3500000
Oiliness(t) 4.3115506 1.4276590 0 9.3000000 Table 5 Mean, standard deviation, minimum value and maximum value for the 17 response variables
The data in Table 5 can be summarized in mean and standard deviation plot for each attribute
Figure 4 Plot of mean and standard deviation for the 17 attributes
The minimum and maximum value along with the 25% percentile, median and 75% percentile is
described by the box plot in Figure 5
Figure 5 Box-plot for the 17 attributes across all samples
| 43
In both of the plots above the difference between the three types of attributes is visible. The five first
(counting from left) attributes are odor attributes, the following seven are flavor and the last five are
texture attributes. The odor attributes and the flavor attributes seem to be comparable both in level and
variation. The texture attributes however differ from the other two by larger means especially for
attributes Flaky, Firm and Juicy. Furthermore the variance for all texture attributes is seen to be
considerably larger than for the other attributes.
From the box plot it is seen that the texture attributes are scored on a large percent of the scale (from 0-
13 out of 15), whereas the range covered by the other two attribute types is limited within 2
centimeters.
The correlations between the dependent variables are given in Fejl! Henvisningskilde ikke fundet..
Three pairs of variables have a correlation above 0.6. These are Cooked Potato(o)-earthy(o), earthy(f)-
earthy(o) and muddy(f)- mushroom(f). Three pairs with a correlation above 0.5. Four pairs with
correlation above 0.4. The rest all have a value less than 0.4. very few variables in the data set are highly
correlated. Contrary to what could be expected the data does not show any signs of multicollinearity
amongst the 17 response variables.
[Everitt, 2002] suggests the variance inflation factors
21
1
i
iR
VIF
as a measure of multicollinearity. Here Ri is the multiple correlation coefficient from regression of the ith
response variable on the rest of the response variables. A VIF value above 10 is suggested as an
indication of multicollinearity. These are calculated for every attribute and no multicollinearities are
found.
Panel assessment As part of the initial explorative analysis the panel is assessed. By viewing the one-way ANOVAs on the
factor Product for each assessor, the criteria described in Chapter 1 can be assessed by the following:
1. repeatability: MSE
Low MSE: good ability to deliver same scores on replicates
3. discrimination: F=MSProduct/MSE
High F-value: good ability to discriminate between samples
Furthermore the validity can be assessed by viewing the correlation of the assessor and the entire panel.
That is
2. validity: correlation with panel mean
44 |
A slight relaxation is considered to be acceptable. Instead of doing the 10x17 ANOVAs and reporting the
results, PanelCheck1 is used on the imputed data set. As mentioned earlier for analysis of variances this
way of imputation increases the likelihood of Type 1 error. But with the knowledge of the nature of the
missing data for this particular data set the results are expected to be a
Optimistic measure of repeatability
As two missing replicates will have the same value in the imputed data set
Optimistic measure of validity
Missing values will all be replaced by the product mean that is likely to correlate with the panel
mean
Pessimistic measure of discrimination
Missing values will all be replaced by the SAME value: the product mean
The correlation is likewise done in PanelCheck and on the imputed data.
In the following plot the 10x17 MSE values are plotted grouped by assessor to compare assessors in
their ability to reproduce results across replications
Figure 6 MSE values from one-way ANOVAs, grouped by assessor
Assessors 3, 4, 5 and 10 all have MSE-values less than 2 on all attributes. Assessors 1, 2, 7 and 9 all have
MSE values less than 5 on all attributes. Finally assessors 6 and 8 have one or more attributes with MSE
value larger than 10. On a whole the panel shows acceptable ability to repeat replicates, with the
possible exception of assessors 2 and 6 and 8 in particular. These show poor ability to repeat scores on
1 software package designed to analyze panel performance by Matforsk
| 45
what seems to be mostly texture attributes. This seems to be the common trend across all assessors.
The ability to reproduce scores is relatively better for flavor attributes and odor attributes than for
texture attributes.
This is illustrated better in following plot that plots the same MSE values but grouped by attribute.
Figure 7 MSE values from one-way ANOVAs, grouped by attribute
As expected from Figure 6 it is the texture attributes that the panel have most difficulties in reproducing.
Especially the attributes Flaky(t), Firm(t) and Juicy(t) differ widely from replicate to replicate. A natural
explanation of this may lie in preparation of the replicates. A well cooked fish may very well be more
firm and less juicy than a rawer one.
From viewing the MSE values it is concluded that assessors 2, 6 and 8 show poor abilities in reproducing
results on the same products. The texture attributes are the ones that the panel has the most difficulty
in “recognizing” the products on.
In order to assess the panels ability in discriminating amongst products the F-values from the one-way
ANOVAs are summarized in a plot. As before the F-values are grouped by assessor (Figure 8) and by
attribute (Figure 9).
46 |
Figure 8 F-values from one-way ANOVAs grouped by assessor
In Figure 8 the F values from the one-way ANOVAs for assessors are plotted and grouped by assessor to
compare assessors in their ability to discriminate samples.
Assessor 4, 5 and 10 has the largest F values. Furthermore the F values for most attributes are over a
significance level of 0.05. These assessors have the best ability to discriminate between products in the
panel. Assessors 1, 2, 3, 7 and 9 have acceptable F values with most attribute F values over a significance
level of 0.05. Assessor 6 and 8 have a small number of attributes with F values over the 0.05 significance
level and are not very able at discriminating between products.
| 47
Figure 9 The F-values from one-way ANOVAs grouped by attribute
From Figure 9 it is seen that the assessors are more successful at discriminating products based on
texture attributes and then on flavor attributes than on the odor attributes. This should be seen in the
light of the results from Figure 7. Good results in repeatability and poor results in discrimination seem to
be positively correlated. The opposite similarly seems to be the case.
From the F-values it is concluded that assessor 6 and assessor 8 do not only have poor ability to replicate
their scores on the same products, they also have poor ability in discriminating between different
products. The rest of the panel does not seem to have the same difficulties with assessors 4, 5 and 10
being the best performers on both criteria.
In order to assess the panel homogeneity correlation plots showing the assessors scores along with the
panel mean are viewed.
48 |
Figure 10 Correlation plot for assessor 1 on the attribute earthy(o)
In the correlation plot above the correlation of the scores from assessor 1 and the rest of the panel is
assessed on the attribute earthy(o). The line corresponds to the panel mean. The white dots the other
assessors’ scores, the red dots the assessor in questions scores for given attribute. A good assessor is in
agreement with the panel if the red dots fall on the panel mean line. Left of the line signifies that the
assessor over scores, right of the line indicates that the assessor underscores.
From Figure 10 it is seen that assessor 1 scores in accordance with the panel on the attribute earthy(o).
For one assessor there are as many of these plots as there are attributes. This gives a total of 10x17
plots. They will not be shown here but the results from viewing them will simply be stated.
Assessor 1: In agreement with panel on most attributes with a slight tendency to over score on T
attributes
Assessor 2: In agreement with panel on most attributes with large spread on T (flaky, firm, juicy) but so
are rest of panel
Assessor 3 and 4: In agreement with panel
Assessor 5: In agreement with panel on most attributes but with a big tendency to underscore on T
along with big spread
Asessor 6: Mostly in agreement, large spread on T (first three)
Assessor 7: in agreement with panel, larger spread on T (first four) over score on oiliness (t)
Assessor 8: In agreement with panel, large spread on T
Assessor 9: Has tendency to either under score or over score
| 49
Assessor 10: In agreement with panel, good spread a slight tendency to over score O and F intensity
From this it can be concluded that the panel’s homogeneity is acceptable. The attributes that cause the
most disagreement are the texture attributes which are also the attributes the assessors are best at
discriminating samples on.
50 |
| 51
Chapter 9
Multivariate Analysis by PCA
To compensate for differences in interpretation of scale between individual assessors data is averaged
over assessors and replicates before PCA is employed. The score-plot and the loading-plot are given
below.
Figure 11 Score-plot of initial dataset averaged by Product (left), correlation loading plot (right)
Outliers The final model will be validated using Martens full cross validation with jack knifing. This is closely
related to choosing the number of principal components to use in the model. The optimal number of
principal components in this model is 15.
Another use of the uncertainty test is to find outliers. The score plot with the Hotelling T2 95%
confidence ellipse is shown in Figure 11 (left) and in Figure 12 the influence plot for the 15th principal
component is given.
The influence plot shows the residual variance and the leverage of the 38 samples for the model with
the optimal number of principal components (15). The leverage is defined by the distance between a
projected point (or projected variable) and the model centre.
52 |
Figure 12 Influence plot for PCA with Sort
As can be seen the sample Sort lies outside the the Hotelling T2 ellipse. Furthermore Sort is seen to have
leverage close to 1 in the influence plot. On the basis of this Sort is taken out of the model. No other
samples deviate extremely from the others.
Scaling or no scaling The loading plot in Figure 11 (right) shows, that the most influential variables are all texture variables.
From the mean and standard deviation plot given in Figure 4 it was seen that the variables (attributes)
differ widely in both mean and particularly in variance. A possible division of the data set is to partition it
into three sets: one for all of the odor attributes, one for all of the flavor attributes and one for of all the
texture attributes. Another possibility is to scale the complete data set. If the data is standardized by the
inverse of the standard deviation
ijscaled
ij
yy
the different variances are leveled out. The PCA is then done on the correlation matrix R instead of the
correlation matrix S
A scaling of the variables may prove advantageous as it will level out the effect of the different variables.
On the other hand the texture variables may be more pronounced in differentiating between samples
and this information may be lost in an analysis of a scaled dataset.
A scaled and an un-scaled analysis are done. The explained variance as function of the number of
principal components in the model for the two models is given below.
| 53
Figure 13 Explained variance for un-scaled model (left) and scaled model (right)
The blue bars are the calibrated variances, the red bars show the validated variances when Martens full
cross validation with jack knifing is applied. Viewing the validation variances it can be seen that the un-
scaled model performs better than the scaled one. Furthermore the suggested number of principal
components is 14 and 15 respectively, for the un-scaled and the scaled model.
Figure 14 correlation loading plots for the un-scaled model (left) and scaled model (right)
Viewing the loading plots it is furthermore seen that the texture variable dominance is not as large as
the one witnessed in Figure 11 above. This suggests that the sample Sort had a misleading effect and was
correctly taken out of the analysis.
On the basis of the comparisons above the un-scaled model is chosen. No further outliers are detected,
This is seen by viewing the Hotelling T2 95% confidence ellipse in the score plot (not given here) and
furthermore underlined by the influence plot for the 14th principal component along with the stability
plots for the scores and the loadings respectively.
the final model
Figure 15 Influence plot for the model without Sort at optimal number of principal components
From the influence plot in Figure 15 no samples are seen to deviate too far from the others. The residual
are at the most found to be 0.004 and the highest leverage is less than 0.7.
The stability plots for the scores and the loadings will give a possibility to assess the uncertainty of the
model.
54 |
Figure 16 stability plot for samples (left) and variables (right)
The two plots are apparently not so telling. But in fact they are. This is a good thing as the partial
perturbations are all satisfactory.
results
Figure 17 score plots for un-scaled model indicated by feed (left) and storage time (right)
58% of the total variation in the data is described by the two first principal components. There seem to
be three clusters. One to the left along the first principal component. One on the top right and one in
the bottom right. No distinct patterns in feed or storage time are found dividing the three groups.
From the score plot in Figure 17(right) where the samples are indicated by different storage times,
time=12 in brown, time=7 in light blue, time=5 in green, time=3 in red and time=0 in blue, a clear trend
is seen along the first principal component. Five of the samples with time 12 are grouped together to
the left, the intermediate time samples are in the middle and the samples with no storage time are
grouped to the right. When viewing the second principal component another time trend is seen. With
samples with time 3 at the bottom and moving upwards till the last three samples with time 12 in the
top. No samples with time 0 seem to be strongly described by the second principal component.
The division of the intermediate samples is not clear cut. Another time trend seems apparent along the
second principal component ranging from samples with time 3 at the bottom and samples with time 12
(other than the ones along the first principal component) at the top.
By viewing the same score-plot in Figure 17 (left) with the samples indicated by type of feed a clear cut
pattern is not observed. A little misleading the color indication is as follows: Blå in Blue, Grå in green,
Grøn in red, Rød in grey, Jis in brown, Gul in light blue and Hvid in pink.
| 55
The samples with feed Gul seem to group in the negative region of both the first and the second
principal components. The most part of the other samples are in first and fourth quadrant (positive first
principal component).
The corresponding loadings are seen in the loading plot.
The correlation loading plot takes the amount of variance explained into account. The outer ellipse is
the unit-circle and indicates 100% explained variance. The inner ellipse indicates 50% of explained
variance. This plot explains the structure of the data in terms of variables.
First of all it is seen that the two attributes Fibrousness and Firm (both texture attributes) are positively
correlated. They lie between the two ellipses and explain most of the variation along the second
principal component. The two attributes Sour(o) and Muddy(f) are the only ones to the left of the plot
along the first principal component. They are negatively correlated in particular the attribute Earthy(f).
This attribute is somewhat positively correlated to all other attributes not mentioned with the exception
of Muddy(o).
There is no sign of strict correlation of variables given by their type (odor, flavor and texture) or types
that are clearly more dominant than the others. This is seen as an indication of no need to split op the
data as suggested before.
Relating this to the score plot it is noticed that the two groups of samples with time=12 are explained by
large values for Sour(o) and Muddy(f) or large values for Fibrousness(t) and Firm(t) respectively. These
attributes seem to be dominant for fish that has been stored for more days. Fresher fish is described
mainly by the other qualities.
Samples with time 0 is described by large values of Earthy(f) and most samples are described by the
attributes in quadrant 1 or 4 respectively.
Inspection of the other principal components doesn’t give any results on difference in samples by feed
either.
The analysis of the un-scaled full data set (without Sort) explained the samples with regard to the time
effect more than anything. The feed effect was either found to be non existing or “drowned” by the time
effect. The three clusters form a circle almost not just a horse shoe as could be expected for two factor
data as this. This leads to the conclusion that the data set is not optimal in describing the Feed effect.
The analysis is not completely useless though. Valuable information can be read from the plots.
samples with feed Gul and Jis seem to be described by large values of the attribute Sour(o).
samples with large storage times are described by large values in attributes of Sour(o) and
Muddy(f) or in Fibrousness and Firm textures.
56 |
Samples with small storage times are described by the rest of the attributes except for
Muddy(o)
Time is a large effect in the analysis
Other PCA models on subsets of data The PCA model mainly explained the difference in variables according to the storage time. In order to
exploit the difference in samples due to the different types of feed other subsets of the data are
exploitet. The subsets are chosen to minimize the effect coming from the storage time and hence to find
“weaker” patterns within the data set.
Only those samples with Time=12 to see what variables explain these, then all samples without
the ones with Time=12 with and without the variables found in to be highly influential for the
samples with Time=12.
Only the ones with same time
Only samples with same Feed
Only on variables that was not highly influential in first run as they seem to explain the Time
effect more than the Feed effect
An assessor at a time
All of the possibilities mentioned above have the disadvantage of not exploiting the data to its fullest
extent. If the data contains misleading data that only contributes to the noise the right possibility will
however be the right choice.
Neither of the suggestions above gave positive results in regard to explaining the effect of different
Feed.
PCA on subset of data not containing Time=0 and Time=12 From the score plot by time another PCA is found interesting. The one exploring the second principal
component without the effect from the samples with time 0 and 12.
The score- and correlation loading plot for this PCA is given below. The explained variance by validating
is found to be lot smaller than for the model on the full dataset. This is probably caused by the fact that
the current model has fewer samples.
| 57
Figure 18 plots for the model without samples with time 0 and 12. Correlation loading plot (top left), explained variance (top right), Score plots indicated by feed (bottom left) and time (bottom right)
In these score plots the effect from feed is clear.
The time effect is still evident dividing the samples in those with time 7 and those with time less than 7.
The samples with time 7 are explained by large values of the attributes Firm and Fibrousness texture. Jis
is described by Sour(o) and Green explained by Muddy(o).
Viewing the second and the third principal components a clearer pattern emerges. In these plots
samples with feed Blå and Grøn are similar and explained by Juicy(t), Sourish(f), Cooked potato(f),
Sweet(f) and Oiliness(t). Hvid and Grå are also similar and described by Muddy(f), Sourish(o), Muddy(o)
and Cooked potato(o).
Figure 19 score plot (left) and loading plot (right) for second and third principal component
As the validation of this model is not acceptable the conclusions made in this model should not be
expected to hold for a similar data. In order to investigate this it is recommended that a similar sensory
profiling is done with storage time in the range of a couple of days and not nearly two weeks. Same
attributes and samples but more replicates or for example four consecutive days. From a data set in this
form valid conclusions on difference in feed can be expected to be found however.
Reinspecting the original score plot in Figure 17 this pattern might have been spotted. But it seems to be
across both the first and second principal component and very sought at the least.
Unless it is the time effect (or the feed time interaction) that is of interest it is not recommended that
the samples are made on largely different storage times, as the time effect is quite dominant and seem
to erase all other variance there may be.
58 |
| 59
Chapter 10
Univariate Analysis by Mixed Model ANOVA and ANCOVA
In this chapter the univariate mixed models that describe the data are presented and analyzed. In
section 10.1 the data is modeled by the three factors Product, Assessor and Replicate in a 3-way ANOVA.
In section 10.2 the property of the factor Product to be described by the factors Feed and Time is
exploited resulting in 4-way mixed models. In section 10.3.2 however the factor Time is modeled as a
covariate.
60 |
10.1 3-way univariate model In this section the products are considered ignoring the information on fish feed and storage time. This
leaves three factors
Product: 38 levels
Assessor: 10 levels
Replicate: 2 levels
As replicate is taken at random of each product, it is modeled as nested within product.
The prime goal of the analysis is to test for product differences, so the main factor product is considered
fixed. As the individual levels of assessors are not of interest, but only the variance, the main effect
assessors is considered random. As mentioned before replicates are random and hence all interactions
likewise assumed random.
Let m
ijkY denote the score on the k’th replicate of product given by assessor. The data previously
described for a single attribute, m, can then be described by a univariate mixed model ANOVA given by
` ijkikijjiijk PRPAApY )( (10.1)
where is the overall mean for the attribute in question. The product main effects, ip , i=1,…,38
represent the differences in level between the average score for the different products. The assessor
main effects,jA , j=1,…,10 represent the assessor variation. The product-assessor interaction
ijPA
expresses the variance of assessors in measuring differences in products, and ikPR )( , k=1,2 expresses
the replicate variation. The error term ijk represents the residual variation.
The variation of the random effects can be written
2,0 Aj NA
2,0 PAij NPA
2,0 RPik NPR
2,0 Ak N
The model can be described in a factor structure diagram given below:
[A×p] [A]
[I] [O]
[R] p
Figure 20 Factor structure diagram for the 3-way mixed model ANOVA
| 61
In Figure 20 the random factors are indicated in brackets. This diagram is helpful in determining the tests
of the fixed effects.
Test of fixed effects Carrying out the mixed model analysis corresponding to the model given by (10.1) gives the following ANOVA table for the fixed effect for the attribute earthy(O):
Source Numerator df Denominator df F value P-value Product 37 271 2.06 0.0006
Table 6 ANOVA table for fixed effects for Earthy(O), 3-way mixed model
From the ANOVA Table 6 ANOVA table for fixed effects for Earthy(O), 3-way mixed model it is seen that the product effect is statistically significant.
Test of random effects parameters Covariance parameter Estimate Lower 95% CI Upper 95% CI
Assessor 0.3058 0.001535 0.6102
Product*Assessor 0.2684 0.1155 0.4213
Replicate(Product) -0.0235 -0.06853 0.02154
Residual 0.9933 0.8477 1.1803
-2 Restricted/Residual log Likelihood 1923.4 Table 7 Covariance parameter estimates for Earthy(O), 3-way mixed model
From Table 7 Covariance parameter estimates for Earthy(O), 3-way mixed model it is seen that the estimate of the replicate effect it is very close to 0. The covariance structure is therefore reduced. In Figure 20 this can be envisioned directly by removing [R] from the model. The results of the reduced model gives the following
Covariance parameter Estimate Lower 95% CI Upper 95% CI
Assessor 0.3060 0.001551 0.6104
Product*Assessor 0.2798 0.1302 0.4295
Residual 0.9702 0.8350 1.1412
-2 Restricted/Residual log Likelihood 1924.2 Table 8 Covariance parameter estimates for Earthy(O), reduced model
A restricted likelihood ratio test is used to compare the two variance structures (Chapter 4), with p=0.3711. This indicates that the replicate effect is non-significant and can be left out of the model.
Post hoc analysis The least square means estimates for product are in Fejl! Henvisningskilde ikke fundet. together with
the Tukey-Kramer adjusted pair wise comparisons. No statistically significant differences between two
different products were found for Earthy (O). This is probably due to the small significance level for the
individual tests in order to protect the family wise type I error, as a consequence of the large number of
pair wise tests performed.
62 |
Validation of model The validation of the model follow the outline in section 10.3.
Residuals
Normality of the standardized residuals is checked by viewing histograms with the fitted normal curve
along with normal probability plots (QQ-plots) of the residuals. This is done in Figure 21
Figure 21 Histogram (left) and QQ-plot (right) for standardized residuals for reduced model
In Figure 21 it is seen that the residuals seem to be symmetrical distributed. The normal distribution
seems to fit quite well, while the tests for normality (Table 9) rejects the hypothesis for normality. These
tests however, are sensitive for small deviations in larger datasets [Brockhoff 2002 ], so the significance
is not necessarily relevant. Furthermore, the straight line in QQ-plot indicates a good fit to the normal
distribution with the exception of eight extreme points.
Test Statistic P-value
Shapiro-Wilk 0.989098 0.0001
Kolmogorow-Smirnov 0.047819 < 0.0100
Cramer-von Mises 0.291452 < 0.0050
Anderson-Darling 1.56188 < 0.0050 Table 9 Tests for normality for reduced model
Variance homogeneity
The variance homogeneity can be investigated by plotting the residuals vs. the predicted values and vs.
the levels of quantitative effects.
| 63
Figure 22 Predicted values vs. standardized residuals for 3-way mixed model
In Figure 22 of the residuals vs the predicted values it is checked for whether the variance depends on the
means. A satisfactory plot should show
1. No patterns
2. Evenly distributed around 0
3. 95% of the standardized residuals between ±2 and 99.9% inside ±3
4. Variance not changing
The standardized residuals as a function of the predicted values is trumpet shaped for predicted values
smaller than 3.5. This is probably due to the large amount of zero scores in the data set and should not
be taken as indication of unsatisfactory residuals. Beyond this point it levels out nicely. The residuals are
distributed evenly and symmetrically around zero. Most of the residuals are between ±2, and 6 residuals
are outside ±3 (>0.1%). This indicates that the assumption of variance homogeneity is not fulfilled.
Figure 23 Standardized residuals (3-way mixed model) vs product (left) and assessor (right).
Figure 23 of the residuals vs. factor levels is checked for group dependant variance homogeneity. Across
products there does not seem to be clear differences in variability. For the assessors, the variation of
assessor 6 and 8 seems to be larger than the rest of the panel. In order examine the influence of the
residuals outside ±3 the model is re-run without these outliers. An alternative approach to determine
the possible outliers is to apply the Mahalanobis distance [Tinsley, 2000].
64 |
Outliers
Outliers are dealt with in the following way:
1. Detect
2. Check external reason
3. Redo analysis without extreme observations and compare
Figure 24 Histogram and QQ-plot for the standardized residuals for 3-way mixed model without outliers.
The histogram and the QQ-plot are similar to the once in Figure 24, except the tails in the QQ-plot are
closer to the line. Furthermore the tests for normality are no longer rejected. For the residual plot
against the predicted values the trumpet shape is less obvious and the standardized residuals are all
within the limits ±2, which indicates that the assumptions for the model are fulfilled.
The outliers that were excluded from the analysis were all from assessor 6 and 8, which corresponds to
conclusion in Chapter 8 panel assessment. When comparing the estimated product levels the
differences are very small, except for the products grøn_7, grå_0b and grå_7 which were 9, 4 and 3%
smaller respectively without the outliers.
Normality of random effects
The normality of the random effects is checked indirectly by evaluating the QQ-plots of averages of the
assessor and the product by assessor interaction. Both effects roughly follows a straight line in the
probability plot which indicates that the assumption is fulfilled.
| 65
Figure 25 QQ-plots for random effects assessor (left) and product by assessor interaction (right).
The analysis, post hoc and validation done for the attribute Earthy(o) is repeated on the 16 remaining
attributes and the results are summarized in the following table.
Factor
Mixed model ANOVA for
Earthy
Ckd
po
tato
Sou
rish
Sou
r
Mu
dd
y
Earthy
Mu
shro
om
Ckd
po
tato
Sou
rish
Sweet
Gree
n
Mu
dd
y
Flaky
Firm
Juicy
Fibro
usn
ess
Oilin
ess
Product + + + + + + + + + + + + + + +
Assessor + + + + + + + + + + + + + + + + +
Product*Ass + + + + + + + + + + + + + + + + +
Replicate(product) + + + + Table 10 Results from the 17 mixed model ANOVAs
In Table 10 a plus sign + indicates that the given factor is statistically significant for the mixed model
ANOVA modeled for the given attribute. The models can be read by the columns for the different
attributes. It is seen that the factor Assessor and the Product x Assessor interaction are statistically
significant for all models. The factor Product is seen to be statistically significant for all attributes except
two: Muddy(o) and Sweet(f). The factor Replicate is tested non significant for all attributes except four:
Muddy(o), Cooked Potato(f), Sweet(f) and Muddy(f).
These results will be used in the analogous multivariate analysis (see section 11.1) but first the 4-way
univariate models will be discussed.
66 |
10.2 4-way univariate mixed model ANOVA In this section products are considered as represented as a combination of feed and storage time. It
should be mentioned that the product Sort does not have a storage time specified and will not be part
of the analysis in this modeling. In the raw data there are a total of 14 measurements on the product
sort. The degrees of freedom will therefore be reduced by 14 in comparison with the 3-way models.
All other products are explicitly explained by both a feed and a storage time.
The factors considered are
Feed: 7 levels
Time: 5 levels
Assessor: 10 levels
Replicate: 2 levels
As before a replicate is considered random and is modeled nested within feed and time. Both feed and
time are considered fixed along with their interaction effect. All other effects are considered random.
Figure 26 Scatter plot of Earthy (O) as a function of Time with regression line.
Figure 26 shows the scatterplot for the attribut earthy (O) against the Time. It is seen that the variation for
times 0, 3, 5 and 7 looks similar in size, and that the variation at Time=12 is smaller. The regression line
is slighly decreasing.
| 67
Figure 27 Earthy (O) vs. Time. Individual patterns (left) and regression lines (right) for the different feeds.
The individual patterns (Figure 27, left) show a tendency of a curvature. Hvid and Grå has an upwards
curvature and Blå, Rød, Grøn and Gul shows a downwards curvature, if any. There seems to be a
grouping of products by feed: Hvid and Grå, Blå and Rød, Grøn and Gul.
From the regression lines by Feed (Figure 27, right) the grouping mentioned above is evident.
Furthermore there seems to be a decrease with Time. The slopes of the different lines look similar
except Grøn and Jis. It should be noted that Jis is only evaluated at a single time point, Time = 3. This
holds throughout the analysis.
Despite the fact that the three graphs all indicate a decrease with Time, the curvatures and the variation
in the scores it is questionable whether there is a linear relationship or not. Therefore the analysis
carried out both as an ANOVA and as an ANCOVA.
68 |
10.2.1 4-way ANOVA When using the same notation as before the model for the 4-way mixed model ANOVA becomes
ijkliklijljlijjilliijkl FTRAFTATFAAfttfY )( (10.2)
Where if is the fixed Feed effect, i= Hvid, Grå, Blå, Rød, Grøn, Jis and Gul and lt is the fixed Time effect
in days, l=0, 3, 5, 7 and 12. The interaction of the two effects ilft is also fixed. All other main effects and
interaction terms are random.
The factor structure diagram for this model is given by
[A×t] [A]
[I] [A×f×t] [A×f] t [O]
f×t f
[R]
Figure 28 Factor structure diagram for the 4-way mixed model ANOVA
The analysis is done following the same principles as section 10.1 the results are presented in the
following on the analysis of the results for the attributes. As the results from this analysis is to be used
to define the model for the MANOVA, the parameter estimates for the individual attributes are not
shown, but only which effects are significant.
Factor
4-way Mixed model ANOVA for
Earthy
Ckd
po
tato
Sou
rish
Sou
r
Mu
dd
y
Earthy
Mu
shro
om
Ckd
po
tato
Sou
rish
Sweet
Gree
n
Mu
dd
y
Flaky
Firm
Juicy
Fibro
usn
ess
Oilin
ess
Feed + + + + + + +
Time + + + + + + + + + + + +
feed*time +
Assessor + + + + + + + + + + + + + + + +
Feed*Assessor + + +
Time*Assessor + + + + +
Feed*Time*Assessor + + + + +
Replicate Table 11 4-way mixed model ANOVAS
In Table 11 the fixed factors, feed, time and feed*time, are listed first and random factors after. As in Table
10 a plus sign + indicates that the factor is statistically significant.
| 69
From Table 11 it is seen that the factor Feed is only statistically significant for 7 attributes and non
significant for 10 attributes. The Time effect is considerably larger and only non significant for 5
attributes.
The factor Assessor is statistically significant for all attributes except for attribute Sour(o). The
interactions are mostly non significant. Replicate is non significant for all attributes.
Post hoc analysis The Tukey-Kramer adjusted pair wise comparisons in relevant cases indicate that samples fed with Blå
are the same as factors fed with Grøn and Rød, that samples fed with Hvid equals samples fed with Grå.
Jis and Gul does nor resemble any of the other feeds.
70 |
10.2.2 3-way mixed model ANCOVA That all feeds (except jis) are evaluated for all five storage times 0, 3, 5, 7 and 12 the factor storage time
can be considered as supplementary measurement on each experimental unit. From the plots in Figure 27
Earthy (O) vs. Time. Individual patterns (left) and regression lines (right) for the different feeds. it is seen that a linear
relationship between the storage time and the scores for attribute Earthy(o) may exist. In this section
the 4-way model is modeled taking this possible relationship into account. The storage time it is
modeled as a covariate.
The model for the 3way ANCOVA is written using the same notation as before, except the Time effect is
included as a covariate.
ijklikijjijklilliijkl FRFAAxfttfY )( (10.3)
It should be noted that the only fixed interaction terms with the covariate effects are included in the
model. The analysis is done following the same principles as section 10.1. The test of the covariate effect
ilft tests the hypothesis of equal slopes and the test of the covariate lt tests the hypothesis of
existence of a linear relationship.
The results of the 3-way ANCOVA are shown below for the attributes.
Source
3-way mixed model ANCOVA for
Earthy
Ckd
po
tato
Sou
rish
Sou
r
Mu
dd
y
Earthy
Mu
shro
om
Ckd
po
tato
Sou
rish
Sweet
Gree
n
Mu
dd
y
Flaky
Firm
Juicy
Fibro
usn
ess
Oilin
ess
Feed + + + + +
Time + + + + + + + + + + + + +
feed*time + +
Assessor + + + + + + + + + + + + + + + + +
Feed*Assessor + + + + + + + +
Replicate + + Table 12 Mixed model ANCOVAs
For the 3-way mixed model ANCOVA it is seen that the factor Feed is only statistically significant for 5 of
the attributes. This is two less compared to the results from the 4-way mixed model ANOVAs. The rest of
the factors are compared to the 4-way mixed model ANOVAs quantitatively the same with a slight
favour to the ANCOVAs.
The estimated slopes are listed in the table below for the relevant models
| 71
Variable Slope Earthy -0.04600 Cooked potato -0.03354 Sourish -0.02240 Sour -0.02801 Earthy -0.06399 Mushroom -0.09129 Sourish -0.05684 Sweet -0.03032 Green -0.05317 Muddy 0.07041 Flaky -0.04113 Juicy -0.08609 Table 13 Estimated slopes in the 3-way mixed model ANCOVAs
From the slopes listed in Table 13 it is seen that the slopes for the different models vary from -0.09129 to
0.07041. Most models have negative slope the only exceptions Fibrousness(t) and Muddy(f).
10.2.3 Validation of 4-way ANOVA and 3-way ANCOVA The validation of the 4-way ANOVA and 3-way ANCOVA is done similarly as described in section 10.1.
The QQ-plots for the standardized residuals (Figure 29) for both models show that the models fit
adequately.
Figure 29 QQ-plot of 4-way ANOVA (left) and 3-way ANCOVA (right).
Comparison of residuals show that the two models fit are almost identical, which could be due to the
covariate Time only has five different levels in this data set, and therefore is explained equally well by a
fixed effect.
72 |
10.3 Discussion of results In this section the results from the 3-way ANOVA, 4-way ANOVA and 3-way ANCOVA mixed models are
summarized and discussed. The results concerning the fixed effects are of interest in terms of
interpretation of which attributes account for product differences.
The random effects are of interest for later modeling of multivariate tests.
First the results from the 3-way mixed model ANOVA is summarized. Only attributes Muddy(o) and
Sweet(f) are not found to discriminate amongst products. Compared to the other two models this can
be translated into the factors Feed and Time combined . Assessor is found to be significant for all
attributes, replicate is found non significant for most attributes and the Product x Assessor interaction is
likewise significant for all attributes. This interaction translates into the interactions Feed x Assessor and
Time x Assessor and Feed x Time x Assessor in the other models.
Generally the results of the two 4 factor models are very similar. Neither the Feed x Time effect or the
replicate effect is significant in most of the models. The Time x Feed interaction (different slopes in the
ANCOVA model) is only significant in a very few cases. Unexpectedly, the Feed effect is only significant in
7 and 5 of the 17 attributes in the ANOVA and ANCOVA model, respectively. Time is generally significant
in all models. The variance term Assessor x Feed interaction is significant in 3 and 8 of the 17 attributes
in the ANOVA and ANCOVA model, respectively. Finally, Assessor is significant in all models.
Feed effect is found in attributes Earthy(o), Earthy(f), Flaky(t) and Oiliness(t) in both models.
Furthermore the attributes Cooked Potato(o), Sourish(o), Sour(o) and Sourish(f) are found to
differentiate between products for the ANOVA and the ANCOVA respectively.
Comparison with the 3-way mixed model ANOVA results are as expected. Once the proper “translations”
are made the results are very similar.
| 73
Chapter 11
Multivariate Analysis by Mixed Model MANOVA with CVA
in this chapter the univariate mixed model ANOVAs are extended to the multivariate analogues. In
section 11.1 a 3-way mixed model MANOVA is analysed with a CVA in section 11.1.1. In section 11.2 a 4-
way mixed model MANOVA is analysed together with CVAs in section 11.2.1.
74 |
11.1 3-way mixed model MANOVA The model for the 3-way mixed model MANOVA is formulated based on the 3-way univariate mixed
model ANOVAs. From section 10.1 results
ijkikijjiijk )(PRPAApY (11.1)
Where is the mean vector for the attributes. The product main effects, ip , i+1,…,38 represent the
differences in level between the average score for the different products. The assessor main effects, jA
, j=1,…,10 represent the assessor variation. The product-assessor interaction ijPA expresses the
variance of assessors in measuring differences in products, and ik)(PR , k=1,2 expresses the replicate
variation. The error term ijk represents the residual variation.
Tests of fixed effects The test for effect of product is tested up against the error term product by assessor interaction. This is
the analogue test performed in the univariate case where the model has been reduced of the factor
replicate (see section 10.1).
The test statistics for this test is given in Table 14
Statistic Value F Value Num DF Den DF P-value
Wilks' Lambda 0.02134986 1.80 629 4010.6 <0.0001
Pillai's Trace 2.94306619 1.53 629 4590 <0.0001
Hotelling-Lawley Trace 5.61148236 2.25 629 2758.2 <0.0001
Roy's Greatest Root 2.51250109 18.33 37 270 <0.0001 Table 14 Test statistics for 3-way mixed model MANOVA
It is seen that the Product effect is highly significant with p < 0.0001 for all four multivariate test
statistics. Compared to the results in section 10.1 the Product was not significant for the attributes
Sweet (f) and Muddy (o). This underlines the strength of using multivariate analysis of variance as
opposed to univariate.
Post hoc analysis Often univariate ANOVAs are performed after showing product differences. This have already been
done in the modeling of the multivariate models. Most attributes were found to differentiate between
products, but no products were found to be different.
The post hoc analysis for the MANOVA can be done similarly as for the univariate case i.e. performing
pair wise comparisons between products. In this report however, it is chosen only to analyze the data
further with a canonical variates analysis, CVA described in Chapter 5, that does not rely on pre specified
combinations of the variables.
The score plot for the canonical variables indicated by feed is shown below.
| 75
Figure 30 Score plot for canonical variables 1 and 2 from CVA of 3-way MANOVA indicated for different feed.
It is seen that product Sort is a possible outlier. As mentioned in Chapter 7, this could be expected.
Furthermore Sort was found to be an outlier in the PCA analysis. The CVA is redone without Sort.
The score plot for the CVA without the sample Sort is shown below.
76 |
he
Figure 31 Score plot for canonical variables 1 and 2 indicated for different time when excluding Sort.
In Figure 31 the products are color coded by their Time. Samples with Time=12 is shown in brown, Time=7
shown in red, Time=5 shown in orange, Time=3 shown in yellow and Time=0 shown in green (Note: this
is not the same color codes used in the PCA section). It is seen that there is a clear trend along the
canonical variable 1. The larger times to the left and descending towards right.
| 77
Figure 32 Score plot for canonical variables 1 and 2 indicated for different feed.
Figure 32 corresponds to Figure 31 except the 37 products are coded by the Feed. Blå is shown with the
color blue, Grå is shown in grey, Grøn is shown in green, Gul is shown in yellow, Hvid is shown in pink, Jis
is shown in orange and Rød is shown in red. A trend, however less obvious, is seen along canonical
variable 2. Hvid and Grå in the top, followed by Rød and Blå , and finally Gul, Grøn and Jis.
78 |
Figure 33 Structural loading plot for CVA without Sort
As the main interest is in what variables are similar and which are different, only the structural loading
plot is given above. The attributes are coded by their number. The attribute name can be seen from Table
2 (b).
Viewing the structural loading plot along with the score plot in Figure 31 the attributes Muddy(f), Firm(t)
and Sour(o) describe the samples with large Time. On the other end of the scale attributes Mushroom(f),
Sourish(f), Green(f) and Sweet(f) are positively correlated and describe samples with small Time
(Time=0). These attributes are hence negatively correlated with the attributes describing the samples
with large Time: Muddy(f), Firm(t) and Sour(o).
The attributes Flaky(t) and Juicy(t) are likewise positively correlated. They have large values for samples
with small Time. Comparing with the score plot in Figure 32 these attributes are seen to explain the
samples with Feed Hvid and Grå, while the attributes in the bottom are likely to explain the samples Gul,
Jis and Grøn.
Comparing this with the results from the PCA will be done in section 11.3.
| 79
Validation of model The validation of the multivariate setting covers
1. Multivariate normality (Sampling distributions of the dependent variables and all linear
combinations of them are normal).
2. Homogeneity of covariance matrices
3. Linearity (linear relationships between all pairs of dependent variables exist)
4. No multicollinearity (correlation r < 0.8 and vif < 10)
5. No singularity (a dependent variable is a linear combination of other dependent variables)
The MANOVA test is fairly robust to departures from multivariate normality, why this section will focus
on validation of equality of variance matrices and multi collinearity. The latter is the most important
validation step of the MANOVA [Rencher, 2001].
In Chapter 8 it was shown that based on the VIF criteria there are no multicollinearity between the
attributes. The correlations were likewise all found to be under 0.8.
11.2 4-way mixed model MANOVA From the results from section 10.2 it was seen that the 4-way ANOVA and the 3-way ANCOVA models
differed very little. As the slopes in the ANCOVA model were found to be very close to zero and
furthermore to be ascending or descending for different attributes only the ANOVA will be modeled in a
multivariate analogue.
When using the same notation as before the model for the 4-way mixed model MANOVA becomes
ijkliklijljlijjilliijkl )(FTRAFTATFAAfttfY (11.3)
Where if is the fixed Feed effect, i= Hvid, Grå, Blå, Rød, Grøn, Jis and Gul and
lt is the fixed Time effect in
days, l=0, 3, 5, 7 and 12. The interaction of the two effectsilft is also fixed. All other main effects and
interaction terms are random.
Test of fixed effects The effect of interest is primarily the Feed. Also the Time and the Feed x Time interaction will be tested.
The test statistics used in the univariate case is modified to accommodate the MANOVA for the data set.
As were seen from section 10.2.1 and 10.2.2 the Replicate effect were shown to be non-significant for all
attributes. The Feed x Assessor interaction is significant for more than half the attributes. The Time x
Assessor interaction is likewise tested out for almost half the attributes.
In the following the Replicate will be left out of the test statistic calculations. The two tests for Feed and
Time Will be done by Feed x Assessor, Time x Assessor respectively. The interaction Feed x Time x
80 |
Assessor was non-significant for all attributes. The multivariate test for the effect Feed x Time will be
carried out in the multivariate case non the less.
The test of the Feed effect
Statistic Value F Value Num DF Den DF P-value
Wilks' Lambda 0.05327347 1.36 102 206.54 0.0322
Pillai's Trace 2.08203250 1.25 102 240 0.0844
Hotelling-Lawley Trace 4.60435857 1.51 102 126.34 0.0135
Roy's Greatest Root 2.46206468 5.79 17 40 <.0001 Table 15 Test statistics for test of Feed effect
As can be seen from the table above the Feed effect is significant on a 0.05 significance level in all
multivariate test statistics apart from Pilai’s Trace. Compared to the univariate tests already carried out
in section 10.2.2 where Feed was only significant for 7 out of 17 attributes, the two results differ.
The test for no overall Time effect, gives the result shown in Table 16
Statistic Value F Value Num DF Den DF P-value
Wilks' Lambda 0.02786711 1.68 68 76.88 0.0135
Pillai's Trace 2.05896145 1.37 68 88 0.0809
Hotelling-Lawley Trace 8.55330218 2.23 68 45.568 0.0022
Roy's Greatest Root 6.37157069 8.25 17 22 <.0001 Table 16 Test statistics for test of Time effect
The Time effect is significant on a 0.05 significance level in all test statistics except from Pilai’s Trace.
This result matches the results from the univariate tests carried out in section 10.2.1 where Time was
found to be significant factor for 12 out of 17 attributes.
For completion the test for no overall Feed x Time effect is listed below.
Statistic Value F Value Num DF Den DF P-value
Wilks' Lambda 0.08962842 1.04 340 1742 0.2942
Pillai's Trace 2.12410575 1.04 340 2482 0.2979
Hotelling-Lawley Trace 2.76877232 1.04 340 1071.8 0.3052
Roy's Greatest Root 0.51675620 3.77 20 146 <.0001 Table 17 Test statistics for Feed x Time effect
The Feed x Time effect is non-significant in all tests except from Roy’s greatest Root. This result is the
same as most results in the analogue univariate tests shown in section 10.2.1 where the feed x time
effect was only found to be statistically significant for one attribute.
Post hoc analysis The univariate tests have already been done in section 10.2.1 to summarize Feed effect is found in
attributes Earthy(o), Earthy(f), Flaky(t), Cooked Potato(o), Sourish(o), Sour(o) and Oiliness(t) in ANOVA.
Furthermore differences were found significant for several pairs of Feed.
| 81
As post hoc analysis to explore the differences in Feed and Time effects respectively CVA’s are carried
out for each case. As mentioned in Chapter 7 Sort does not have a Time and does not take part in the
analysis.
As the CVA from the Feed effect test, seeks the canoncal variates that lead to the largest spread in
products given by the Feed only the plot color coded by the different Feeds is given below.
Figure 34 Canonical score plot maximizing spread of products by their Feed, indicated by Feed
From the canonical score plot above essentially the same conclusions can be drawn as from the similar
score plot in Figure 32 for the CVA from the 3-way mixed model MANOVA. There seem to be grouping of
the products by feed with Gul, Grøn and Jis in one group and Rød and Blå followed by Hvid and Grå in
another.
The grouping however is more distinct here than in Figure 32. This is a result of the different model and
the multivariate test, ensuring maximal spread according to Feed. Another consequence of this can be
read directly from the plot where the separation of Feed is explained by the first canonical variate, and
not the second as in Figure 32.
Again the interest is on the structural interpretation and the structural loading plot is shown below.
82 |
Figure 35 Structural loadings for CVA on the 4-way MANOVA test of Feed
From the structural loading plot it is seen that the leftmost samples in the score plot, that is those with
Feed Gul, Jis and Grøn, are described by high values in the attributes Sour(o), Sweet(f), Mushroom(f),
Muddy(o) and Sourish(f). These attributes are highly correlated and negatively correlated with the
attributes on the other end of the scale: Juicy(t), Earthy(f) and in particular Flaky(t). These attributes are
characterized in high values for the rightmost samples in the score plot Figure 34 which are Hvid and Grå.
Comparing to the univariate results of section 10.2.1 these are not the exact same attributes found
influential in the univariate analysis.
| 83
Figure 36 Score plot for the CVA on the 4-way mixed model MANOVA test of the factor Time
The score plot for the CVA resulting from the multivariate test for Time effect is shown above. This plot
is clearly different than the one in Figure 34 other than just the color indication. The canonical variates are
found to spread the products the most according to the factor Time. As a result the Time effect is seen
along the first canonical variate. Compared to Figure 30 where the Time effect is similarly most evident
along the first canonical variate, the grouping seems more underlined here.
There is not a clear time trend in the sense that the groups are not plotted in a strict descending order.
Time=12 is the leftmost group and 0 is the rightmost group as could be expected. But the groups for
Time=3, 5 and 7 do not express an explicit ranking given by the first canonical variate. Viewing only the
samples from the three intermediate Times an other time trend can be spotted along the second
principal component. Samples with Time=3 in the bottom and samples with Time=7 at the top.
This phenomenon of two time trends along both the first and the second component was also seen in
the PCA of the full samples except Sort. The same pattern in the score plot was found with Time=12 and
Time=0 explained by the first component and the samples with intermediate Times explained by the
second. This is an indication of the Time effect being so dominant that the PCA performs like a CVA from
an multivariate test of the Time effect.
84 |
As before the structural loading plot is shown.
Figure 37 Structural loading plot for the CVA for the mixed model MANOVA test of the factor Time
An interesting pattern is seen from the loading plot for the CVA separated in Time. The attributes Firm(t)
and Flaky(t) both explain the larger Time samples along the second canonical variate, but at the same
time Firm(t) describes the samples with Time=0 along the first canonical variate. The same is the case
with the attribute Muddy(f) the other way around as the attribute is seen to be negatively correlated
with the attribute Flaky(t). The attributes Sour(o) and Fibrousness(t) are also influential in describing the
Time effect.
Compared to the loading plot for the PCA in Figure 14 a similar pattern can be found. Only in the case of
the PCA attributes found less important here (the attributes in the middle of the loading plot) are highly
negatively correlated with the attributes connected with the samples with Time=12. They are grouped
with the attribute Flaky(t) in the PCA. This could be an indication that the Time effect and the Feed x
Time interaction are in fact more evident than the Feed effect in this particular data set.
The validations of the two models follow the outline given in section 11.1.
| 85
11.3 Comparison with PCA In the following the different models of the ANOVAs and Manovas are listed. For simplicity the replicate
effect is omitted. The model only including Product is included
Model 1: Product ANOVA: F=MSProduct/MSE MANOVA test of Product
CVA
Model 2: Product + Assessor + Product*Assessor ANOVA: F=MSproduct/MSproduct*assessor MANOVA test of Product
CVA
Model 3: Feed + Time + Feed*Time + Assessor + Feed*Assessor + Time*Assessor ANOVA: F=MSFeed/MSFeed*Assessor MANOVA test of Feed
CVA
ANOVA:F=MSTime/MSTime*Assessor MANOVA test of Time CVA
Table 18 Different simplified ANOVA and MANOVA models
The 3-way mixed model MANOVA in section 11.1 resembles Model 2 in Table 18. The 4-way mixed model
MANOVA in section 11.2 resembles Model 3 a and b respectively. These are mixed models taking the
variation from the Assessors into account.
Model 1 has not been modeled in this report (not regarding the 1-way ANOVAs done for each assessor
in the panel assessment). The MANOVA with the CVA in this case would be an exact analogue of the PCA
done in Chapter 9, only taking the error structure into account and having p-values for test statistics for
the test of Product differences.
The models in Table 18 model the data set with increasing complexity and detail. The results are similarly
increasing in quality.
In the PCA a Time trend was found along both the first and the second principal components. The Time
effect was found to be explained by attributes Muddy(f), Firm(t), Sour(o) and Fibrousness(t) for large
Times and all other attributes excluding Muddy(o) and those already mentioned explaining samples
with small Times. Interpretation was made difficult resulting from a overshadowing Time effect.
In the PCA the Feed effect was indecisive.
In the 3-way mixed model MANOVA with CVA a Time trend was found along the first canonical variate.
Attributes Sour(o), Muddy(f) and Firm(t) explain the samples with large Time. Attributes Green(f),
Sourish(f), Mushroom(f) and sweet(f) describe samples with small Time.
A feed effect is found along the second canonical variable. Gul, Grøn and Jis described by the attribute
Sourish(f) and Rød and Blå followed by Hvid and Grå described by the attributes Flaky(t) and Juicy(t).
86 |
As in the PCA the interpretation of the results is not completely straight forward resulting from the large
effect of the Time.
The Feed effect and Time effect is modeled directly into the model for the 4-way mixed model
MANOVA.
The results from the CVA, maximising the spread amongst samples by their Feed, leads to a Feed effect
visible along the first canonical variable. The grouping is clear with Gul, Grøn and Jis in one followed by
Rød and Blå and then by Hvid and Grå. The attributes Sweet(f), Sour(o), Mushroom(f), Muddy(o) and
Sourish(f) explains the samples in the end containing Gul. The attributes Earthy(f), Juicy(f) and Flaky(t)
describes the samples in the end containing Hvid.
The result from the CVA, maximizing the spread amongst samples by Time, leads to a Time effect visible
along both the first and the second canonical variable. Samples are clearly separated along the first
canonical variable with samples with Time=12 in one end and samples with time 0 in the other end. The
effect along the second canonical variate is less pronounced than that in the first. It seems to be a time
effect underlying the Time effect in the first. Here the samples with Time=3 is in one end and samples
with time=7 in the other. The attributes Firm(t), Flaky(t), Sour(o), Muddy(f) and Fibrousness(t) are found
to explain the difference in Time. The interpretation is not completely clear as there seem to be a two
way interpretation for the attributes Firm(t) and Muddy(f).
The results from the different analysis’ differ slightly in what attributes are most influential (and also
from the attributes found influential in univariate analysis). Other than that a clear Time effect is found
for all models. No clear Feed effect is found by the PCA. The 3-way mixed model MANOVA with CVA
showed sign of Feed effect but not fully interpretable. Only the 4-way mixed model MANOVA with CVA
gave clear results both on the Feed effect and the Time effect.
| 87
Chapter 12
Multivariate Analysis by 50-50 MANOVA
So far either a PCA have been done or a MANOVA have been done. It is a possibility to do a PCA on the
data first and then do a MANOVA on a number of the principal components found reasonable. This way
the analysis falls into the same box as the PCA in Fejl! Henvisningskilde ikke fundet.. Another way of
doing a combined PCA and MANOVA is the 50-50 MANOVA. This method does not average the data
over assessors and replicates and hence fall into the same box as the MANOVAs in . The 50-50 MANOVA
has been implemented in Matlab and R and is free to download and use. It is however still quite rough
and options of coding variables as random do not exist. In order to test the mixed model MANOVA as in
section 11.1 precautions have to be made.
As no scaling is done in the PCA, no scaling will be done in the 50-50 MANOVA.
First the model resulting in the following table is run.
Source DF exVarSS nPC nBu exVarPC exVarBU p-Value
Code 37 0.159523 5 12 0.784 1.000 0.000000
Assessor 9 0.270473 6 11 0.826 1.000 0.000000
code*assessor 270 0.308476 11 6 0.909 1.000 0.000000
code*replicate 37 0.028832 9 8 0.862 1.000 0.770615
assessor*replicate 9 0.005878 8 9 0.831 1.000 0.836566
Error 269 0.215208 Table 19 Result from 50-50 MANOVA with Replicate in model
As can be seen the two replicate interactions are non significant. The model averaging over replicate,
only containing the factors Product, Assessor and Product*Assessor resembles the mixed Model 2 in
Table 18.
The results from this model is given below
Source DF exVarSS nPC nBu exVarPC exVarBU p-Value
Code 37 0.159560 6 11 0.818 1.000 0.000000
Assessor 9 0.270401 6 11 0.817 1.000 0.000000
code*assessor 270 0.308477 11 6 0.908 1.000 0.000000
Error 315 0.249526 Table 20 Results from the 50-50 MANOVA
88 |
As can be seen the factor Product is tested statistically significant. The 50-50 MANOVA gives the same
result as the corresponding 3-way mixed model MANOVA.
The comparison here is not so much in favor of the 50-50 MANOVA as no indication of multicollinearity
in the data. In this case where the optimal number of principal components were found to be 14 (only a
reduction in dimension of three) the two models are expected to be quite similar. The PCA done in this
section is not the same as the one done in Chapter 9 but the principle and models are still applicable for
visualization and explanation
The validation of the 50-50 MANOVA is done similar as for PCA in Chapter 9 and will not be repeated
here.
| 89
Chapter 13
Conclusion and Discussion
The analysis of a sensory profiling set describing quality of fish on 17 odor, flavor and texture attributes
is carried out by various multivariate methods.
In Chapter 8 initial descriptive analysis of the data showed no indication of multicollinearities between
attributes.
In Chapter 9 a PCA is employed on the data averaged over assessors and replicates. Product difference is
found on the sensory attributes, mainly as a result of the different storage times. Validation and
reliability of the model is assessed by full cross validation with jack-knifing.
As no feed effect is found in the preliminary analysis a further PCA is done on a subset of the data
containing fewer samples limiting the difference of the storage time. Though not reliable this PCA
suggests that the products are differentiated as a result of the different feeds. This remodeling of the
data is recommended in later profiling data sets, if indeed it is the feed effect that is of interest and PCA
is the method that is used for analysis.
In Chapter 10 the data is modeled by mixed models in which only the variation from assessors (and
replicates) are taken into account. The results from 3-way and 4-way mixed model ANOVAs and
ANCOVAs indicate product differences for most attributes and feed effect for less than half of the
attributes. In all analysis the effect of the replicates are seen to be non-significant.
Multivariate mixed models are modeled based on the results from the univariate analysis. In Chapter 10
3-way and 4-way mixed model MANOVAs indicate product difference and feed effect. These differences
are explored by means of CVA.
Finally the results from the 50-50 MANOVA are presented in Chapter 12. The difference in products is
found to be significant.
The multivariate methods MANOVA and 50-50 MANOVA gave similar results. Product difference is
found. Furthermore difference is found as a result of both feed and storage time. The nature of the
differences is not explained by these methods.
90 |
Univariate ANOVAs( and ANCOVAs) give an indication of what attributes show differences in the
products.
Similar information can be found in a truly multivariate setting by means of CVA and PCA. The results
from these differ slightly and overall the CVA results are easiest to interpret.
From analysis of this particular data set it can be concluded that mixed model MANOVA with CVA is the
best means of analysis. In the univariate analysis product difference is not found for more than half the
attributes. The results from the PCA indicate product differences but the systematic variance of the data
is dominated by the storage time effect and the feed effect is not found by analysis of this data set. The
50-50 MANOVA does not really offer anything new in this case. It should be emphasized again though
that this particular data set has no multicollinearities as could be expected for an average sensory
profiling data set.
The results from the mixed model MANOVA with CVA is restated as the results from the final analysis:
Analysis indicate product differences. Fish samples fed by Gul, Grøn and Jis are described by the
attributes Sweet(f), Sour(o), Mushroom(f), Muddy(o) and Sourish(f). Samples fed with Rød and Blå and
even more Hvid and Grå are better explained by attributes Earthy(f), Juicy(f) and Flaky(t).
The attributes Firm(t), Flaky(t), Sour(o), Muddy(f) and Fibrousness(t) are found to explain the difference
in samples with different storage times.
| 91
Appendix A
Characteristic Roots and Vectors
The method of the PC’s is based on a key result from Matrix algebra. Let U be an orthonormal matrix
that is the columns in U; ui and uj holds i and ji
1i
T
i uu (A.1)
0j
T
i uu (A.2)
A pp symmetric, nonsingular matrix (such as the covariance matrix S) may be reduced to a diagonal
matrix L by
LSUU T
(A.3)
Diagonal elements of L ( pll ,...,1 ) are called characteristic roots, latent roots or eigenvalues of S.
Columns of U ( puu ,...,1 ) are called e characteristic vectors or eigenvectors of S.
(Geometrically: elements of e are the direction cosines of the new axes related to the old:)
’s may be obtained from the solution of the characteristic equation:
0 IS l (A.4)
e may then be obtained by solution of the equations
0 IS l (A.5)
And
iTi
i
itt
tu (A.6)
92 |
Appendix B
General Test on a Subset of the X’s
This appendix is for use of derivation of the 50-50 test used in the 50-50 MANOVA. Specific test statistics
will not be discussed. The point of interest is the p-value, as a function of X and Y denoted as YX,pv .
Consider the following model
)()()()( qnqnnnqn E BXY
Where the difference from the former convention is that there is n x’s where n is the number of
observations. It is assumed that 10 nxxX has full rank n, the rows of E are uncorrelated
multinormal ,0qN , 10 n1x which means that the B0 is the intercept terms.
Here the test considered is
010 ,...,: nm BBH against the alternative
0
0
1
11
,...,
,...,:
npm
pmm
ABB
BBH (B1)
The H0 model contains the first m x’s and the testing is of the next p (Note that p in this section is not
the same as in the former) .
A p-value for a test statistic testing H0 in (B1) for the linear combination of for the linear combination of
Y is desired.
Here the details of retrieving the p-value will not be given. For full detail on deduction see Langsrud
(2002).
Consider the linear combination Z of Y.
YXzzzT
Q
T
n 110
where the orthogonal matrix QX is given by the QR decomposition of X.
| 93
RQXXX
Under H0 in (B1) 1,..., nm zz are observations from ,0qN . Under HA it is clear that 1,..., npm zz still
follow this distribution. These vectors are called error observations. Similarly 1,..., pmm zz are referred
to as the hypothesis observations. The hypothesis test can be viewed as a comparison of the hypothesis
observation with the error observations. Hence the p-value can be expressed as
T
nm
ppmn
pppv 1,...,, zz
0
I (B2)
Where it is noted that the dimensions of the input matrices to pv, pmn )( and qmn )( ,
correspond to the number of responses, q, the degrees of freedom for the hypothesis, p, and the
degrees of freedom for the error (n-m-p). It is further noted that the test described by (B1) and (B2)
collapses when the number of responses pmnq .
94 |
Appendix C
Correlations of Response Variables
Pearson Correlation Coefficients, N = 632 Prob > |r| under H0: Rho=0
Earthy_O_
Cooked_potato_O_
Sourish_O_
Sour_O_ Muddy_O_
earthy_f_ Mushroom_F_
Cooked_potato_F_
Sourish_F_
Sweet_F_
Green_F_
Muddy_F_
Flaky_T_ Firm_T_ Juicy_T_ Fibrousness_T_
Oiliness_T_
Earthy_O_
Earthy(O)
1.000
00
0.65338
<.0001
0.500
49
<.000
1
0.068
42
0.085
7
0.206
73
<.000
1
0.421
50
<.000
1
0.3670
5
<.0001
0.22004
<.0001
0.134
95
0.000
7
0.024
31
0.541
9
0.188
81
<.000
1
-
0.040
83
0.305
4
0.085
07
0.032
5
-
0.008
96
0.822
2
0.092
00
0.020
7
-
0.04403
0.2690
0.059
36
0.136
1
Cooked_pota
to_O_
Cooked
potato(O)
0.653
38
<.000
1
1.00000
0.474
03
<.000
1
0.119
85
0.002
5
0.201
44
<.000
1
0.291
37
<.000
1
0.2648
0
<.0001
0.34806
<.0001
0.046
19
0.246
2
0.077
12
0.052
6
0.208
05
<.000
1
0.004
70
0.906
0
0.215
29
<.000
1
0.100
15
0.011
8
0.156
35
<.000
1
0.08397
0.0348
0.154
16
<.000
1
Sourish_O_
Sourish(O
)
0.500
49
<.000
1
0.47403
<.0001
1.000
00
-
0.006
28
0.874
8
0.078
47
0.048
6
0.317
46
<.000
1
0.2675
3
<.0001
0.20909
<.0001
0.337
82
<.000
1
0.179
68
<.000
1
0.128
79
0.001
2
-
0.019
01
0.633
4
0.147
49
0.000
2
0.069
69
0.080
0
0.092
08
0.020
6
0.04068
0.3073
0.095
05
0.016
8
Sour_O_
Sour(O)
0.068
42
0.085
7
0.11985
0.0025
-
0.006
28
0.874
8
1.000
00
0.483
10
<.000
1
-
0.076
34
0.055
1
-
0.0422
7
0.2887
-0.03058
0.4428
-
0.151
72
0.000
1
-
0.104
13
0.008
8
-
0.017
07
0.668
5
0.139
06
0.000
5
0.112
48
0.004
6
0.060
16
0.130
9
0.005
07
0.898
7
0.04237
0.2875
-
0.012
56
0.752
7
Muddy_O_
Muddy(O)
0.206
73
<.000
1
0.20144
<.0001
0.078
47
0.048
6
0.483
10
<.000
1
1.000
00
-
0.107
40
0.006
9
-
0.0388
0
0.3301
-0.01038
0.7946
-
0.152
30
0.000
1
-
0.107
37
0.006
9
0.109
86
0.005
7
0.387
89
<.000
1
0.032
35
0.416
9
-
0.085
03
0.032
6
0.003
64
0.927
2
-
0.11768
0.0030
0.023
87
0.549
2
earthy_f_
Earthy(O)
1
0.421
50
<.000
1
0.29137
<.0001
0.317
46
<.000
1
-
0.076
34
0.055
1
-
0.107
40
0.006
9
1.000
00
0.6779
2
<.0001
0.34530
<.0001
0.285
23
<.000
1
0.034
40
0.387
9
0.057
06
0.151
9
-
0.192
34
<.000
1
0.116
30
0.003
4
0.062
23
0.118
1
0.136
07
0.000
6
0.00437
0.9127
0.063
97
0.108
2
Mushroom_
F_
Mushroom(
F)
0.367
05
<.000
1
0.26480
<.0001
0.267
53
<.000
1
-
0.042
27
0.288
7
-
0.038
80
0.330
1
0.677
92
<.000
1
1.0000
0
0.29445
<.0001
0.232
37
<.000
1
0.150
86
0.000
1
0.282
89
<.000
1
-
0.277
98
<.000
1
0.079
05
0.047
0
0.073
76
0.063
9
0.192
34
<.000
1
-
0.06708
0.0920
0.030
16
0.449
2
Cooked_pota
to_F_
Cooked
potato(F)
0.220
04
<.000
1
0.34806
<.0001
0.209
09
<.000
1
-
0.030
58
0.442
8
-
0.010
38
0.794
6
0.345
30
<.000
1
0.2944
5
<.0001
1.00000
0.236
32
<.000
1
0.127
27
0.001
3
0.163
36
<.000
1
-
0.102
73
0.009
8
0.041
85
0.293
6
0.119
60
0.002
6
0.095
63
0.016
2
0.14112
0.0004
0.136
83
0.000
6
Sourish_F_
Sourish(F
)
0.134
95
0.000
7
0.04619
0.2462
0.337
82
<.000
1
-
0.151
72
0.000
1
-
0.152
30
0.000
1
0.285
23
<.000
1
0.2323
7
<.0001
0.23632
<.0001
1.000
00
0.144
91
0.000
3
0.106
31
0.007
5
-
0.068
30
0.086
2
0.074
26
0.062
1
0.210
89
<.000
1
0.082
56
0.038
0
0.11523
0.0037
0.069
73
0.079
8
Sweet_F_
Sweet(F)
0.024
31
0.541
0.07712
0.0526
0.179
68
<.000
-
0.104
13
-
0.107
37
0.034
40
0.387
0.1508
6
0.0001
0.12727
0.0013
0.144
91
0.000
1.000
00
0.177
77
<.000
-
0.213
34
0.155
78
<.000
0.133
15
0.000
0.015
07
0.705
0.03988
0.3168
0.064
40
0.105
| 95
Pearson Correlation Coefficients, N = 632 Prob > |r| under H0: Rho=0
Earthy_O
_
Cooked_potat
o_O_
Sourish_
O_
Sour_O_ Muddy_O
_
earthy_f_ Mushroom
_F_
Cooked_pota
to_F_
Sourish_
F_
Sweet_F
_
Green_F
_
Muddy_F
_
Flaky_T_ Firm_T_ Juicy_T_ Fibrousnes
s_T_
Oiliness_
T_
9
1
0.008
8
0.006
9
9
3
1
<.000
1
1
8
4
8
Green_F_
Green(F)
0.188
81
<.000
1
0.20805
<.0001
0.128
79
0.001
2
-
0.017
07
0.668
5
0.109
86
0.005
7
0.057
06
0.151
9
0.2828
9
<.0001
0.16336
<.0001
0.106
31
0.007
5
0.177
77
<.000
1
1.000
00
0.044
56
0.263
4
0.115
23
0.003
7
0.082
99
0.037
0
-
0.016
14
0.685
5
-
0.00644
0.8716
-
0.036
64
0.357
8
Muddy_F_
Muddy(F)
-
0.040
83
0.305
4
0.00470
0.9060
-
0.019
01
0.633
4
0.139
06
0.000
5
0.387
89
<.000
1
-
0.192
34
<.000
1
-
0.2779
8
<.0001
-0.10273
0.0098
-
0.068
30
0.086
2
-
0.213
34
<.000
1
0.044
56
0.263
4
1.000
00
-
0.081
07
0.041
6
-
0.019
82
0.618
9
-
0.208
97
<.000
1
0.06834
0.0861
-
0.038
38
0.335
4
Flaky_T_
Flaky(T)
0.085
07
0.032
5
0.21529
<.0001
0.147
49
0.000
2
0.112
48
0.004
6
0.032
35
0.416
9
0.116
30
0.003
4
0.0790
5
0.0470
0.04185
0.2936
0.074
26
0.062
1
0.155
78
<.000
1
0.115
23
0.003
7
-
0.081
07
0.041
6
1.000
00
0.299
51
<.000
1
0.513
14
<.000
1
0.15498
<.0001
0.337
58
<.000
1
Firm_T_
Firm(T)
-
0.008
96
0.822
2
0.10015
0.0118
0.069
69
0.080
0
0.060
16
0.130
9
-
0.085
03
0.032
6
0.062
23
0.118
1
0.0737
6
0.0639
0.11960
0.0026
0.210
89
<.000
1
0.133
15
0.000
8
0.082
99
0.037
0
-
0.019
82
0.618
9
0.299
51
<.000
1
1.000
00
0.105
40
0.008
0
0.52107
<.0001
0.200
16
<.000
1
Juicy_T_
Juicy(T)
0.092
00
0.020
7
0.15635
<.0001
0.092
08
0.020
6
0.005
07
0.898
7
0.003
64
0.927
2
0.136
07
0.000
6
0.1923
4
<.0001
0.09563
0.0162
0.082
56
0.038
0
0.015
07
0.705
4
-
0.016
14
0.685
5
-
0.208
97
<.000
1
0.513
14
<.000
1
0.105
40
0.008
0
1.000
00
-
0.04567
0.2516
0.468
67
<.000
1
Fibrousness_
T_
Fibrousne
ss(T)
-
0.044
03
0.269
0
0.08397
0.0348
0.040
68
0.307
3
0.042
37
0.287
5
-
0.117
68
0.003
0
0.004
37
0.912
7
-
0.0670
8
0.0920
0.14112
0.0004
0.115
23
0.003
7
0.039
88
0.316
8
-
0.006
44
0.871
6
0.068
34
0.086
1
0.154
98
<.000
1
0.521
07
<.000
1
-
0.045
67
0.251
6
1.00000
0.306
56
<.000
1
Oiliness_T_
Oiliness(
T)
0.059
36
0.136
1
0.15416
<.0001
0.095
05
0.016
8
-
0.012
56
0.752
7
0.023
87
0.549
2
0.063
97
0.108
2
0.0301
6
0.4492
0.13683
0.0006
0.069
73
0.079
8
0.064
40
0.105
8
-
0.036
64
0.357
8
-
0.038
38
0.335
4
0.337
58
<.000
1
0.200
16
<.000
1
0.468
67
<.000
1
0.30656
<.0001
1.000
00
96 |
| 97
Appendix D
3-way Mixed Model ANOVA results
Least square mean estimates Product LSM Estimate Lower 95% CI Upper 95% CI StdErr
BLÅ_0 4.335 3.683918 4.986082 0.327247
BLÅ_12a 3.52998 2.823776 4.236184 0.356148
BLÅ_12b 3.456858 2.713481 4.200234 0.375478
BLÅ_3a 4.162077 3.485894 4.838259 0.340448
BLÅ_3b 3.381858 2.638481 4.125234 0.375478
BLÅ_5 3.722562 3.016332 4.428792 0.356161
BLÅ_7 3.78 3.128918 4.431082 0.327247
GRÅ_0a 4.071462 3.365289 4.777636 0.356132
GRÅ_0b 4.201233 3.495026 4.90744 0.356149
GRÅ_12 3.919437 3.213207 4.625667 0.356161
GRÅ_5 4.528956 3.822639 5.235272 0.356205
GRÅ_7 4.0275 3.376418 4.678582 0.327247
GRØN_0 4.4775 3.826418 5.128582 0.327247
GRØN_12 3.211094 2.512981 3.909207 0.352031
GRØN_3 3.480837 2.774664 4.187011 0.356132
GRØN_5 3.558105 2.851901 4.264309 0.356148
GRØN_7 3.957483 3.251276 4.66369 0.356149
GUL_0 3.9525 3.301418 4.603582 0.327247
GUL_12a 3.114 2.370624 3.857377 0.375478
GUL_12b 3.282039 2.576028 3.98805 0.356049
GUL_3a 3.490212 2.784039 4.196386 0.356132
GUL_3b 3.306858 2.563481 4.050234 0.375478
GUL_5 3.84873 3.142526 4.554934 0.356148
GUL_7 3.835608 3.129401 4.541815 0.356149
HVID_0a 4.753743 4.077561 5.429926 0.340448
HVID_0b 3.591414 2.885403 4.297425 0.356049
HVID_12 3.750687 3.044457 4.456917 0.356161
HVID_5 4.650831 3.944514 5.357147 0.356205
HVID_7 4.4925 3.841418 5.143582 0.327247
JIS_3 3.357081 2.650764 4.063397 0.356205
RØD_0 4.3125 3.661418 4.963582 0.327247
RØD_12 3.62373 2.917526 4.329934 0.356148
98 |
RØD_3 3.887077 3.210894 4.563259 0.340448
RØD_5 3.806937 3.100707 4.513167 0.356161
RØD_7 3.645 2.993918 4.296082 0.327247
GRÅ_3 4.564076 3.820745 5.307406 0.375455
HVID_3 4.135504 3.392174 4.878835 0.375455
SORT 3.364076 2.620745 4.107406 0.375455
Pair wise comparisons using tukey adjustment Pair wise comparisons between products. In order to prevent inflation of the type 1 error, Tukey
adjustment has been applied i.e. adjusted 95% confidence intervals are given together with the adjusted
p-values. The unadjusted p-values are shown for the complete picture.
Product A Product B Est difference Adj Lower CI
Adj Upper
CI Adj p-value Unadj p-value
BLÅ_0 BLÅ_12a 0.805019939 -0.819249928 2.429289806 0.995758053 0.05379515
BLÅ_0 BLÅ_12b 0.878142419 -0.811314209 2.567599047 0.990698139 0.043197553
BLÅ_0 BLÅ_3a 0.172923287 -1.399085391 1.744931965 1 0.667610278
BLÅ_0 BLÅ_3b 0.953142419 -0.736314209 2.642599047 0.969081612 0.028307491
BLÅ_0 BLÅ_5 0.612437857 -1.011876442 2.236752157 0.9999851 0.1417707
BLÅ_0 BLÅ_7 0.555 -0.973590881 2.083590881 0.99999404 0.157063631
BLÅ_0 GRÅ_0a 0.263537535 -1.360680836 1.887755907 1 0.526542295
BLÅ_0 GRÅ_0b 0.133766531 -1.490508195 1.758041258 1 0.747812603
BLÅ_0 GRÅ_12 0.415562857 -1.208751442 2.039877157 1 0.318276213
BLÅ_0 GRÅ_5 -0.193955607 -1.818417558 1.430506345 1 0.641148132
BLÅ_0 GRÅ_7 0.3075 -1.221090881 1.836090881 1 0.432449694
BLÅ_0 GRØN_0 -0.1425 -1.671090881 1.386090881 1 0.715897501
BLÅ_0 GRØN_12 1.123905898 -0.486599819 2.734411616 0.719203821 0.006777259
BLÅ_0 GRØN_3 0.854162535 -0.770055836 2.478380907 0.98882841 0.040816709
BLÅ_0 GRØN_5 0.776894939 -0.847374928 2.401164806 0.997735197 0.062667066
BLÅ_0 GRØN_7 0.377516531 -1.246758195 2.001791258 1 0.364510635
BLÅ_0 GUL_0 0.3825 -1.146090881 1.911090881 1 0.328982138
BLÅ_0 GUL_12a 1.220999562 -0.468457066 2.91045619 0.642857466 0.005088986
BLÅ_0 GUL_12b 1.05296078 -0.570980201 2.676901762 0.848202642 0.011842139
BLÅ_0 GUL_3a 0.844787535 -0.779430836 2.469005907 0.990601005 0.043061983
BLÅ_0 GUL_3b 1.028142419 -0.661314209 2.717599047 0.921136648 0.018086363
BLÅ_0 GUL_5 0.486269939 -1.137999928 2.110539806 0.999999964 0.243029124
BLÅ_0 GUL_7 0.499391531 -1.124883195 2.123666258 0.999999924 0.230580378
BLÅ_0 HVID_0a -0.418743379 -1.990752057 1.153265298 0.999999999 0.298793364
BLÅ_0 HVID_0b 0.74358578 -0.880355201 2.367526762 0.998998553 0.074656304
BLÅ_0 HVID_12 0.584312857 -1.040001442 2.208627157 0.999995263 0.160912556
BLÅ_0 HVID_5 -0.315830607 -1.940292558 1.308631345 1 0.448020656
BLÅ_0 HVID_7 -0.1575 -1.686090881 1.371090881 1 0.687502719
BLÅ_0 JIS_3 0.977919393 -0.646542558 2.602381345 0.930321789 0.019356859
BLÅ_0 RØD_0 0.0225 -1.506090881 1.551090881 1 0.954169027
BLÅ_0 RØD_12 0.711269939 -0.912999928 2.335539806 0.99958719 0.088160073
BLÅ_0 RØD_3 0.447923287 -1.124085391 2.019931965 0.999999991 0.266453059
| 99
BLÅ_0 RØD_5 0.528062857 -1.096251442 2.152377157 0.999999653 0.2049891
BLÅ_0 RØD_7 0.69 -0.838590881 2.218590881 0.999241643 0.078844094
BLÅ_0 GRÅ_3 -0.229075843 -1.918453632 1.460301946 1 0.596592068
BLÅ_0 HVID_3 0.199495585 -1.489882204 1.888873374 1 0.644807371
BLÅ_0 SORT 0.970924157 -0.718453632 2.660301946 0.960513729 0.025506731
BLÅ_12a BLÅ_12b 0.07312248 -1.706132919 1.852377879 1 0.872517427
BLÅ_12a BLÅ_3a -0.632096652 -2.298061078 1.033867775 0.999982697 0.139288916
BLÅ_12a BLÅ_3b 0.14812248 -1.631132919 1.927377879 1 0.745166458
BLÅ_12a BLÅ_5 -0.192582082 -1.908833232 1.523669069 1 0.661347986
BLÅ_12a BLÅ_7 -0.250019939 -1.874289806 1.374249928 1 0.547964569
BLÅ_12a GRÅ_0a -0.541482404 -2.257517536 1.174552729 0.999999845 0.218578634
BLÅ_12a GRÅ_0b -0.671253408 -2.387444278 1.044937463 0.999964736 0.127536251
BLÅ_12a GRÅ_12 -0.389457082 -2.105708232 1.326794069 1 0.375949548
BLÅ_12a GRÅ_5 -0.998975546 -2.711794584 0.713843492 0.952398352 0.023426423
BLÅ_12a GRÅ_7 -0.497519939 -2.121789806 1.126749928 0.999999932 0.232326389
BLÅ_12a GRØN_0 -0.947519939 -2.571789806 0.676749928 0.952279329 0.023399164
BLÅ_12a GRØN_12 0.318885959 -1.382565384 2.020337302 1 0.464485699
BLÅ_12a GRØN_3 0.049142596 -1.666892536 1.765177729 1 0.910971005
BLÅ_12a GRØN_5 -0.028125 -1.73714156 1.68089156 1 0.948766502
BLÅ_12a GRØN_7 -0.427503408 -2.143694278 1.288687463 1 0.331166377
BLÅ_12a GUL_0 -0.422519939 -2.046789806 1.201749928 0.999999999 0.310243849
BLÅ_12a GUL_12a 0.415979623 -1.363275776 2.195235022 1 0.361688118
BLÅ_12a GUL_12b 0.247940841 -1.46770765 1.963589333 1 0.572682455
BLÅ_12a GUL_3a 0.039767596 -1.676267536 1.755802729 1 0.927903138
BLÅ_12a GUL_3b 0.22312248 -1.556132919 2.002377879 1 0.624467912
BLÅ_12a GUL_5 -0.31875 -2.02776656 1.39026656 1 0.466688611
BLÅ_12a GUL_7 -0.305628408 -2.021819278 1.410562463 1 0.487037792
BLÅ_12a HVID_0a -1.223763318 -2.889727745 0.442201108 0.605004249 0.004418855
BLÅ_12a HVID_0b -0.061434159 -1.77708265 1.654214333 1 0.888809056
BLÅ_12a HVID_12 -0.220707082 -1.936958232 1.495544069 1 0.615668376
BLÅ_12a HVID_5 -1.120850546 -2.833669584 0.591968492 0.834682245 0.011091614
BLÅ_12a HVID_7 -0.962519939 -2.586789806 0.661749928 0.942143337 0.021313099
BLÅ_12a JIS_3 0.172899454 -1.539919584 1.885718492 1 0.693519549
BLÅ_12a RØD_0 -0.782519939 -2.406789806 0.841749928 0.997420564 0.060801349
BLÅ_12a RØD_12 -0.09375 -1.80276656 1.61526656 1 0.830408343
BLÅ_12a RØD_3 -0.357096652 -2.023061078 1.308867775 1 0.402937857
BLÅ_12a RØD_5 -0.276957082 -1.993208232 1.439294069 1 0.52878947
BLÅ_12a RØD_7 -0.115019939 -1.739289806 1.509249928 1 0.782184896
BLÅ_12a GRÅ_3 -1.034095782 -2.805264552 0.737072988 0.951768328 0.023282408
BLÅ_12a HVID_3 -0.605524354 -2.376693124 1.165644416 0.999998697 0.182639323
BLÅ_12a SORT 0.165904218 -1.605264552 1.937072988 1 0.714597172
BLÅ_12b BLÅ_3a -0.705219132 -2.431363795 1.020925531 0.999905599 0.111507903
BLÅ_12b BLÅ_3b 0.075 -1.752015553 1.902015553 1 0.872661256
BLÅ_12b BLÅ_5 -0.265704562 -2.040305007 1.508895884 1 0.558933917
BLÅ_12b BLÅ_7 -0.323142419 -2.012599047 1.366314209 1 0.4554058
BLÅ_12b GRÅ_0a -0.614604884 -2.389227471 1.160017704 0.999998182 0.17702598
BLÅ_12b GRÅ_0b -0.744375888 -2.518915645 1.03016387 0.999832264 0.102301172
BLÅ_12b GRÅ_12 -0.462579562 -2.237180007 1.312020884 0.999999999 0.30924625
100 |
BLÅ_12b GRÅ_5 -1.072098026 -2.847113987 0.702917936 0.92761084 0.018962107
BLÅ_12b GRÅ_7 -0.570642419 -2.260099047 1.118814209 0.999999048 0.187939522
BLÅ_12b GRØN_0 -1.020642419 -2.710099047 0.668814209 0.927430263 0.018936563
BLÅ_12b GRØN_12 0.245763479 -1.518889364 2.010416323 1 0.586669061
BLÅ_12b GRØN_3 -0.023979884 -1.798602471 1.750642704 1 0.957922771
BLÅ_12b GRØN_5 -0.10124748 -1.880502879 1.678007919 1 0.824177704
BLÅ_12b GRØN_7 -0.500625888 -2.275165645 1.27391387 0.999999993 0.271204815
BLÅ_12b GUL_0 -0.495642419 -2.185099047 1.193814209 0.99999998 0.252584042
BLÅ_12b GUL_12a 0.342857143 -1.48415841 2.169872696 1 0.463950882
BLÅ_12b GUL_12b 0.174818362 -1.60017846 1.949815183 1 0.700606472
BLÅ_12b GUL_3a -0.033354884 -1.807977471 1.741267704 1 0.941498069
BLÅ_12b GUL_3b 0.15 -1.677015553 1.977015553 1 0.748562676
BLÅ_12b GUL_5 -0.39187248 -2.171127879 1.387382919 1 0.390138674
BLÅ_12b GUL_7 -0.378750888 -2.153290645 1.39578887 1 0.404940034
BLÅ_12b HVID_0a -1.296885798 -3.023030461 0.429258865 0.55084816 0.003608392
BLÅ_12b HVID_0b -0.134556638 -1.90955346 1.640440183 1 0.76725654
BLÅ_12b HVID_12 -0.293829562 -2.068430007 1.480770884 1 0.518122679
BLÅ_12b HVID_5 -1.193973026 -2.968988987 0.581042936 0.789634863 0.009058973
BLÅ_12b HVID_7 -1.035642419 -2.725099047 0.653814209 0.914481877 0.017269961
BLÅ_12b JIS_3 0.099776974 -1.675238987 1.874792936 1 0.82628311
BLÅ_12b RØD_0 -0.855642419 -2.545099047 0.833814209 0.993890563 0.048796197
BLÅ_12b RØD_12 -0.16687248 -1.946127879 1.612382919 1 0.714251624
BLÅ_12b RØD_3 -0.430219132 -2.156363795 1.295925531 1 0.330901914
BLÅ_12b RØD_5 -0.350079562 -2.124680007 1.424520884 1 0.44139966
BLÅ_12b RØD_7 -0.188142419 -1.877599047 1.501314209 1 0.663749364
BLÅ_12b GRÅ_3 -1.107218262 -2.943227112 0.728790587 0.928890439 0.019145628
BLÅ_12b HVID_3 -0.678646834 -2.514655683 1.157362016 0.999990765 0.149736696
BLÅ_12b SORT 0.092781738 -1.743227112 1.928790587 1 0.843587857
BLÅ_3a BLÅ_3b 0.780219132 -0.945925531 2.506363795 0.99922144 0.078445213
BLÅ_3a BLÅ_5 0.43951457 -1.226549557 2.105578698 0.999999999 0.303470802
BLÅ_3a BLÅ_7 0.382076713 -1.189931965 1.954085391 1 0.343026243
BLÅ_3a GRÅ_0a 0.090614248 -1.571587267 1.752815764 1 0.831448127
BLÅ_3a GRÅ_0b -0.039156756 -1.705169838 1.626856327 1 0.926881994
BLÅ_3a GRÅ_12 0.24263957 -1.423424557 1.908703698 1 0.569715215
BLÅ_3a GRÅ_5 -0.366878894 -2.033132645 1.299374858 1 0.390275484
BLÅ_3a GRÅ_7 0.134576713 -1.437431965 1.706585391 1 0.738210203
BLÅ_3a GRØN_0 -0.315423287 -1.887431965 1.256585391 1 0.433629193
BLÅ_3a GRØN_12 0.950982611 -0.701395143 2.603360365 0.959797588 0.025255986
BLÅ_3a GRØN_3 0.681239248 -0.980962267 2.343440764 0.999898801 0.110388299
BLÅ_3a GRØN_5 0.603971652 -1.061992775 2.269936078 0.999994257 0.157682221
BLÅ_3a GRØN_7 0.204593244 -1.461419838 1.870606327 1 0.631661288
BLÅ_3a GUL_0 0.209576713 -1.362431965 1.781585391 1 0.60277659
BLÅ_3a GUL_12a 1.048076275 -0.678068388 2.774220938 0.923140716 0.018348429
BLÅ_3a GUL_12b 0.880037493 -0.785519783 2.54559477 0.987986245 0.039881706
BLÅ_3a GUL_3a 0.671864248 -0.990337267 2.334065764 0.999925583 0.115352962
BLÅ_3a GUL_3b 0.855219132 -0.870925531 2.581363795 0.995782833 0.053876292
BLÅ_3a GUL_5 0.313346652 -1.352617775 1.979311078 1 0.462933583
BLÅ_3a GUL_7 0.326468244 -1.339544838 1.992481327 1 0.444445638
| 101
BLÅ_3a HVID_0a -0.591666667 -2.202942932 1.019609598 0.999992135 0.152421845
BLÅ_3a HVID_0b 0.570662493 -1.094894783 2.23621977 0.999998621 0.18168754
BLÅ_3a HVID_12 0.41138957 -1.254674557 2.077453698 1 0.335403017
BLÅ_3a HVID_5 -0.488753894 -2.155007645 1.177499858 0.99999998 0.252662699
BLÅ_3a HVID_7 -0.330423287 -1.902431965 1.241585391 1 0.412109776
BLÅ_3a JIS_3 0.804996106 -0.861257645 2.471249858 0.997288086 0.060085955
BLÅ_3a RØD_0 -0.150423287 -1.722431965 1.421585391 1 0.708724168
BLÅ_3a RØD_12 0.538346652 -1.127617775 2.204311078 0.999999705 0.207717089
BLÅ_3a RØD_3 0.275 -1.336276265 1.886276265 1 0.505335094
BLÅ_3a RØD_5 0.35513957 -1.310924557 2.021203698 1 0.405545409
BLÅ_3a RØD_7 0.517076713 -1.054931965 2.089085391 0.999999526 0.199721724
BLÅ_3a GRÅ_3 -0.40199913 -2.128028736 1.324030475 1 0.363517567
BLÅ_3a HVID_3 0.026572298 -1.699457308 1.752601904 1 0.952067903
BLÅ_3a SORT 0.79800087 -0.928028736 2.524030475 0.998795033 0.07189738
BLÅ_3b BLÅ_5 -0.340704562 -2.115305007 1.433895884 1 0.453713907
BLÅ_3b BLÅ_7 -0.398142419 -2.087599047 1.291314209 1 0.357870109
BLÅ_3b GRÅ_0a -0.689604884 -2.464227471 1.085017704 0.99996967 0.13001472
BLÅ_3b GRÅ_0b -0.819375888 -2.593915645 0.95516387 0.998824147 0.072260399
BLÅ_3b GRÅ_12 -0.537579562 -2.312180007 1.237020884 0.99999995 0.237496593
BLÅ_3b GRÅ_5 -1.147098026 -2.922113987 0.627917936 0.852900454 0.012122075
BLÅ_3b GRÅ_7 -0.645642419 -2.335099047 1.043814209 0.999979482 0.136465096
BLÅ_3b GRØN_0 -1.095642419 -2.785099047 0.593814209 0.847939924 0.01182645
BLÅ_3b GRØN_12 0.170763479 -1.593889364 1.935416323 1 0.705573436
BLÅ_3b GRØN_3 -0.098979884 -1.873602471 1.675642704 1 0.827611311
BLÅ_3b GRØN_5 -0.17624748 -1.955502879 1.603007919 1 0.698965623
BLÅ_3b GRØN_7 -0.575625888 -2.350165645 1.19891387 0.999999673 0.205988565
BLÅ_3b GUL_0 -0.570642419 -2.260099047 1.118814209 0.999999048 0.187939522
BLÅ_3b GUL_12a 0.267857143 -1.55915841 2.094872696 1 0.567142646
BLÅ_3b GUL_12b 0.099818362 -1.67517846 1.874815183 1 0.826210369
BLÅ_3b GUL_3a -0.108354884 -1.882977471 1.666267704 1 0.81157857
BLÅ_3b GUL_3b 0.075 -1.752015553 1.902015553 1 0.872661256
BLÅ_3b GUL_5 -0.46687248 -2.246127879 1.312382919 0.999999999 0.306050041
BLÅ_3b GUL_7 -0.453750888 -2.228290645 1.32078887 1 0.318537123
BLÅ_3b HVID_0a -1.371885798 -3.098030461 0.354258865 0.413580281 0.002098105
BLÅ_3b HVID_0b -0.209556638 -1.98455346 1.565440183 1 0.644885456
BLÅ_3b HVID_12 -0.368829562 -2.143430007 1.405770884 1 0.417357386
BLÅ_3b HVID_5 -1.268973026 -3.043988987 0.506042936 0.667385437 0.005579706
BLÅ_3b HVID_7 -1.110642419 -2.800099047 0.578814209 0.82765566 0.010731153
BLÅ_3b JIS_3 0.024776974 -1.750238987 1.799792936 1 0.956535115
BLÅ_3b RØD_0 -0.930642419 -2.620099047 0.758814209 0.977774295 0.032219755
BLÅ_3b RØD_12 -0.24187248 -2.021127879 1.537382919 1 0.595665033
BLÅ_3b RØD_3 -0.505219132 -2.231363795 1.220925531 0.999999981 0.253695035
BLÅ_3b RØD_5 -0.425079562 -2.199680007 1.349520884 1 0.350038427
BLÅ_3b RØD_7 -0.263142419 -1.952599047 1.426314209 1 0.54322476
BLÅ_3b GRÅ_3 -1.182218262 -3.018227112 0.653790587 0.857912675 0.01243328
BLÅ_3b HVID_3 -0.753646834 -2.589655683 1.082362016 0.999895269 0.109834353
BLÅ_3b SORT 0.017781738 -1.818227112 1.853790587 1 0.969834933
BLÅ_5 BLÅ_7 -0.057437857 -1.681752157 1.566876442 1 0.890187721
102 |
BLÅ_5 GRÅ_0a -0.348900322 -2.065065788 1.367265144 1 0.427581873
BLÅ_5 GRÅ_0b -0.478671326 -2.190607363 1.23326471 0.999999995 0.275478811
BLÅ_5 GRÅ_12 -0.196875 -1.90589156 1.51214156 1 0.652920844
BLÅ_5 GRÅ_5 -0.806393464 -2.518710913 0.905923985 0.998296205 0.066794734
BLÅ_5 GRÅ_7 -0.304937857 -1.929252157 1.319376442 1 0.46377535
BLÅ_5 GRØN_0 -0.754937857 -2.379252157 0.869376442 0.998668563 0.070418679
BLÅ_5 GRØN_12 0.511468041 -1.191351089 2.214287171 0.99999996 0.241422077
BLÅ_5 GRØN_3 0.241724678 -1.474440788 1.957890144 1 0.582453041
BLÅ_5 GRØN_5 0.164457082 -1.551794069 1.880708232 1 0.708332219
BLÅ_5 GRØN_7 -0.234921326 -1.946857363 1.47701471 1 0.592195
BLÅ_5 GUL_0 -0.229937857 -1.854252157 1.394376442 1 0.580559289
BLÅ_5 GUL_12a 0.608561704 -1.166038741 2.383162149 0.999998589 0.181302361
BLÅ_5 GUL_12b 0.440522923 -1.275156661 2.156202507 1 0.316534188
BLÅ_5 GUL_3a 0.232349678 -1.483815788 1.948515144 1 0.597157297
BLÅ_5 GUL_3b 0.415704562 -1.358895884 2.190305007 1 0.360750163
BLÅ_5 GUL_5 -0.126167918 -1.842419069 1.590083232 1 0.774101303
BLÅ_5 GUL_7 -0.113046326 -1.824982363 1.59888971 1 0.796548653
BLÅ_5 HVID_0a -1.031181237 -2.697245365 0.634882891 0.905022393 0.016230265
BLÅ_5 HVID_0b 0.131147923 -1.584531661 1.846827507 1 0.765366847
BLÅ_5 HVID_12 -0.028125 -1.73714156 1.68089156 1 0.948766502
BLÅ_5 HVID_5 -0.928268464 -2.640585913 0.784048985 0.982345661 0.035033903
BLÅ_5 HVID_7 -0.769937857 -2.394252157 0.854376442 0.998079034 0.065047497
BLÅ_5 JIS_3 0.365481536 -1.346835913 2.077798985 1 0.404926065
BLÅ_5 RØD_0 -0.589937857 -2.214252157 1.034376442 0.999993994 0.156934877
BLÅ_5 RØD_12 0.098832082 -1.617419069 1.815083232 1 0.822107002
BLÅ_5 RØD_3 -0.16451457 -1.830578698 1.501549557 1 0.699868984
BLÅ_5 RØD_5 -0.084375 -1.79339156 1.62464156 1 0.847145635
BLÅ_5 RØD_7 0.077562143 -1.546752157 1.701876442 1 0.852101404
BLÅ_5 GRÅ_3 -0.841513701 -2.620721446 0.937694045 0.998154899 0.065631513
BLÅ_5 HVID_3 -0.412942272 -2.192150018 1.366265474 1 0.365186167
BLÅ_5 SORT 0.358486299 -1.420721446 2.137694045 1 0.431715401
BLÅ_7 GRÅ_0a -0.291462465 -1.915680836 1.332755907 1 0.483715121
BLÅ_7 GRÅ_0b -0.421233469 -2.045508195 1.203041258 0.999999999 0.311717961
BLÅ_7 GRÅ_12 -0.139437143 -1.763751442 1.484877157 1 0.737516423
BLÅ_7 GRÅ_5 -0.748955607 -2.373417558 0.875506345 0.998857302 0.072686273
BLÅ_7 GRÅ_7 -0.2475 -1.776090881 1.281090881 1 0.527411756
BLÅ_7 GRØN_0 -0.6975 -2.226090881 0.831090881 0.99906355 0.075663026
BLÅ_7 GRØN_12 0.568905898 -1.041599819 2.179411616 0.999996995 0.168492261
BLÅ_7 GRØN_3 0.299162535 -1.325055836 1.923380907 1 0.472249131
BLÅ_7 GRØN_5 0.221894939 -1.402374928 1.846164806 1 0.593850274
BLÅ_7 GRØN_7 -0.177483469 -1.801758195 1.446791258 1 0.66969031
BLÅ_7 GUL_0 -0.1725 -1.701090881 1.356090881 1 0.659544355
BLÅ_7 GUL_12a 0.665999562 -1.023457066 2.35545619 0.999957765 0.124579178
BLÅ_7 GUL_12b 0.49796078 -1.125980201 2.121901762 0.99999993 0.231819811
BLÅ_7 GUL_3a 0.289787535 -1.334430836 1.914005907 1 0.486229181
BLÅ_7 GUL_3b 0.473142419 -1.216314209 2.162599047 0.999999995 0.274711767
BLÅ_7 GUL_5 -0.068730061 -1.692999928 1.555539806 1 0.868775821
BLÅ_7 GUL_7 -0.055608469 -1.679883195 1.568666258 1 0.893661414
| 103
BLÅ_7 HVID_0a -0.973743379 -2.545752057 0.598265298 0.904205034 0.016146788
BLÅ_7 HVID_0b 0.18858578 -1.435355201 1.812526762 1 0.650303841
BLÅ_7 HVID_12 0.029312857 -1.595001442 1.653627157 1 0.943826007
BLÅ_7 HVID_5 -0.870830607 -2.495292558 0.753631345 0.985041008 0.037098893
BLÅ_7 HVID_7 -0.7125 -2.241090881 0.816090881 0.998594164 0.069616693
BLÅ_7 JIS_3 0.422919393 -1.201542558 2.047381345 0.999999999 0.309844563
BLÅ_7 RØD_0 -0.5325 -2.061090881 0.996090881 0.999997889 0.174510897
BLÅ_7 RØD_12 0.156269939 -1.467999928 1.780539806 1 0.707212246
BLÅ_7 RØD_3 -0.107076713 -1.679085391 1.464931965 1 0.790287219
BLÅ_7 RØD_5 -0.026937143 -1.651251442 1.597377157 1 0.948372056
BLÅ_7 RØD_7 0.135 -1.393590881 1.663590881 1 0.730247476
BLÅ_7 GRÅ_3 -0.784075843 -2.473453632 0.905301946 0.998703454 0.070811016
BLÅ_7 HVID_3 -0.355504415 -2.044882204 1.333873374 1 0.411568928
BLÅ_7 SORT 0.415924157 -1.273453632 2.105301946 1 0.336818468
GRÅ_0a GRÅ_0b -0.129771004 -1.845876295 1.586334286 1 0.767814623
GRÅ_0a GRÅ_12 0.152025322 -1.564140144 1.868190788 1 0.729461703
GRÅ_0a GRÅ_5 -0.457493142 -2.173920716 1.258934432 0.999999999 0.298496756
GRÅ_0a GRÅ_7 0.043962465 -1.580255907 1.668180836 1 0.915834573
GRÅ_0a GRØN_0 -0.406037535 -2.030255907 1.218180836 1 0.329446722
GRÅ_0a GRØN_12 0.860368363 -0.842292297 2.563029023 0.994117076 0.049249834
GRÅ_0a GRØN_3 0.590625 -1.11839156 2.29964156 0.999998278 0.177946728
GRÅ_0a GRØN_5 0.513357404 -1.202677729 2.229392536 0.999999965 0.243380699
GRÅ_0a GRØN_7 0.113978996 -1.602126295 1.830084286 1 0.795394558
GRÅ_0a GUL_0 0.118962465 -1.505255907 1.743180836 1 0.77491095
GRÅ_0a GUL_12a 0.957462026 -0.817160562 2.732084614 0.983542071 0.035904122
GRÅ_0a GUL_12b 0.789423245 -0.926114221 2.504960711 0.998898653 0.073234737
GRÅ_0a GUL_3a 0.58125 -1.12776656 2.29026656 0.999998861 0.184908713
GRÅ_0a GUL_3b 0.764604884 -1.010017704 2.539227471 0.999704562 0.093368115
GRÅ_0a GUL_5 0.222732404 -1.493302729 1.938767536 1 0.612388567
GRÅ_0a GUL_7 0.235853996 -1.480251295 1.951959286 1 0.591628094
GRÅ_0a HVID_0a -0.682280915 -2.344482431 0.979920601 0.999895341 0.109847308
GRÅ_0a HVID_0b 0.480048245 -1.235489221 2.195585711 0.999999995 0.275108763
GRÅ_0a HVID_12 0.320775322 -1.395390144 2.036940788 1 0.465725808
GRÅ_0a HVID_5 -0.579368142 -2.295795716 1.137059432 0.999999064 0.188230758
GRÅ_0a HVID_7 -0.421037535 -2.045255907 1.203180836 0.999999999 0.311925928
GRÅ_0a JIS_3 0.714381858 -1.002045716 2.430809432 0.999858333 0.104992597
GRÅ_0a RØD_0 -0.241037535 -1.865255907 1.383180836 1 0.562413933
GRÅ_0a RØD_12 0.447732404 -1.268302729 2.163767536 0.999999999 0.308793944
GRÅ_0a RØD_3 0.184385752 -1.477815764 1.846587267 1 0.664979739
GRÅ_0a RØD_5 0.264525322 -1.451640144 1.980690788 1 0.547419053
GRÅ_0a RØD_7 0.426462465 -1.197755907 2.050680836 0.999999999 0.305744237
GRÅ_0a GRÅ_3 -0.492613379 -2.267056712 1.281829954 0.999999996 0.278906413
GRÅ_0a HVID_3 -0.06404195 -1.838485283 1.710401383 1 0.887935624
GRÅ_0a SORT 0.707386621 -1.067056712 2.481829954 0.999945475 0.120406931
GRÅ_0b GRÅ_12 0.281796326 -1.43013971 1.993732363 1 0.520571137
GRÅ_0b GRÅ_5 -0.327722138 -2.03999187 1.384547594 1 0.455108835
GRÅ_0b GRÅ_7 0.173733469 -1.450541258 1.798008195 1 0.676266595
GRÅ_0b GRØN_0 -0.276266531 -1.900541258 1.348008195 1 0.50679657
104 |
GRÅ_0b GRØN_12 0.990139367 -0.712625061 2.692903795 0.954154516 0.02379523
GRÅ_0b GRØN_3 0.720396004 -0.995709286 2.436501295 0.999829584 0.102048603
GRÅ_0b GRØN_5 0.643128408 -1.073062463 2.359319278 0.999987133 0.144208376
GRÅ_0b GRØN_7 0.24375 -1.46526656 1.95276656 1 0.577714725
GRÅ_0b GUL_0 0.248733469 -1.375541258 1.873008195 1 0.5500256
GRÅ_0b GUL_12a 1.08723303 -0.687306727 2.861772788 0.914971834 0.017327493
GRÅ_0b GUL_12b 0.919194249 -0.796428499 2.634816997 0.985163451 0.037201638
GRÅ_0b GUL_3a 0.711021004 -1.005084286 2.427126295 0.999871598 0.106564767
GRÅ_0b GUL_3b 0.894375888 -0.88016387 2.668915645 0.994364899 0.04989278
GRÅ_0b GUL_5 0.352503408 -1.363687463 2.068694278 1 0.422837838
GRÅ_0b GUL_7 0.365625 -1.34339156 2.07464156 1 0.40383673
GRÅ_0b HVID_0a -0.552509911 -2.218522993 1.113503172 0.999999411 0.196054492
GRÅ_0b HVID_0b 0.609819249 -1.105803499 2.325441997 0.999996486 0.165931123
GRÅ_0b HVID_12 0.450546326 -1.26138971 2.162482363 0.999999999 0.304615676
GRÅ_0b HVID_5 -0.449597138 -2.16186687 1.262672594 0.999999999 0.305727614
GRÅ_0b HVID_7 -0.291266531 -1.915541258 1.333008195 1 0.484023984
GRÅ_0b JIS_3 0.844152862 -0.86811687 2.556422594 0.996127981 0.055064434
GRÅ_0b RØD_0 -0.111266531 -1.735541258 1.513008195 1 0.789121753
GRÅ_0b RØD_12 0.577503408 -1.138687463 2.293694278 0.999999137 0.189591142
GRÅ_0b RØD_3 0.314156756 -1.351856327 1.980169838 1 0.461792042
GRÅ_0b RØD_5 0.394296326 -1.31763971 2.106232363 1 0.368853196
GRÅ_0b RØD_7 0.556233469 -1.068041258 2.180508195 0.999998639 0.181910635
GRÅ_0b GRÅ_3 -0.362842375 -2.141973821 1.416289072 1 0.426127005
GRÅ_0b HVID_3 0.065729054 -1.713402393 1.844860501 1 0.885304599
GRÅ_0b SORT 0.837157625 -0.941973821 2.616289072 0.998322461 0.067020048
GRÅ_12 GRÅ_5 -0.609518464 -2.321835913 1.102798985 0.999996358 0.16532688
GRÅ_12 GRÅ_7 -0.108062857 -1.732377157 1.516251442 1 0.795060304
GRÅ_12 GRØN_0 -0.558062857 -2.182377157 1.066251442 0.99999852 0.180493033
GRÅ_12 GRØN_12 0.708343041 -0.994476089 2.411162171 0.999859948 0.105109665
GRÅ_12 GRØN_3 0.438599678 -1.277565788 2.154765144 1 0.318784322
GRÅ_12 GRØN_5 0.361332082 -1.354919069 2.077583232 1 0.411345419
GRÅ_12 GRØN_7 -0.038046326 -1.749982363 1.67388971 1 0.930851018
GRÅ_12 GUL_0 -0.033062857 -1.657377157 1.591251442 1 0.936653996
GRÅ_12 GUL_12a 0.805436704 -0.969163741 2.580037149 0.999156006 0.077225956
GRÅ_12 GUL_12b 0.637397923 -1.078281661 2.353077507 0.999989555 0.147682761
GRÅ_12 GUL_3a 0.429224678 -1.286940788 2.145390144 1 0.329219431
GRÅ_12 GUL_3b 0.612579562 -1.162020884 2.387180007 0.999998329 0.178447085
GRÅ_12 GUL_5 0.070707082 -1.645544069 1.786958232 1 0.872205902
GRÅ_12 GUL_7 0.083828674 -1.628107363 1.79576471 1 0.848378989
GRÅ_12 HVID_0a -0.834306237 -2.500370365 0.831757891 0.994942987 0.051370994
GRÅ_12 HVID_0b 0.328022923 -1.387656661 2.043702507 1 0.455591126
GRÅ_12 HVID_12 0.16875 -1.54026656 1.87776656 1 0.699879304
GRÅ_12 HVID_5 -0.731393464 -2.443710913 0.980923985 0.999753574 0.096215666
GRÅ_12 HVID_7 -0.573062857 -2.197377157 1.051251442 0.999997089 0.169097015
GRÅ_12 JIS_3 0.562356536 -1.149960913 2.274673985 0.999999545 0.200416012
GRÅ_12 RØD_0 -0.393062857 -2.017377157 1.231251442 1 0.345137531
GRÅ_12 RØD_12 0.295707082 -1.420544069 2.011958232 1 0.501289905
GRÅ_12 RØD_3 0.03236043 -1.633703698 1.698424557 1 0.939547708
| 105
GRÅ_12 RØD_5 0.1125 -1.59651656 1.82151656 1 0.797172103
GRÅ_12 RØD_7 0.274437143 -1.349877157 1.898751442 1 0.509622434
GRÅ_12 GRÅ_3 -0.644638701 -2.423846446 1.134569045 0.999994341 0.157930038
GRÅ_12 HVID_3 -0.216067272 -1.995275018 1.563140474 1 0.635451262
GRÅ_12 SORT 0.555361299 -1.223846446 2.334569045 0.999999885 0.223571417
GRÅ_5 GRÅ_7 0.501455607 -1.123006345 2.125917558 0.999999915 0.228717605
GRÅ_5 GRØN_0 0.051455607 -1.573006345 1.675917558 1 0.90157182
GRÅ_5 GRØN_12 1.317861505 -0.385194502 3.020917511 0.478564236 0.00271895
GRÅ_5 GRØN_3 1.048118142 -0.668309432 2.764545716 0.918070847 0.017700209
GRÅ_5 GRØN_5 0.970850546 -0.741968492 2.683669584 0.967082624 0.027582109
GRÅ_5 GRØN_7 0.571472138 -1.140797594 2.28374187 0.999999304 0.193226608
GRÅ_5 GUL_0 0.576455607 -1.048006345 2.200917558 0.99999663 0.166634699
GRÅ_5 GUL_12a 1.414955168 -0.360060793 3.18997113 0.406406097 0.00203515
GRÅ_5 GUL_12b 1.246916387 -0.46905951 2.962892284 0.630361497 0.004856786
GRÅ_5 GUL_3a 1.038743142 -0.677684432 2.755170716 0.925970202 0.018731471
GRÅ_5 GUL_3b 1.222098026 -0.552917936 2.997113987 0.746342852 0.007573846
GRÅ_5 GUL_5 0.680225546 -1.032593492 2.393044584 0.99994999 0.12181687
GRÅ_5 GUL_7 0.693347138 -1.018922594 2.40561687 0.999922536 0.114702608
GRÅ_5 HVID_0a -0.224787773 -1.891041524 1.441465978 1 0.598465012
GRÅ_5 HVID_0b 0.937541387 -0.77843451 2.653517284 0.980227769 0.033640344
GRÅ_5 HVID_12 0.778268464 -0.934048985 2.490585913 0.999132412 0.076810935
GRÅ_5 HVID_5 -0.121875 -1.83089156 1.58714156 1 0.780687619
GRÅ_5 HVID_7 0.036455607 -1.588006345 1.660917558 1 0.930176002
GRÅ_5 JIS_3 1.171875 -0.53714156 2.88089156 0.754244417 0.007819587
GRÅ_5 RØD_0 0.216455607 -1.408006345 1.840917558 1 0.602967352
GRÅ_5 RØD_12 0.905225546 -0.807593492 2.618044584 0.987942094 0.039834805
GRÅ_5 RØD_3 0.641878894 -1.024374858 2.308132645 0.999975246 0.133363164
GRÅ_5 RØD_5 0.722018464 -0.990298985 2.434335913 0.999812519 0.100534434
GRÅ_5 RØD_7 0.883955607 -0.740506345 2.508417558 0.98135214 0.034358748
GRÅ_5 GRÅ_3 -0.035120237 -1.810579931 1.740339457 1 0.938436773
GRÅ_5 HVID_3 0.393451192 -1.382008502 2.168910886 1 0.387223108
GRÅ_5 SORT 1.164879763 -0.610579931 2.940339457 0.830703344 0.010884952
GRÅ_7 GRØN_0 -0.45 -1.978590881 1.078590881 0.999999978 0.250951621
GRÅ_7 GRØN_12 0.816405898 -0.794099819 2.426911616 0.99379707 0.048527989
GRÅ_7 GRØN_3 0.546662535 -1.077555836 2.170880907 0.999999132 0.189504853
GRÅ_7 GRØN_5 0.469394939 -1.154874928 2.093664806 0.999999987 0.259728907
GRÅ_7 GRØN_7 0.070016531 -1.554258195 1.694291258 1 0.866343018
GRÅ_7 GUL_0 0.075 -1.453590881 1.603590881 1 0.848080489
GRÅ_7 GUL_12a 0.913499562 -0.775957066 2.60295619 0.983005463 0.03550531
GRÅ_7 GUL_12b 0.74546078 -0.878480201 2.369401762 0.99894881 0.073931268
GRÅ_7 GUL_3a 0.537287535 -1.086930836 2.161505907 0.999999449 0.197184613
GRÅ_7 GUL_3b 0.720642419 -0.968814209 2.410099047 0.999760568 0.096668422
GRÅ_7 GUL_5 0.178769939 -1.445499928 1.803039806 1 0.667439145
GRÅ_7 GUL_7 0.191891531 -1.432383195 1.816166258 1 0.644663003
GRÅ_7 HVID_0a -0.726243379 -2.298252057 0.845765298 0.99881223 0.072111301
GRÅ_7 HVID_0b 0.43608578 -1.187855201 2.060026762 0.999999998 0.294898555
GRÅ_7 HVID_12 0.276812857 -1.347501442 1.901127157 1 0.505967677
GRÅ_7 HVID_5 -0.623330607 -2.247792558 1.001131345 0.999977422 0.134883612
106 |
GRÅ_7 HVID_7 -0.465 -1.993590881 1.063590881 0.999999944 0.235539059
GRÅ_7 JIS_3 0.670419393 -0.954042558 2.294881345 0.999882115 0.107935147
GRÅ_7 RØD_0 -0.285 -1.813590881 1.243590881 1 0.46684269
GRÅ_7 RØD_12 0.403769939 -1.220499928 2.028039806 1 0.332165417
GRÅ_7 RØD_3 0.140423287 -1.431585391 1.712431965 1 0.72728351
GRÅ_7 RØD_5 0.220562857 -1.403751442 1.844877157 1 0.596076767
GRÅ_7 RØD_7 0.3825 -1.146090881 1.911090881 1 0.328982138
GRÅ_7 GRÅ_3 -0.536575843 -2.225953632 1.152801946 0.999999815 0.215574544
GRÅ_7 HVID_3 -0.108004415 -1.797382204 1.581373374 1 0.802890657
GRÅ_7 SORT 0.663424157 -1.025953632 2.352801946 0.999961318 0.126018872
GRØN_0 GRØN_12 1.266405898 -0.344099819 2.876911616 0.439348581 0.002321356
GRØN_0 GRØN_3 0.996662535 -0.627555836 2.620880907 0.91351927 0.017158408
GRØN_0 GRØN_5 0.919394939 -0.704874928 2.543664806 0.967681353 0.027794311
GRØN_0 GRØN_7 0.520016531 -1.104258195 2.144291258 0.99999977 0.211943742
GRØN_0 GUL_0 0.525 -1.003590881 2.053590881 0.999998532 0.180638866
GRØN_0 GUL_12a 1.363499562 -0.325957066 3.05295619 0.377121317 0.001791802
GRØN_0 GUL_12b 1.19546078 -0.428480201 2.819401762 0.599915158 0.004336045
GRØN_0 GUL_3a 0.987287535 -0.636930836 2.611505907 0.922167377 0.018220376
GRØN_0 GUL_3b 1.170642419 -0.518814209 2.860099047 0.733684254 0.007201531
GRØN_0 GUL_5 0.628769939 -0.995499928 2.253039806 0.999972252 0.131480965
GRØN_0 GUL_7 0.641891531 -0.982383195 2.266166258 0.999955303 0.123652489
GRØN_0 HVID_0a -0.276243379 -1.848252057 1.295765298 1 0.492821291
GRØN_0 HVID_0b 0.88608578 -0.737855201 2.510026762 0.980600894 0.033874267
GRØN_0 HVID_12 0.726812857 -0.897501442 2.351127157 0.999362224 0.081474819
GRØN_0 HVID_5 -0.173330607 -1.797792558 1.451131345 1 0.677009667
GRØN_0 HVID_7 -0.015 -1.543590881 1.513590881 1 0.969436623
GRØN_0 JIS_3 1.120419393 -0.504042558 2.744881345 0.742873273 0.007469601
GRØN_0 RØD_0 0.165 -1.363590881 1.693590881 1 0.673466826
GRØN_0 RØD_12 0.853769939 -0.770499928 2.478039806 0.98891321 0.040915013
GRØN_0 RØD_3 0.590423287 -0.981585391 2.162431965 0.99998642 0.143312496
GRØN_0 RØD_5 0.670562857 -0.953751442 2.294877157 0.999881327 0.107828335
GRØN_0 RØD_7 0.8325 -0.696090881 2.361090881 0.981118938 0.034206696
GRØN_0 GRÅ_3 -0.086575843 -1.775953632 1.602801946 1 0.84141128
GRØN_0 HVID_3 0.341995585 -1.347382204 2.031373374 1 0.429545154
GRØN_0 SORT 1.113424157 -0.575953632 2.802801946 0.823667515 0.010534748
GRØN_12 GRØN_3 -0.269743363 -1.972404023 1.432917297 1 0.5363123
GRØN_12 GRØN_5 -0.347010959 -2.048462302 1.354440384 1 0.426074656
GRØN_12 GRØN_7 -0.746389367 -2.449153795 0.956375061 0.999578673 0.087777846
GRØN_12 GUL_0 -0.741405898 -2.351911616 0.869099819 0.998889678 0.073044871
GRØN_12 GUL_12a 0.097093663 -1.66755918 1.861746507 1 0.829896514
GRØN_12 GUL_12b -0.070945118 -1.769146397 1.627256161 1 0.870422072
GRØN_12 GUL_3a -0.279118363 -1.981779023 1.423542297 1 0.522252846
GRØN_12 GUL_3b -0.095763479 -1.860416323 1.668889364 1 0.832191841
GRØN_12 GUL_5 -0.637635959 -2.339087302 1.063815384 0.999987118 0.144125473
GRØN_12 GUL_7 -0.624514367 -2.327278795 1.078250061 0.999992362 0.152843279
GRØN_12 HVID_0a -1.542649278 -3.195027032 0.109728477 0.112058368 0.000313107
GRØN_12 HVID_0b -0.380320118 -2.078521397 1.317881161 1 0.382175319
GRØN_12 HVID_12 -0.539593041 -2.242412171 1.163226089 0.999999826 0.216574409
| 107
GRØN_12 HVID_5 -1.439736505 -3.142792511 0.263319502 0.274355273 0.001074827
GRØN_12 HVID_7 -1.281405898 -2.891911616 0.329099819 0.410906959 0.002061727
GRØN_12 JIS_3 -0.145986505 -1.849042511 1.557069502 1 0.737864708
GRØN_12 RØD_0 -1.101405898 -2.711911616 0.509099819 0.759308171 0.007953048
GRØN_12 RØD_12 -0.412635959 -2.114087302 1.288815384 1 0.344030769
GRØN_12 RØD_3 -0.675982611 -2.328360365 0.976395143 0.999902778 0.110963029
GRØN_12 RØD_5 -0.595843041 -2.298662171 1.106976089 0.999997636 0.172534787
GRØN_12 RØD_7 -0.433905898 -2.044411616 1.176599819 0.999999998 0.29325161
GRØN_12 GRÅ_3 -1.352981742 -3.117405425 0.411441942 0.500900823 0.002968607
GRØN_12 HVID_3 -0.924410313 -2.688833997 0.840013371 0.989463391 0.041522548
GRØN_12 SORT -0.152981742 -1.917405425 1.611441942 1 0.73497245
GRØN_3 GRØN_5 -0.077267596 -1.793302729 1.638767536 1 0.860448327
GRØN_3 GRØN_7 -0.476646004 -2.192751295 1.239459286 0.999999996 0.278676289
GRØN_3 GUL_0 -0.471662535 -2.095880907 1.152555836 0.999999985 0.257424488
GRØN_3 GUL_12a 0.366837026 -1.407785562 2.141459614 1 0.419880644
GRØN_3 GUL_12b 0.198798245 -1.516739221 1.914335711 1 0.650998909
GRØN_3 GUL_3a -0.009375 -1.71839156 1.69964156 1 0.982911663
GRØN_3 GUL_3b 0.173979884 -1.600642704 1.948602471 1 0.701913392
GRØN_3 GUL_5 -0.367892596 -2.083927729 1.348142536 1 0.402856694
GRØN_3 GUL_7 -0.354771004 -2.070876295 1.361334286 1 0.419840786
GRØN_3 HVID_0a -1.272905915 -2.935107431 0.389295601 0.504159426 0.003020189
GRØN_3 HVID_0b -0.110576755 -1.826114221 1.604960711 1 0.801307206
GRØN_3 HVID_12 -0.269849678 -1.986015144 1.446315788 1 0.53939154
GRØN_3 HVID_5 -1.169993142 -2.886420716 0.546434432 0.765512194 0.008186975
GRØN_3 HVID_7 -1.011662535 -2.635880907 0.612555836 0.898391312 0.015572623
GRØN_3 JIS_3 0.123756858 -1.592670716 1.840184432 1 0.778325811
GRØN_3 RØD_0 -0.831662535 -2.455880907 0.792555836 0.992687686 0.046381089
GRØN_3 RØD_12 -0.142892596 -1.858927729 1.573142536 1 0.745109511
GRØN_3 RØD_3 -0.406239248 -2.068440764 1.255962267 1 0.340360203
GRØN_3 RØD_5 -0.326099678 -2.042265144 1.390065788 1 0.458362141
GRØN_3 RØD_7 -0.164162535 -1.788380907 1.460055836 1 0.693152765
GRØN_3 GRÅ_3 -1.083238379 -2.857681712 0.691204954 0.918329008 0.017732434
GRØN_3 HVID_3 -0.65466695 -2.429110283 1.119776383 0.999991175 0.150495814
GRØN_3 SORT 0.116761621 -1.657681712 1.891204954 1 0.797248746
GRØN_5 GRØN_7 -0.399378408 -2.115569278 1.316812463 1 0.363911641
GRØN_5 GUL_0 -0.394394939 -2.018664806 1.229874928 1 0.343494699
GRØN_5 GUL_12a 0.444104623 -1.335150776 2.223360022 1 0.330195441
GRØN_5 GUL_12b 0.276065841 -1.43958265 1.991714333 1 0.529971318
GRØN_5 GUL_3a 0.067892596 -1.648142536 1.783927729 1 0.877236022
GRØN_5 GUL_3b 0.25124748 -1.528007919 2.030502879 1 0.581495795
GRØN_5 GUL_5 -0.290625 -1.99964156 1.41839156 1 0.506877195
GRØN_5 GUL_7 -0.277503408 -1.993694278 1.438687463 1 0.527962947
GRØN_5 HVID_0a -1.195638318 -2.861602745 0.470326108 0.658708852 0.005400477
GRØN_5 HVID_0b -0.033309159 -1.74895765 1.682339333 1 0.939573704
GRØN_5 HVID_12 -0.192582082 -1.908833232 1.523669069 1 0.661347986
GRØN_5 HVID_5 -1.092725546 -2.805544584 0.620093492 0.870175721 0.013256676
GRØN_5 HVID_7 -0.934394939 -2.558664806 0.689874928 0.960024868 0.025368481
GRØN_5 JIS_3 0.201024454 -1.511794584 1.913843492 1 0.646834387
108 |
GRØN_5 RØD_0 -0.754394939 -2.378664806 0.869874928 0.998685866 0.070612117
GRØN_5 RØD_12 -0.065625 -1.77464156 1.64339156 1 0.880821597
GRØN_5 RØD_3 -0.328971652 -1.994936078 1.336992775 1 0.440951417
GRØN_5 RØD_5 -0.248832082 -1.965083232 1.467419069 1 0.571439111
GRØN_5 RØD_7 -0.086894939 -1.711164806 1.537374928 1 0.834545583
GRØN_5 GRÅ_3 -1.005970782 -2.777139552 0.765197988 0.966178481 0.027270358
GRØN_5 HVID_3 -0.577399354 -2.348568124 1.193769416 0.999999627 0.203741767
GRØN_5 SORT 0.194029218 -1.577139552 1.965197988 1 0.668896442
GRØN_7 GUL_0 0.004983469 -1.619291258 1.629258195 1 0.990441929
GRØN_7 GUL_12a 0.84348303 -0.931056727 2.618022788 0.99797879 0.064309466
GRØN_7 GUL_12b 0.675444249 -1.040178499 2.391066997 0.999958993 0.125062339
GRØN_7 GUL_3a 0.467271004 -1.248834286 2.183376295 0.999999998 0.288219495
GRØN_7 GUL_3b 0.650625888 -1.12391387 2.425165645 0.999992423 0.153043374
GRØN_7 GUL_5 0.108753408 -1.607437463 1.824944278 1 0.804589116
GRØN_7 GUL_7 0.121875 -1.58714156 1.83089156 1 0.780687619
GRØN_7 HVID_0a -0.796259911 -2.462272993 0.869753172 0.99776598 0.062863552
GRØN_7 HVID_0b 0.366069249 -1.349553499 2.081691997 1 0.40507647
GRØN_7 HVID_12 0.206796326 -1.50513971 1.918732363 1 0.637243228
GRØN_7 HVID_5 -0.693347138 -2.40561687 1.018922594 0.999922536 0.114702608
GRØN_7 HVID_7 -0.535016531 -2.159291258 1.089258195 0.999999508 0.199094473
GRØN_7 JIS_3 0.600402862 -1.11186687 2.312672594 0.999997507 0.171707354
GRØN_7 RØD_0 -0.355016531 -1.979291258 1.269258195 1 0.393751728
GRØN_7 RØD_12 0.333753408 -1.382437463 2.049944278 1 0.447899555
GRØN_7 RØD_3 0.070406756 -1.595606327 1.736419838 1 0.868941108
GRØN_7 RØD_5 0.150546326 -1.56138971 1.862482363 1 0.731355171
GRØN_7 RØD_7 0.312483469 -1.311791258 1.936758195 1 0.452789271
GRØN_7 GRÅ_3 -0.606592375 -2.385723821 1.172539072 0.999998785 0.18382637
GRØN_7 HVID_3 -0.178020946 -1.957152393 1.601110501 1 0.696067408
GRØN_7 SORT 0.593407625 -1.185723821 2.372539072 0.999999315 0.193508935
GUL_0 GUL_12a 0.838499562 -0.850957066 2.52795619 0.995655142 0.053462483
GUL_0 GUL_12b 0.67046078 -0.953480201 2.294401762 0.999881126 0.107801303
GUL_0 GUL_3a 0.462287535 -1.161930836 2.086505907 0.999999991 0.266981217
GUL_0 GUL_3b 0.645642419 -1.043814209 2.335099047 0.999979482 0.136465096
GUL_0 GUL_5 0.103769939 -1.520499928 1.728039806 1 0.803024631
GUL_0 GUL_7 0.116891531 -1.507383195 1.741166258 1 0.778733325
GUL_0 HVID_0a -0.801243379 -2.373252057 0.770765298 0.993215297 0.047383987
GUL_0 HVID_0b 0.36108578 -1.262855201 1.985026762 1 0.385629357
GUL_0 HVID_12 0.201812857 -1.422501442 1.826127157 1 0.627667969
GUL_0 HVID_5 -0.698330607 -2.322792558 0.926131345 0.999718133 0.094105142
GUL_0 HVID_7 -0.54 -2.068590881 0.988590881 0.999996991 0.168540339
GUL_0 JIS_3 0.595419393 -1.029042558 2.219881345 0.999992479 0.153168339
GUL_0 RØD_0 -0.36 -1.888590881 1.168590881 1 0.358180636
GUL_0 RØD_12 0.328769939 -1.295499928 1.953039806 1 0.429608897
GUL_0 RØD_3 0.065423287 -1.506585391 1.637431965 1 0.870917524
GUL_0 RØD_5 0.145562857 -1.478751442 1.769877157 1 0.726440589
GUL_0 RØD_7 0.3075 -1.221090881 1.836090881 1 0.432449694
GUL_0 GRÅ_3 -0.611575843 -2.300953632 1.077801946 0.999994458 0.15828067
GUL_0 HVID_3 -0.183004415 -1.872382204 1.506373374 1 0.672372786
| 109
GUL_0 SORT 0.588424157 -1.100953632 2.277801946 0.999997897 0.174573598
GUL_12a GUL_12b -0.168038781 -1.943035603 1.606958041 1 0.711684636
GUL_12a GUL_3a -0.376212026 -2.150834614 1.398410562 1 0.408114109
GUL_12a GUL_3b -0.192857143 -2.019872696 1.63415841 1 0.680272753
GUL_12a GUL_5 -0.734729623 -2.513985022 1.044525776 0.999880609 0.107729878
GUL_12a GUL_7 -0.72160803 -2.496147788 1.052931727 0.99991489 0.113179632
GUL_12a HVID_0a -1.639742941 -3.365887604 0.086401722 0.09277371 0.000249289
GUL_12a HVID_0b -0.477413781 -2.252410603 1.297583041 0.999999998 0.294126025
GUL_12a HVID_12 -0.636686704 -2.411287149 1.137913741 0.999995565 0.162018215
GUL_12a HVID_5 -1.536830168 -3.31184613 0.238185793 0.227374655 0.000821214
GUL_12a HVID_7 -1.378499562 -3.06795619 0.310957066 0.35175271 0.001596955
GUL_12a JIS_3 -0.243080168 -2.01809613 1.531935793 1 0.592950501
GUL_12a RØD_0 -1.198499562 -2.88795619 0.490957066 0.684418196 0.005951364
GUL_12a RØD_12 -0.509729623 -2.288985022 1.269525776 0.99999999 0.263868205
GUL_12a RØD_3 -0.773076275 -2.499220938 0.953068388 0.999350667 0.081200988
GUL_12a RØD_5 -0.692936704 -2.467537149 1.081663741 0.999966075 0.128172906
GUL_12a RØD_7 -0.530999562 -2.22045619 1.158457066 0.999999861 0.220390233
GUL_12a GRÅ_3 -1.450075405 -3.286084254 0.385933444 0.428709795 0.00223486
GUL_12a HVID_3 -1.021503977 -2.857512826 0.814504873 0.974440247 0.030544628
GUL_12a SORT -0.250075405 -2.086084254 1.585933444 1 0.594946866
GUL_12b GUL_3a -0.208173245 -1.923710711 1.507364221 1 0.635714207
GUL_12b GUL_3b -0.024818362 -1.799815183 1.75017846 1 0.956462116
GUL_12b GUL_5 -0.566690841 -2.282339333 1.14895765 0.999999471 0.19784689
GUL_12b GUL_7 -0.553569249 -2.269191997 1.162053499 0.999999717 0.208391502
GUL_12b HVID_0a -1.47170416 -3.137261436 0.193853117 0.191148807 0.000642856
GUL_12b HVID_0b -0.309375 -2.01839156 1.39964156 1 0.479883826
GUL_12b HVID_12 -0.468647923 -2.184327507 1.247031661 0.999999997 0.286684982
GUL_12b HVID_5 -1.368791387 -3.084767284 0.34718451 0.404833165 0.002021504
GUL_12b HVID_7 -1.21046078 -2.834401762 0.413480201 0.57006578 0.003878629
GUL_12b JIS_3 -0.075041387 -1.791017284 1.64093451 1 0.864424698
GUL_12b RØD_0 -1.03046078 -2.654401762 0.593480201 0.876951295 0.01375335
GUL_12b RØD_12 -0.341690841 -2.057339333 1.37395765 1 0.437043438
GUL_12b RØD_3 -0.605037493 -2.27059477 1.060519783 0.999993965 0.15685219
GUL_12b RØD_5 -0.524897923 -2.240577507 1.190781661 0.999999934 0.232876649
GUL_12b RØD_7 -0.36296078 -1.986901762 1.260980201 1 0.383170499
GUL_12b GRÅ_3 -1.282036624 -3.060456624 0.496383377 0.648647343 0.005200188
GUL_12b HVID_3 -0.853465195 -2.631885196 0.924954806 0.997594506 0.061799715
GUL_12b SORT -0.082036624 -1.860456624 1.696383377 1 0.857069002
GUL_3a GUL_3b 0.183354884 -1.591267704 1.957977471 1 0.686685695
GUL_3a GUL_5 -0.358517596 -2.074552729 1.357517536 1 0.414935219
GUL_3a GUL_7 -0.345396004 -2.061501295 1.370709286 1 0.432216255
GUL_3a HVID_0a -1.263530915 -2.925732431 0.398670601 0.522292732 0.00323809
GUL_3a HVID_0b -0.101201755 -1.816739221 1.614335711 1 0.817841697
GUL_3a HVID_12 -0.260474678 -1.976640144 1.455690788 1 0.553565743
GUL_3a HVID_5 -1.160618142 -2.877045716 0.555809432 0.780312487 0.008707205
GUL_3a HVID_7 -1.002287535 -2.626505907 0.621930836 0.908033696 0.016547772
GUL_3a JIS_3 0.133131858 -1.583295716 1.849559432 1 0.762025026
GUL_3a RØD_0 -0.822287535 -2.446505907 0.801930836 0.993929118 0.048882344
110 |
GUL_3a RØD_12 -0.133517596 -1.849552729 1.582517536 1 0.76130366
GUL_3a RØD_3 -0.396864248 -2.059065764 1.265337267 1 0.351602088
GUL_3a RØD_5 -0.316724678 -2.032890144 1.399440788 1 0.471371892
GUL_3a RØD_7 -0.154787535 -1.779005907 1.469430836 1 0.709853821
GUL_3a GRÅ_3 -1.073863379 -2.848306712 0.700579954 0.925962008 0.018730743
GUL_3a HVID_3 -0.64529195 -2.419735283 1.129151383 0.999993801 0.156403982
GUL_3a SORT 0.126136621 -1.648306712 1.900579954 1 0.781369924
GUL_3b GUL_5 -0.54187248 -2.321127879 1.237382919 0.999999942 0.235002209
GUL_3b GUL_7 -0.528750888 -2.303290645 1.24578887 0.999999968 0.245253439
GUL_3b HVID_0a -1.446885798 -3.173030461 0.279258865 0.292176548 0.001191166
GUL_3b HVID_0b -0.284556638 -2.05955346 1.490440183 1 0.531496774
GUL_3b HVID_12 -0.443829562 -2.218430007 1.330770884 1 0.329230787
GUL_3b HVID_5 -1.343973026 -3.118988987 0.431042936 0.53195369 0.003359239
GUL_3b HVID_7 -1.185642419 -2.875099047 0.503814209 0.707507678 0.006501647
GUL_3b JIS_3 -0.050223026 -1.825238987 1.724792936 1 0.912032613
GUL_3b RØD_0 -1.005642419 -2.695099047 0.683814209 0.938958082 0.020742976
GUL_3b RØD_12 -0.31687248 -2.096127879 1.462382919 1 0.487019271
GUL_3b RØD_3 -0.580219132 -2.306363795 1.145925531 0.999999161 0.190073964
GUL_3b RØD_5 -0.500079562 -2.274680007 1.274520884 0.999999994 0.27174332
GUL_3b RØD_7 -0.338142419 -2.027599047 1.351314209 1 0.434776839
GUL_3b GRÅ_3 -1.257218262 -3.093227112 0.578790587 0.756889945 0.007903141
GUL_3b HVID_3 -0.828646834 -2.664655683 1.007362016 0.999243806 0.078885235
GUL_3b SORT -0.057218262 -1.893227112 1.778790587 1 0.903151653
GUL_5 GUL_7 0.013121592 -1.703069278 1.729312463 1 0.976184215
GUL_5 HVID_0a -0.905013318 -2.570977745 0.760951108 0.981803486 0.034659995
GUL_5 HVID_0b 0.257315841 -1.45833265 1.972964333 1 0.558264237
GUL_5 HVID_12 0.098042918 -1.618208232 1.814294069 1 0.823503835
GUL_5 HVID_5 -0.802100546 -2.514919584 0.910718492 0.998465777 0.068329822
GUL_5 HVID_7 -0.643769939 -2.268039806 0.980499928 0.999952217 0.122561115
GUL_5 JIS_3 0.491649454 -1.221169584 2.204468492 0.999999989 0.262945439
GUL_5 RØD_0 -0.463769939 -2.088039806 1.160499928 0.999999991 0.265469052
GUL_5 RØD_12 0.225 -1.48401656 1.93401656 1 0.607306033
GUL_5 RØD_3 -0.038346652 -1.704311078 1.627617775 1 0.928388495
GUL_5 RØD_5 0.041792918 -1.674458232 1.758044069 1 0.924251666
GUL_5 RØD_7 0.203730061 -1.420539806 1.827999928 1 0.624395238
GUL_5 GRÅ_3 -0.715345782 -2.486514552 1.055822988 0.99992688 0.115637742
GUL_5 HVID_3 -0.286774354 -2.057943124 1.484394416 1 0.527414893
GUL_5 SORT 0.484654218 -1.286514552 2.255822988 0.999999997 0.285842651
GUL_7 HVID_0a -0.918134911 -2.584147993 0.747878172 0.977634522 0.032144032
GUL_7 HVID_0b 0.244194249 -1.471428499 1.959816997 1 0.578488114
GUL_7 HVID_12 0.084921326 -1.62701471 1.796857363 1 0.846427272
GUL_7 HVID_5 -0.815222138 -2.52749187 0.897047594 0.997917218 0.063875758
GUL_7 HVID_7 -0.656891531 -2.281166258 0.967383195 0.999924673 0.11515555
GUL_7 JIS_3 0.478527862 -1.23374187 2.190797594 0.999999995 0.275715663
GUL_7 RØD_0 -0.476891531 -2.101166258 1.147383195 0.999999979 0.252215435
GUL_7 RØD_12 0.211878408 -1.504312463 1.928069278 1 0.629845954
GUL_7 RØD_3 -0.051468244 -1.717481327 1.614544838 1 0.903991068
GUL_7 RØD_5 0.028671326 -1.68326471 1.740607363 1 0.947861627
| 111
GUL_7 RØD_7 0.190608469 -1.433666258 1.814883195 1 0.64687591
GUL_7 GRÅ_3 -0.728467375 -2.507598821 1.050664072 0.999900904 0.110724188
GUL_7 HVID_3 -0.299895946 -2.079027393 1.479235501 1 0.510604939
GUL_7 SORT 0.471532625 -1.307598821 2.250664072 0.999999999 0.301223267
HVID_0a HVID_0b 1.16232916 -0.503228117 2.827886436 0.719194887 0.006803302
HVID_0a HVID_12 1.003056237 -0.663007891 2.669120365 0.930247404 0.019345461
HVID_0a HVID_5 0.102912773 -1.563340978 1.769166524 1 0.809444853
HVID_0a HVID_7 0.261243379 -1.310765298 1.833252057 1 0.516585368
HVID_0a JIS_3 1.396662773 -0.269590978 3.062916524 0.292209061 0.0011913
HVID_0a RØD_0 0.441243379 -1.130765298 2.013252057 0.999999994 0.273633809
HVID_0a RØD_12 1.130013318 -0.535951108 2.795977745 0.774642963 0.008502896
HVID_0a RØD_3 0.866666667 -0.744609598 2.477942932 0.984273656 0.036472519
HVID_0a RØD_5 0.946806237 -0.719257891 2.612870365 0.965925479 0.027184106
HVID_0a RØD_7 1.108743379 -0.463265298 2.680752057 0.696913626 0.006242339
HVID_0a GRÅ_3 0.189667536 -1.53636207 1.915697142 1 0.667936384
HVID_0a HVID_3 0.618238965 -1.107790641 2.344268571 0.999995744 0.162709676
HVID_0a SORT 1.389667536 -0.33636207 3.115697142 0.382776413 0.001837277
HVID_0b HVID_12 -0.159272923 -1.874952507 1.556406661 1 0.717032363
HVID_0b HVID_5 -1.059416387 -2.775392284 0.65655951 0.907556004 0.016496094
HVID_0b HVID_7 -0.90108578 -2.525026762 0.722855201 0.975380184 0.030992123
HVID_0b JIS_3 0.234333613 -1.481642284 1.95030951 1 0.59399074
HVID_0b RØD_0 -0.72108578 -2.345026762 0.902855201 0.999452931 0.083820796
HVID_0b RØD_12 -0.032315841 -1.747964333 1.68333265 1 0.94137237
HVID_0b RØD_3 -0.295662493 -1.96121977 1.369894783 1 0.488431234
HVID_0b RØD_5 -0.215522923 -1.931202507 1.500156661 1 0.623867972
HVID_0b RØD_7 -0.05358578 -1.677526762 1.570355201 1 0.897486543
HVID_0b GRÅ_3 -0.972661624 -2.751081624 0.805758377 0.979932018 0.033458649
HVID_0b HVID_3 -0.544090195 -2.322510196 1.234329806 0.999999934 0.232878366
HVID_0b SORT 0.227338376 -1.551081624 2.005758377 1 0.617774147
HVID_12 HVID_5 -0.900143464 -2.612460913 0.812173985 0.988895466 0.040894392
HVID_12 HVID_7 -0.741812857 -2.366127157 0.882501442 0.999047918 0.075413785
HVID_12 JIS_3 0.393606536 -1.318710913 2.105923985 1 0.369796534
HVID_12 RØD_0 -0.561812857 -2.186127157 1.062501442 0.999998242 0.17759163
HVID_12 RØD_12 0.126957082 -1.589294069 1.843208232 1 0.772727278
HVID_12 RØD_3 -0.13638957 -1.802453698 1.529674557 1 0.749265928
HVID_12 RØD_5 -0.05625 -1.76526656 1.65276656 1 0.8977452
HVID_12 RØD_7 0.105687143 -1.518627157 1.730001442 1 0.799468108
HVID_12 GRÅ_3 -0.813388701 -2.592596446 0.965819045 0.999028772 0.075113505
HVID_12 HVID_3 -0.384817272 -2.164025018 1.394390474 1 0.398704531
HVID_12 SORT 0.386611299 -1.392596446 2.165819045 1 0.396512395
HVID_5 HVID_7 0.158330607 -1.466131345 1.782792558 1 0.703567329
HVID_5 JIS_3 1.29375 -0.41526656 3.00276656 0.532427801 0.003365652
HVID_5 RØD_0 0.338330607 -1.286131345 1.962792558 1 0.41638625
HVID_5 RØD_12 1.027100546 -0.685718492 2.739919584 0.933407853 0.019828043
HVID_5 RØD_3 0.763753894 -0.902499858 2.430007645 0.998978292 0.074355715
HVID_5 RØD_5 0.843893464 -0.868423985 2.556210913 0.996150383 0.055145378
HVID_5 RØD_7 1.005830607 -0.618631345 2.630292558 0.904617036 0.016188803
HVID_5 GRÅ_3 0.086754763 -1.688704931 1.862214457 1 0.848696681
112 |
HVID_5 HVID_3 0.515326192 -1.260133502 2.290785886 0.999999985 0.257659311
HVID_5 SORT 1.286754763 -0.488704931 3.062214457 0.636428581 0.004968045
HVID_7 JIS_3 1.135419393 -0.489042558 2.759881345 0.715970601 0.006718536
HVID_7 RØD_0 0.18 -1.348590881 1.708590881 1 0.645739472
HVID_7 RØD_12 0.868769939 -0.755499928 2.493039806 0.985537051 0.037523348
HVID_7 RØD_3 0.605423287 -0.966585391 2.177431965 0.99997539 0.133460155
HVID_7 RØD_5 0.685562857 -0.938751442 2.309877157 0.999808674 0.100212601
HVID_7 RØD_7 0.8475 -0.681090881 2.376090881 0.975653729 0.031126051
HVID_7 GRÅ_3 -0.071575843 -1.760953632 1.617801946 1 0.868610654
HVID_7 HVID_3 0.356995585 -1.332382204 2.046373374 1 0.409612508
HVID_7 SORT 1.128424157 -0.560953632 2.817801946 0.801759739 0.009547552
JIS_3 RØD_0 -0.955419393 -2.579881345 0.669042558 0.947193984 0.022294992
JIS_3 RØD_12 -0.266649454 -1.979468492 1.446169584 1 0.543423951
JIS_3 RØD_3 -0.529996106 -2.196249858 1.136257645 0.999999807 0.214913638
JIS_3 RØD_5 -0.449856536 -2.162173985 1.262460913 0.999999999 0.305462627
JIS_3 RØD_7 -0.287919393 -1.912381345 1.336542558 1 0.489106654
JIS_3 GRÅ_3 -1.206995237 -2.982454931 0.568464457 0.770500458 0.008357498
JIS_3 HVID_3 -0.778423808 -2.553883502 0.997035886 0.999576801 0.087774423
JIS_3 SORT -0.006995237 -1.782454931 1.768464457 1 0.987726084
RØD_0 RØD_12 0.688769939 -0.935499928 2.313039806 0.999788586 0.09863248
RØD_0 RØD_3 0.425423287 -1.146585391 1.997431965 0.999999998 0.291166876
RØD_0 RØD_5 0.505562857 -1.118751442 2.129877157 0.999999894 0.224896408
RØD_0 RØD_7 0.6675 -0.861090881 2.196090881 0.999610173 0.089048771
RØD_0 GRÅ_3 -0.251575843 -1.940953632 1.437801946 1 0.561061542
RØD_0 HVID_3 0.176995585 -1.512382204 1.866373374 1 0.682530347
RØD_0 SORT 0.948424157 -0.740953632 2.637801946 0.971072846 0.029084538
RØD_12 RØD_3 -0.263346652 -1.929311078 1.402617775 1 0.537242142
RØD_12 RØD_5 -0.183207082 -1.899458232 1.533044069 1 0.676872085
RØD_12 RØD_7 -0.021269939 -1.645539806 1.602999928 1 0.959221968
RØD_12 GRÅ_3 -0.940345782 -2.711514552 0.830822988 0.987070408 0.0389435
RØD_12 HVID_3 -0.511774354 -2.282943124 1.259394416 0.999999987 0.259796562
RØD_12 SORT 0.259654218 -1.511514552 2.030822988 1 0.567165208
RØD_3 RØD_5 0.08013957 -1.585924557 1.746203698 1 0.851028655
RØD_3 RØD_7 0.242076713 -1.329931965 1.814085391 1 0.547797434
RØD_3 GRÅ_3 -0.67699913 -2.403028736 1.049030475 0.999962378 0.126475149
RØD_3 HVID_3 -0.248427702 -1.974457308 1.477601904 1 0.574242416
RØD_3 SORT 0.52300087 -1.203028736 2.249030475 0.999999949 0.237376308
RØD_5 RØD_7 0.161937143 -1.462377157 1.786251442 1 0.697120983
RØD_5 GRÅ_3 -0.757138701 -2.536346446 1.022069045 0.999772187 0.097451176
RØD_5 HVID_3 -0.328567272 -2.107775018 1.450640474 1 0.471091073
RØD_5 SORT 0.442861299 -1.336346446 2.222069045 1 0.331535875
RØD_7 GRÅ_3 -0.919075843 -2.608453632 0.770301946 0.981412117 0.034398055
RØD_7 HVID_3 -0.490504415 -2.179882204 1.198873374 0.999999985 0.257501465
RØD_7 SORT 0.280924157 -1.408453632 1.970301946 1 0.516322554
GRÅ_3 HVID_3 0.428571429 -1.398444124 2.255586982 1 0.360092088
GRÅ_3 SORT 1.2 -0.627015553 3.027015553 0.829040716 0.010801313
HVID_3 SORT 0.771428571 -1.055586982 2.598444124 0.999807012 0.10007628
| 113
Appendix E
Program Code
In this appendix some of the code used in the analysis is given.
114 |
Mixed model ANOVA and ANCOVA from Chapter 10 Here on attribute Earthy(o)
3-way mixed model ANOVA proc mixed data=sasuser.fisk2 order=internal method=reml cl nobound;
class code assessor replicate;
model earthy_o_=code/ ddfm=satterth intercept;
random assessor code*assessor replicate(code);
lsmeans code;
run;
4-way mixed model ANOVA proc mixed data=sasuser.fisk2 order=internal method=reml cl nobound;
class foder istid assessor replicate;
model earthy_o_=foder istid foder*istid/ ddfm=satterth intercept;
random assessor foder*assessor istid*assessor istid*foder*assessor
replicate(foder*istid);
*lsmeans foder;
run;
3-way mixed model ANCOVA proc mixed data=sasuser.fisk2 order=internal method=reml cl nobound;
class foder assessor replicate;
model earthy_o_=istid foder foder*istid/ddfm=satterth;
random assessor assessor*foder replicate(foder);
run;
quit;
| 115
3-way mixed model MANOVA with CVA *1-way Full model for hver assessor, TODO: så brug andre datasæt et for hver assessor;
data udensort;
set sasuser.fisk2;
if foder ne 'sort';
run;
*Full model uden assessor*replicate(code;
proc glm data=udensort order=internal outstat=fiskstat;
class code assessor replicate;
model earthy_o_ cooked_potato_o_ sourish_o_ sour_o_ muddy_o_
earthy_f_ mushroom_f_ cooked_potato_f_ sourish_f_ sweet_f_ green_f_ muddy_f_
flaky_t_ firm_t_ juicy_t_ fibrousness_t_ oiliness_t_
= code assessor code*assessor replicate(code)/nouni;
*means code/lsd e=code*assessor;
manova h=code e=code*assessor/canonical;
*random assessor code*assessor replicate(code);
run;
116 |
4-way mixed model MANOVA with CVA data udensort;
set sasuser.fisk2;
if foder ne 'sort';
run;
*Full model uden assessor*replicate(foder*istid);
proc glm data=udensort order=internal outstat=fiskstat4;
class foder istid assessor replicate;
model earthy_o_ cooked_potato_o_ sourish_o_ sour_o_ muddy_o_
earthy_f_ mushroom_f_ cooked_potato_f_ sourish_f_ sweet_f_ green_f_ muddy_f_
flaky_t_ firm_t_ juicy_t_ fibrousness_t_ oiliness_t_
= foder istid assessor
foder*istid assessor*foder assessor*istid
assessor*foder*istid replicate(foder*istid);
*assessor*replicate(foder*istid);
*manova h = foder e=foder*assessor/canonical;
manova h = istid e=istid*assessor/canonical;
*manova h = foder*istid e=foder*assessor*istid;
random assessor
assessor*foder assessor*istid
assessor*foder*istid replicate(foder*istid);
run;
quit;
| 117
References
Borga, M., & Landelius, T., Knutsson, H (1997). A Unified Approach to PCA, PLS, MLR and CCA. Brockhoff, P.B. (2000). Mixed Linear Models. Online Lecture Notes for course on Mixed LinearModels
from an applied perspective. (URL: http://www.imm.dtu.dk/˜pbb/st113).
Brockhoff, P.B. (2007) The design and univariate analysis of sensory profile data
Campbell, N.A., & Atchley, W.R. (1981). The geometry of canonical variate analysis. Systematic Zoology, 30(3), 268-280.
Esbensen, K.H. (2002) Multivariate Data Analysis -in practice (5th Edition).
Everitt, B.G. (2001), A Handbook of Statistical Analyses Using SAS (2nd Edition). Taylor & Franciss LTD
Gittins, R. (1985). Canonical analysis: a review with applications in ecology. Berlin: Springer Verlag.
Hartley, H.O. and Rao, J.N.K. (1967), "Maximum-Likelihood Estimation for the Mixed Analysis of Variance
Model," Biometrika, 54, 93 -108.
Henderson, C.R. (1984), Applications of Linear Models in Animal Breeding, University of Guelph.
Jackson, J. E. (2003). A User's Guide to Principal Components. Wiley-Interscience.
Jolliffe, I.T. (2002) Principal Component Analysis. Springer.
Krzanowski, W.J. (2000). Principles of Multivariate Analysis: A users perspective, Oxford statistical
science.
Langsrud, Ø. (2002). 50-50 multivariate analysis of variance for collinear responses
Little, R.J.A. (1995), "Modeling the Drop-Out Mechanism in Repeated-Measures Studies," Journal of the
American Statistical Association, 90, 1112 -1121.
118 |
Martens, H., Martens, M. (2001) Multivariate Analysis of Quality : An Introduction. Wiley.
Rencher, A.C. (2002) Methods of Multivariate Analysis. Wiley.
Tinsley, E.A. (2000) Handbook of Applied Multivariate Statistics and Mathematical Modeling. Acedemic
Press.
| 119