Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Designs in constrained experimental regions Mixture Experiments Prof Eddie Schrevens Biosystems Department Faculty of Bioscience Engineering K. Universiteit Leuven Belgium [email protected]
Outline
• Paul Darius and mixtures
• Part I
o Definitions
o Design and analysis of mixture experiments
o Whole simplex designs
• Multiple grain cracker: simple mixture
• Composed fertilizer: constrained, double mixture amount experiment
• Part II
o Constrained mixture experiments with non-homomorphic experimental regions
• Investigating the ill conditioning
o Optimal design revisited
Part I
Mixtures?
Paul Darius and design and analysis of mixtures
Why mixtures?
• Investigate the effects of composition and process variables
on responses
o Investigate the importance of the compositional components
o Investigate the importance of the process variables
o Investigate respective interactions
o Predict the optimal response
• Pancake, concrete, …
Mixture constraints
• Mixture of q components xi for i in 1 tot q
1
1q
i
i
x
0 1ix
The factor spaces
Simplex factorspace for a two component mixture
x1
x2
x1 = 1 x2 = 1
(1,0) (0,1)
(1,0)
(0,1) (0,0)
(0,0,1)
(0,1,0)
(1,0,0)
x2
x1
x1 = 1
(1,0,0)
(0,1,0)
x2 = 1
(0,0,1)
x3 = 1
Simplex factorspace for a 3 component mixture
x3
q=2 q=3
Consequences
• Experimental factors not independent
o One exact collinearity/dependence relation
• Classical, orthogonal experimental designs not applicable
• Specific design and analysis methodology
• Multivariate approach is necessary
Additional constraints
• Single component constraints
• Mulitiple component constraints in p of the
q components
1
p
j ij i j
i
a c x b
0 1i i il x u
Complex constrained mixture experiment
0.10 x1 0.40
0.25 x2 0.50
0.20 x3 0.60
2.3 x1 + 3 x2 + 4 x3 3 x1 = 1
x2 = 1 x3 = 1
Experimental regions
in high dimensionality
Irregular convex
hyper-polyhedrons
Process variables
Combined region of a 22 factorial design and
a {3,2} simplex lattice design
x2 = 1
-1
+1
-1 +1 z1
z1
z2
z2
x1 = 1
x3 = 1
a b
x3 = 1
x3 = 1
x2 = 1
x2 = 1
x1 = 1
Combined region for a 3 component mixture
(x1,x2,x3) and 1 process variable (z) in 2 levels
x1 = 1
z = 1
z = 2
Experimental factor not bound by the mixture constraints
Multiple mixtures
Y2 = 1 Y3 = 1
Y1 = 1
X3 = 1
X1 = 1
{3,2;3,2} double lattice design
X2 = 1
3
1
1i
i
Y p
3
1
i
i
X p
Mixture of mixtures of mixtures…
Double mixture: each mixture is
itself a mixture or a mixture of
mixtures, mixed with proportions p
and 1-p
Defined by multiple component
equalities
Analysis of mixture experiments • Numeric responses
o Specific mixture models: Sheffé canonical polynomials for simple mixtures
o Incorporation of the mixture constraint in the model
o Combination of submodels for process variable and multiple mixture models by multiplying the models
o Relies on OLS and RSM in classical approaches
o Ridge regression, PCA and PLS regression multicolinearity
• Categorical responses o Generalized linear modelsmodels
o PLSD
o TBM
1
q
i i
i
y x
1
q q q
i i ij i j
i i j
y x x x
Mixture experimental design
• Transform to independent factors
• Whole simplex experiments
• Homomorphic experimental regions
o Same shape as the overall simplex
• Complex constrained experimental regions
o Irregular convex hyper-polyhedron
Transform to independent factors
Advantages
Known methodology for design
and analysis
Disadvantages
Difficult interpretation
Exp factors are linear
combinations of the original
mixture components
Interactions, multiple mixtures
For mixtures with additional
constraints near-
multicollinearity problems
persist
(1/3,
1/3,
1/3
)
[0,0,0]
(0,0)
w2
w1
x3
x2
x1
x3
x2
x1
Transformation to independent variables
~
~
~
Get rid of the mixture constraint
Whole simplex experiments
{3,3} simplex lattice design
(0,1/2,0,
1/2)
(1/2,0,
1/2,0)
(0,0,1/2,
1/2)
(1/2,
1/2,0,0) (
1/2,0,0,
1/2)
(0,1/2,
1/2,0)
x4 = 1
x3 = 1
x2 = 1
x1 = 1
x3 = 1 x2 = 1
x1 = 1
{4,2} simplex lattice design
{q,m} simplex lattice design
3 component simplex centroid design
(1/2,0,
1/2,0)
(1/3,0,
1/3,
1/3)
(0,0,1/2,
1/2)
(1/2,0,0,
1/2) (
1/2,
1/2,0,0)
(0,1/2,
1/2,0)
x2 = 1 x4 = 1
x3 = 1
x1 = 1
x3 = 1 x2 = 1
x1 = 1
Simplex centroid designs
4 component simplex
centroid design
(0,1/3,
1/3,
1/3) (0,
1/2,0,
1/2)
(1/4,
1/4,
1/4,
1/4)
(1/3,
1/3,
1/3,0)
(1/3,
1/3,0,
1/3)
Many appropriate optimal design plans exist
Homomorphic experimental regions
x1 = 0.35
x3 = 0.20x2 = 0.15
x3’ = 1 x2
’ = 1
x1 = 1
x1’ = 1
x3 = 1 x2 = 1
x1 0.35
x2 0.15
x3 0.20
' i ii q
i
i 1
x lx
1 l
i i0 l x 1
q
i
i 1
l 1
L-pseudocomponents xi’
Lower bounds Whole simplex design in pseudocomponents
xi’
Whole simplex experiments in raw mixture
components or pseudocomponents: examples
• Most ‘optimal’ approaches for design and analysis of
mixture experiments
• Straightforward design and analysis, even for complex
problems
• Examples
o Multigrain crackers
o Composed liquid fertilizer as a constrained, double mixture amount
experiment
Multigrain crackers
Objectives
o Develop a multiple grain cracker where a proportion p of the
wheat flower is replaced with buckwheat, oats, barley and rye
o Optimising the flour composition of a multiple grain cracker in
relation to consumer preference, scored by magnitude estimation
Multigrain crackers
Defining constraints of the experimental region
o All components of the dough mixture are constant except flour
o Proportion p of the wheat flour can be replaced by buckwheat, oats, barley and/or rye
• 1-p Pwheat 1
• 0 Pbuckwheat p
• 0 Poats p
• 0 Pbarley p
• 0 Prye p
o Homomorphic experimental region with 5 components
o Pseudocomponents transformation to whole simplex
o {5,2} simplex lattice is proposed
Multigrain crackers
Mixture Wheat Buckwheat Oats Barley Rye
1 1 0 0 0 0
2 0 1 0 0 0
3 0 0 1 0 0
4 0 0 0 1 0
5 0 0 0 0 1
6 0.5 0.5 0 0 0
7 0.5 0 0.5 0 0
8 0.5 0 0 0.5 0
9 0.5 0 0 0 0.5
10 0 0.5 0.5 0 0
11 0 0.5 0 0.5 0
12 0 0.5 0 0 0.5
13 0 0 0.5 0.5 0
14 0 0 0.5 0 0.5
15 0 0 0 0.5 0.5
{5,2} simplex lattice design in pseudocomponents
Multigrain crackers
Full model: quadratic Scheffé canonical polynomial
o 5 linear terms
o 10 cross product terms
Pref. = 10000wheat + 01000buckwheat + 00100oats + 00010barley
+ 00001rye + 11000wheat*buckwheat + 10100wheat*oats +
10010wheat*barley + 10001wheat*rye + 01100buckwheat*oats +
01010buckwheat*barley + 01001buckwheat*rye + 00110oats*barley
+ 00101oats*rye + 00011barley*rye
Multigrain crackers
Model reduction based on all possible models and
maximum R2
pref. = 66.62 wheat + 76.51 buckwheat + 62.41
oats + 40.99 barley + 32.81 rye – 129.39
wheat*buckwheat
R2 = 0.98 CV=6 %
0.2
0.1
0
0 0.1 0.2
Buckwheat
Wheat
Oats
Barley
Adj. pref. (%)
1 0
1
62.4
Adj. pref. (%)
62.7
0.9 0
0.9
Barley
Oats
63.2
0.8 Oats 0
0.8
Barley
Adj. pref. (%)
63.8
0.9
0
0.9
Oats
Barley
64.2
0.8
0
0.8
Oats
Barley
Adj. pref. (%)
65.2
0.8
0
0.8
Oats Barley
41.5
37.1 39.3
32.8 36.2 39.6
Adj. pref. (%)
Adj. pref. (%)
Figure 5: Adjusted preference of the Belgian experts for 3 wheat and buckwheat levels
Multigrain crackers
Conclusions
o Replacing the wheat flour partially with buckwheat and oats had a
positive effect on the consumers preference
o A antagonistic interaction between wheat and buckwheat persists
o Rye and barley addition resulted in low preference
Composed liquid fertilizer as a constrained, double
mixture amount experiment
Objectives
Optimize the composition of a composed liquid fertilizer in
relation to plant response (Ca content of Rye grass)
Composed liquid fertilizer as a constrained, double
mixture amount experiment
Defining constraints Aqueous solutions of inorganic ions
K+ Ca2+ Mg2+ NO3- H2PO4
- SO42-
with C = total concentration in units of charge (meq/l)
Prepared by dissolving salts
fi KNO3, Ca(NO3)2, MgSO4, …
Ionic balance constraint: balance of charge [K+] + [Ca2+] + [Mg2+] = [NO3
-] + [H2PO4-] + [SO42-] =C/2
Double mixture constraints: multicomponent equality
constraints
Composed liquid fertilizer as a constrained
double mixture amount experiment
4
5 3
1
2
Ca2+
= 1
K+ = 1
Mg2+
= 1 Cation factorspace
6
5
6 4
2 3
1
NO3- = 1
H2PO4- = 1 SO4
2- = 1
Anion factorspace
0.44 K+ 0.79
0.09 Ca2+ 0.44
0.12 Mg2+ 0.47
0.56 NO3- 0.82
0.09 H2PO4- 0.35
0.09 SO42- 0.35
{3,2} Simlex Lattice
Composed liquid fertilizer as a constrained, double
mixture amount experiment
• Mixtures ({3,2} SimLat) in the separate mixtures are transformed to pseudocomponents
• Total concentration (C) is defined as a process variable
• The mixture design is the combination of the two constituent mixtures (Cations, Anions), mixed with proportion 0.5
• This design is repeated at the levels of the process variable (C = total concentration or amount)
Composed liquid fertilizer as a constrained, double
mixture amount experiment
• Pseudocomponents transformations
+K 0.44k
0.35
2Ca 0.09ca
0.35
2Mg 0.12mg
0.35
-
3NO 0.56n
0.26
-
2 4H PO 0.09p
0.26
2-
4SO 0.09s
0.26
Total milli-eq concentration (C)
n = 0
p = 0
s = 0.5
n = 0
p = 0.5
s = 0
n = 0.5
p = 0
s = 0
k
mg ca
12.5 meq/l 50 meq/l
Double mixture experiment with one process variable in two levels
n = 0.5
p = 0
s = 0
k
mg ca
n = 0
p = 0
s = 0.5
n = 0
p = 0.5
s = 0
Design representation
Composed liquid fertilizer as a constrained, double
mixture amount experiment
Number of treatment combinations
o Cations x anions: 6 x 6 = 36
o 2 levels in the process variable: 36 * 2 = 72
o 5 replications per treatment: 360 exp units
o Checkpoints?
Composed liquid fertilizer as a constrained, double
mixture amount experiment
Model formulation
o The second degree double mixture model with process variable is
obtained by combining the process variable model with a second
degree double mixture model, resulting in a total of 72 terms
f(x,z) = b0100100 kn + b0
100010 kp + b0100001 ks + b0
100110 knp + b0100101 kns + b0
100011 kps +
b0010100 can + b0
010010 cap + b0010001 cas + b0
010110 canp + b0010101 cans + b0
010011 caps +
b0001100 mgn + b0
001010 mgp + b0001001 mgs + b0
001110 mgnp + b0001101 mgns + b0
001011 mgps +
b0110100 kcan + b0
11001 kcap + b0110001 kcas + b0110110 kcanp + b0
110101 kcans + b0110011
kcaps + b0101100 kmgn + b0
101010 kmgp + b0101001 kmgs + b0
101110 kmgnp + b0101101 kmgns +
b0101011 kmgps + b0
011100 camgn + b0011010 camgp + b0
011001 camgs + b0011110 camgnp +
b0011101 camgns + b0
011011 camgps + b1100100 knz + b1
100010 kpz + b1100001 ksz + b1
100110 knpz +
b1100101 knsz + b1
100011 kpsz + b1010100 canz + b1
010010 capz + b1010001 casz + b1
010110 canpz +
b1010101 cansz + b1
010011 capsz + b1001100 mgnz + b1
001010 mgpz + b1001001 mgsz + b1
001110
mgnpz + b1001101 mgnsz + b1
001011 mgpsz + b1110100 kcanz + b1
110010 kcapz + b1110001 kcasz +
b1110110 kcanpz + b1
110101 kcansz + b1110011 kcapsz + b1
101100 kmgnz + b1101010 kmgpz +
b1101001 kmgsz + b1
101110 kmgnpz + b1101101 kmgnsz + b1
101011 kmgpsz + b1011100 camgnz +
b1011010 camgpz + b1
011001 camgsz + b1011110 camgnpz + b1
011101 camgnsz + b1011011 camgpsz
With x: mixture variable
z: process variable
k, ca, mg, n, p and s: pseudocomponents in proportions
Composed liquid fertilizer as a constrained, double mixture
amount: model formulation
bz k ca mg n p s: parameters
Composed liquid fertilizer as a constrained, double
mixture amount
Reduced Model 25 terms:
calcium = 211.65 kn + 217.64 kp + 237.35 ks + 672.47 can
+ 657.87 cap + 700.02 cas + 245.40 cap
+ 176.66 mgn + 125.13 mgp + 123.59 mgs
+ 928.30 kcan + 542.34 kcap + 2437.85 kcans
- 282.50 kmgs + 661.36 camgn + 716.75 camgp
+ 318.91 camgs +2418.63 camgns + 37.37 knz
+ 88.21 canz + 78.72 mgnz + 36.88 mgpz
+ 60.66 mgsz + 795.67 kcapz + 573.17 kcasz
R2 = 0.99, CV = 4.9
n = 0.25, p = 0, s = 0.25 n = 0.25, p = 0.25, s = 0
n = 0, p = 0.25, s = 0.25 n = 0, p = 0, s = 0.5 n = 0, p = 0.5, s = 0
n = 0.5, p = 0, s = 0
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca0.5
0
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
12.5 mval/l
k = 0.25, ca = 0, mg = 0.25 k = 0.25, ca = 0.25, mg = 0
k = 0, ca = 0.25, mg = 0.25 k = 0, ca = 0, mg = 0.5 k = 0, ca = 0.5, mg = 0
k = 0.5, ca = 0, mg = 0
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
2020
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
2020
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
2020
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
2020
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
2020
n 0.250
0.25
0.5
p
12.5 mval/l
n = 0.25, p = 0, s = 0.25 n = 0.25, p = 0.25, s = 0
n = 0, p = 0.25, s = 0.25 n = 0, p = 0, s = 0.5 n = 0, p = 0.5, s = 0
n = 0.5, p = 0, s = 0
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
Calcium (mmol/kg DW)
200
110
200.5
k 0.250
0.25
0.5
ca
50 mval/l
k = 0.25, ca = 0, mg = 0.25 k = 0.25, ca = 0.25, mg = 0
k = 0, ca = 0.25, mg = 0.25 k = 0, ca = 0, mg = 0.5 k = 0, ca = 0.5, mg = 0
k = 0.5, ca = 0, mg = 0
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
Calcium (mmol/kg DW)
200
110
200.5
n 0.250
0.25
0.5
p
50 mval/l
Composed liquid fertilizer as a constrained, double
mixture amount experiment
Conclusions
o Ca in the fertilizer increases the Ca content in the Rye grass
o Marked Ca*K antagonistic effect on the Ca content in the Rye
grass
o Marked Mg*Ca antagonistic effect on Ca content in the Rye grass
o No cation – anion interactions
o Total concentration has an additive effect, not interacting with
composition
Part II
Complex constrained mixtures
Irregular, convex hyper-polyhedrons
Optimal design strategy revisited
Mixture experimental design
• Transform to independent factors o Classical design and analysis
• Whole simplex experiments o {q,m} simplex lattice, simplex centroid (D-optimal with respect to
corresponding mixture models)
• Homomorphic experimental regions o Same shape as the overall simplex
o Pseudo-components transformation expands the experimental
region to the whole simplex
o Whole simplex experiments in pseudocomponents
• Complex constrained experimental regions o Irregular, convex hyper-polyhedrons
Irregular, convex (hyper)-polyhedron
Experimental region for the optimization of
a 4 component explosive
Complex constrained experimental regions
Classical approach to design
• Each specific problem has its own optimal design related
to size, shape and location of the experimental region
defined by the set of additional constraints
• Optimal design theory is indispensable
• Implies computer aided design of experiments, especially
in high dimensionality
• Extremely computational intensive exchange
algorithms
• Ill conditioning is more rule than exception
Complex constrained experimental regions
Classical procedure
• Constraints expert or screening
• Consistency check of constraints!
• Vertices of the hyper-polyhedron are computed
• ‘Mixture’ model is chosen
• Centroids of different dimensionality are
calculated
List of candidate points: vertices extended
with centroids
Complex constrained experimental regions
Classical procedure
• Optimality criteria is chosen: D, G, V, A, …
• Optimal design is selected by an exchange algorithm taking into account the selected model o Branch and bound, excursion algorithms or all possible subsets
• This algorithm is runned for different numbers of design points
Optimal design for the pre-specified model
and optimality criterion
Classical design procedures: example
Constraints
20.22 0.47x
30.06 0.56x
10.22 0.72x
X1
X2
X3
Vertex
Edge centroid
Plane centroid
List of 9 candidate points
4
2
3
7
8
5
6
9 1
List of candidate points for the linear mixture
model
Id X1 X2 X3
1 0.720 0.220 0.060
2 0.470 0.470 0.060
3 0.220 0.470 0.310
4 0.220 0.220 0.560
5 0.470 0.220 0.310
6 0.595 0.345 0.060
7 0.345 0.470 0.185
8 0.220 0.345 0.435
9 0.408 0.345 0.248
Design matrix X
List of candidate points for the quadratic
mixture model
Id X1 X2 X3 X1X2 X1X3 X2X3
1 0.720 0.220 0.060 0.158400 0.043200 0.013200
2 0.470 0.470 0.060 0.220900 0.028200 0.028200
3 0.220 0.470 0.310 0.103400 0.068200 0.145700
4 0.220 0.220 0.560 0.048400 0.123200 0.123200
5 0.470 0.220 0.310 0.103400 0.145700 0.068200
6 0.595 0.345 0.060 0.205275 0.035700 0.020700
7 0.345 0.470 0.185 0.162150 0.063825 0.086950
8 0.220 0.345 0.435 0.075900 0.095700 0.150075
9 0.408 0.345 0.248 0.140760 0.101184 0.085560
Extended design matrix X
Design procedures: example
• Given the expanded design matrix X for OLS estimation, representing model specifications
• For optimal parameter estimation: o A-optimality minimises the trace of (X’X/n)-1, resulting in a minimal
variance of the parameters
o D-optimality minimizes the determinant of (X’X/n) -1
o E-optimality minimizes the maximum eigenvalue of (X’X/n) -1
• For optimal response estimation: o G-optimality minimises the maximum prediction variance over a
specified set of design points (normalised for the number of design points)
o V-optimality minimises the average prediction variance over a specified set of design points (normalised for the number of design points)
Design procedures: example, 1st order
• Calculation of optimality criteria for all subsets of 4, 5, 6, 7, 8, 9 points out of the 9 candidates
• Set of points 1,2,3,4 has the lowest D-criterion for a first order linear mixture model
Number of
design points
D-optimality Candidate point
number
4 1638 1 2 3 4
5 2133 1 2 3 4 5
1 2 3 4 7
6 2457 1 2 3 4 5 7
1 2 3 4 5 8
7 2804 1 2 3 4 5 6 7
1 2 3 4 5 7 8
8 3013 1 2 3 4 5 6 7 8
9 3812 1 2 3 4 5 6 7 8 9
Optimal design
Design procedures: example 2d order
• Calculation of optimality criteria for all subsets of 6, 7, 8, 9 points out of the 9 candidates of the expanded matrix
• Quadratic mixture model
• Set of points 1,2,3,4,6,8 has the lowest D-criterion for a second order linear mixture model
Number
design
points
D-optimality Candidate point
number
6 3.5 E14 1 2 3 4 6 8
7 5.9 E14 1 2 3 4 5 6 8
8 6.6 E14 1 2 3 4 5 6 8 9
9 8.5 E14 1 2 3 4 5 6 7 8 9
Optimal design
How to deal with ill conditioning
(multicollinearity) in experiments with
mixtures?
• In many situations the additional constraints induce ill-conditioning or near-collinearities in the design matrix on top of the mixture constraint (exact collinearity)
• On top of this, the ill conditioning is increased by model formulation
• Implications for OLS o Large estimates
o Large variances and covariances of the estimates
o Incorrect signs of the estimates
o Unreliable test statistics
o Unreliable variable selection
How to deal with ill conditioning
(multicollinearity) in experiments with
mixtures?
• Duality o Parameter predicted response estimation
o Alternative optimality criteria ill conditioning
• Investigate ill conditioning of a selected optimal
design
• Decrease ill conditioning by re-design
• (Decrease ill conditioning by re-modelling)
Investigate collinearity structures
Classical approaches: VIF
OLS of y=X and svd of X=ULV’
var() = σ2 (X’X)-1 = σ2 (VL-2V’)
For the kth element of
2qkj2 2
k 2j 1 j
vvar( )
lkVIF
VIF=diag (X’X)-1 = diag(V’L-2V)
Investigate collinearity structures of constrained
mixture design matrices
Classical approaches: VIF
W=ULV’
VIF=diag (W’W)-1 = diag(V’L-2V)
ij
ijn
2
ij
i 1
xw
x
Standardizing the terms of the model
generates the most stable parameter
estimation
Investigate collinearity structures of constrained
mixture design matrices
Classical approaches: VIF
VIFi can be decomposed in components associated with the respective singular values
VIF decomposition
2qij
i 2j 1 j
vVIF
l
2
ij
ij 2
j
vVIF
l The proportion of VIFi
Corresponding to lj ij
i
i
VIFVIF
VIF
High variance inflation can be allocated to eigenvectors (PC)
with small corresponding singular values lj
With X or W = ULV’
Investigate collinearity structures of constrained
mixture design matrices
• An exploratory multivariate analysis approach
• Different factorization of the standardized design
matrix o Factorisation 1: correlational structure
o Factorisation 2: collinearity constraints
• Graphical representation by Biplots o Rank 2 approximation
o Superimposing standardized scores and loading plot
Factorisation 1: correlational structure
Starting from the standardized design matrix Z , the correlation matrix R=Z’Z
Singular value decomposition: Z=ULV’
Factorisation 1 Z=GH’
With G=U and H=VL
Then R=H’H Correlation
Z=GH’ Values of design points are orthogonal projections on the exp factors
G Standardized principal components scores
Superimposing the first and second columns of G and H in a biplot results in a rank 2 graphical approximation of the correlational structure of the design
Factorisation 2: collinearity constraints
Starting from the standardized design matrix Z
Singular value decomposition: Z=ULV’
Factorisation 2 Z=PQ’
With P=UL and Q=V
Then Q: principal component loadings
P: principal components scores
Superimposing the columns of P and Q, corresponding to the smallest singular values results in a rank 2 graphical approximation or biplot, that gives information about collinearity equations
Factorisation 1: correlational structure
Example: {3,1} simplex lattice
X1
X2
X3
First component
Second
com
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1
2 3
X1
X2 X3
Id X1 X2 X3
1 1 0 0
2 0 1 0
3 0 0 1
Factorisation 1: correlational structure
Example: {3,1} simplex lattice
X1
X2
X3
Correlation matrix R
1 -0.5 -0.5
-0.5 1 -0.5
-0.5 -0.5 1
Id X1 X2 X3
1 1 0 0
2 0 1 0
3 0 0 1 Singular values of X
1
1
1
VIF decomposition
VIF VIF1 VIF2 VIF3
1 1 0 0
1 0 1 0
1 0 0 1
X1
X2
X3
Id X1 X2 X3
1 1 0 0
2 0 1 0
3 0 0 1
Factorisation 1: correlational structure
Example: {3,2} simplex lattice
X1
X2
X3
First component
Second
com
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1
2
3
4
5
6
X1
X2
X3
X1X2
X1X3
X2X3
Id X1 X2 X3 X1X2 X1X3 X2X3
1 1.0 0.0 0.0 0.00 0.00 0.00
2 0.0 1.0 0.0 0.00 0.00 0.00
3 0.0 0.0 1.0 0.00 0.00 0.00
4 0.5 0.5 0.0 0.25 0.00 0.00
5 0.0 0.5 0.5 0.00 0.00 0.25
6 0.5 0.0 0.5 0.00 0.25 0.00
Factorisation 1: correlational structure
Example: {3,2} simplex lattice
X1
X2
X3
Singular values of W
1.41
1.15
1.15
0.7
0.7
0.5
VIF decomposition
VIF VIF1 VIF2 VIF3 VIF4 VIF5 VIF6
1 1.5 0.1 0.2 0 0 0.8 0.4
2 1.5 0.1 0.05 0.15 0.6 0.2 0.4
3 1.5 0.1 0.05 0.15 0.6 0.2 0.4
12 1.5 0.066 0.07 0.22 0.4 0.13 0.6
13 1.5 0.066 0.07 0.22 0.4 0.13 0.6
23 1.5 0.066 0.3 0 0 0.53 0.6
Correlation matrix
1 -0.5 -0.5 0.2 0.2 -0.4
-0.5 1 -0.5 0.2 -0.4 0.2
-0.5 -0.5 1 -0.4 0.2 0.2
0.2 0.2 -0.4 1 -0.2 -0.2
0.2 -0.4 0.2 -0.2 1 -0.2
-0.4 0.2 0.2 -0.2 -0.2 1
X1
X2
X3
Factorisation 1: correlational structure
Example: constrained mixture Id X1 X2 X3
1 0.7 0.1 0.2
2 0.7 0.2 0.1
3 0.3 0.6 0.1
4 0.2 0.6 0.2
First component
Second
com
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1
2 3
4
X1 X2
X3
1
3
2 4
1
2
3
0.2 0.7
0.1 0.6
0.1 0.2
x
x
x
X1
X2
X3
Factorisation 1: correlational structure
Example: constrained mixture
Singular values of W
1.57
0.67
0.29
Id X1 X2 X3
1 0.7 0.1 0.2
2 0.7 0.2 0.1
3 0.3 0.6 0.1
4 0.2 0.6 0.2
Correlation matrix
1 -0.97 -0.11
-0.97 1 -0.11
-0.11 -0.11 1
VIF decomposition
VIF VIF1 VIF2 VIF3
1 3.87 0.1283436 0.9900934 2.7530263
2 3.06 0.1217495 1.2260788 1.7133913
3 7.25 0.1558498 0.005732 7.0896378
1
3
2 4
Factorisation 1: correlational structure
Example: optimization of bread dough composition
Component Lower
bounds
Upper
bounds
Water 0.2 0.4
Flour 0.5 0.8
Salt 0.03 0.044
Additive 0.0091 0.0095
Yeast 0.0045 0.0048
Constraints
16 vertices
Extended with 65 2,3,4,5
dimensional centroids
81 candidate points 16 points D-optimal design
Factorisation 1: correlational structure
Example: optimization of bread dough composition with a
linear mixture model
First component
Second c
om
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
12
34
56
78
910
1112
1314
1516
WaterFlour
Ad
Salt
Yeast
Factorisation 1: correlational structure
Example: optimization bread dough composition with
quadratic mixture model
1234
5678
9101112
13141516
Water1 X1X4 X1X5 Flour2
X1X2
X1X3
X2X3 Salt3
X2X4
X2X5
Yeast5
Ad4
X4X5
Factorisation 2: collinearity constraints
Example: optimization of bread dough composition with a
linear mixture model
Fifth component
Fourth c
om
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
WaterFlour
Ad
Salt
Yeast
Singular values of Z
1.41
1.00
1.00
1.00
2.120 E-16
PC5 = -0.7 Water -0.7 Flour -
0.005 Salt -0.001Ad -
0.001Yeast 0
‘Near’ collinearity constraint
Factorisation 2: collinearity constraints
Example: optimization of bread dough composition with a
linear mixture model
VIF decomposition
VIF VIF1 VIF2 VIF3 VIF4 VIF5
1 320.11888 0.0389517 5.8668324 0.5888333 18.047538 295.57672
2 1363.468 0.0409593 2.334777 3.4602724 78.632188 1278.9998
3 31.279872 0.0414719 0.0990308 27.645318 0.1196886 3.3743626
4 2123.2871 0.0423696 0.0644128 1.7034599 254.33501 1867.1418
5 953.12182 0.0423552 0.0650038 1.774838 846.05622 105.18341
Singular values
of W
2.20 0.34 0.17 0.029 0.017
Factorisation 2: collinearity constraints
Example: optimization of bread dough composition with a
quadratic mixture model
Fifteenth component
Fourteen
th c
om
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
12345678910111213141516
Water1Flour2
Ad3
Salt4
Yeast5X1X2X1X3
X1X4
X1X5
X2X3
X2X4
X2X5
X3X4X3X5X4X5
Component Singular values
1 2.84
2 1.95
3 1.43
4 1.02
5 0.19
6 0.04
7 0.03
8 0.02
9 0.018
10 0.01
11 2.8 E-16 ~ 0
12 2.6 E-16 ~ 0
13 2.2 E-16 ~ 0
14 1.5 E-16 ~ 0
15 4.8 E-17 ~ 0
Factorisation 2: collinearity constraints
Example: optimization of bread
dough composition with quadratic
mixture model
Thirteenth component
Tw
elfth
com
ponent
-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0
-1.0
-0.8
-0.6
-0.4
-0.2
0.0
0.2
0.4
0.6
0.8
1.0
12345678910111213141516Water1
Flour2
Ad3
Salt4
Yeast5
X1X2
X1X3
X1X4
X1X5
X2X3
X2X4
X2X5
X3X4X3X5X4X5
Component Singular values
1 2.84
2 1.95
3 1.43
4 1.02
5 0.19
6 0.04
7 0.03
8 0.02
9 0.018
10 0.01
11 2.8 E-16 ~ 0
12 2.6 E-16 ~ 0
13 2.2 E-16 ~ 0
14 1.5 E-16 ~ 0
15 4.8 E-17 ~ 0
Factorisation 2: collinearity constraints
Example: optimization of bread dough
composition with quadratic mixture model
11 2.8 E-16 ~ 0
12 2.6 E-16 ~ 0
13 2.2 E-16 ~ 0
14 1.5 E-16 ~ 0
15 4.8 E-17 ~ 0
Component Singular values
Corresponding PC are defining
the collinearity constraints
PC11 PC12 PC13 PC14 PC15
Water1 -0.29 -0.06 0.44 0.33 0.54
Flour2 0.09 -0.47 -0.22 0.39 0.49
Ad3 0.55 0.27 0.06 0.01 0.25
Salt4 -0.03 0.01 -0.04 -0.10 0.08
Yeast5 -0.06 0.13 -0.11 0.10 0.01
X1X2 0.33 -0.36 -0.58 0.05 -0.04
X1X3 -0.32 -0.25 -0.12 0.01 -0.15
X1X4 0.18 -0.07 0.21 0.49 -0.39
X1X5 0.19 -0.43 0.35 -0.32 -0.03
X2X3 -0.48 -0.27 -0.07 0.01 -0.19
X2X4 0.17 -0.06 0.23 0.49 -0.39
X2X5 0.19 -0.44 0.37 -0.33 -0.03
X3X4 0.006 -0.007 0.01 0.03 -0.02
X3X5 0.01 -0.03 0.02 -0.02 -0.003
X4X5 0.001 -0.001 0.001 -0.0002 -0.0008
Factorisation 2: collinearity constraints
PC loadings corresponding with singular values ~ 0
How to use this information for more ‘optimal’
design?
• Instead of optimizing variance properties of parameters and predicted response under OLS estimation
• Minimize correlational structure
o Scree plot of singular values
o Optimize factorisation 1
• Minimize collinearity structure
o Optimize factorisation 2
• Redesign to uniform singular value profile
Guarantee design
in the ‘full’ region
in the exchange
algorithm
• Protected design points, fi vertices
• Adapt constraints
Solutions for the bread example
• Solutions for ill conditioning o Redesign
• Categorisation of the components
• Define small ranged components as process variables
o (Remodel)
• Model reduction
• Adapted models
Conclusion
• Plenty of interesting ideas for better mixture design
optimality criteria for constrained mixture experiments by
detailled study of ill conditioning
• Correlational and collinearity structures
• Computational problems to calculate for realistic problems
in industry exchange algorithms in high dimensionality
Thank you for your attention
Questions?