Space-Filling Designs for Space-Filling Designs for High-Dimensional Mixture High-Dimensional Mixture Experiments with Multiple Experiments with Multiple

ConstraintsConstraintsJohn J. BorkowskiJohn J. Borkowski

Montana State UniversityMontana State UniversityBozeman, MTBozeman, MT

ICAQM 2006 ConferenceICAQM 2006 ConferenceTaipei, TaiwanTaipei, TaiwanJune 10, 2006June 10, 2006

OUTLINEOUTLINE

1.1. MotivationMotivation2.2. Number-theoretic (NT) design Number-theoretic (NT) design

generation in the hypercubegeneration in the hypercube3.3. Number-theoretic mixture design Number-theoretic mixture design

(NTMD) generation.(NTMD) generation.4.4. The High-Dimensional Multiple-The High-Dimensional Multiple-

Component Constraint (MCC) ProblemComponent Constraint (MCC) Problem5.5. An example: 8 components, 5 MCCsAn example: 8 components, 5 MCCs6.6. Final CommentsFinal Comments

1.1. MotivationMotivation Constrained mixture experimentsConstrained mixture experiments:: qq components (or ingredients) components (or ingredients) xxi i is the proportion of the iis the proportion of the ithth

componentcomponent

for i = 1, 2, … , for i = 1, 2, … , q q and and ΣΣ xxii = 1 = 1

Single-component constraints (SCC)Single-component constraints (SCC)

0 0 ≤ L≤ Lii ≤ ≤ xxii ≤ U ≤ Uii ≤ 1 ≤ 1

Multiple-component constraints (MCC)Multiple-component constraints (MCC)

CCii ≤ ≤ ΣΣ A Aiixxii ≤ D ≤ Dii

Motivation (cont.)Motivation (cont.) The goal is to generate designs with points The goal is to generate designs with points

“scattered uniformly” throughout constrained “scattered uniformly” throughout constrained mixture spaces defined by SCCs and MCCs.mixture spaces defined by SCCs and MCCs.

The designs must contain ”boundary” and The designs must contain ”boundary” and “interior” points, even for high-dimensional “interior” points, even for high-dimensional regions (e.g., 8 or more mixture components).regions (e.g., 8 or more mixture components).

TodayToday Discuss several number-theoretic (NT) Discuss several number-theoretic (NT)

approaches for generating space-filling approaches for generating space-filling mixture designs (NTMDs) in highly-mixture designs (NTMDs) in highly-constrained regionsconstrained regions

NotationNotation

x = x = ((xx11, x, x22, , … … xxss)) NN = desired = desired design sizedesign size CCss = [0,1] = [0,1]ss (unit cube)(unit cube) TTss = { = {x x : : ∑∑xxii = 1, = 1, xxii ≥ 0≥ 0} } (simplex)(simplex)

TTss(a,b) = {(a,b) = {xx TTss: 0 : 0 ≤ ≤ aaii ≤ ≤ xxii ≤ ≤ bbii ≤ 1 ≤ 1}}

where a = (where a = (aa11, …, , …, aass) and b = () and b = (bb11, …, , …, bbss)) (constrained subspace of simplex)(constrained subspace of simplex)

2. NT-Design Point Generation in C2. NT-Design Point Generation in Css

1.1. Lattice Point Lattice Point (LP)(LP) method method

2.2. Square root sequence Square root sequence (SRS)(SRS) method method

3.3. Powers of the (s+1)Powers of the (s+1)st st root root (PR)(PR) method method

4.4. Cyclotomic field Cyclotomic field (CF)(CF) method method

5.5. Halton-setHalton-set (H1)(H1) methodmethod

6.6. Hammersley-set Hammersley-set (H2)(H2) method method

LPLP : form lattices from integers : form lattices from integers

SRS, PR, CFSRS, PR, CF : use the fractional part of a number : use the fractional part of a number

H1, H2H1, H2 : based on radical inverses of : based on radical inverses of integersintegers

NT-Design Point Generators in CNT-Design Point Generators in Css (Fang and Wang 1994)(Fang and Wang 1994)

HHkk= (= (hh1k1k, , hh2k2k, …, , …, hhsksk)) is the NT is the NT design point generator design point generator for the kfor the kthth design point X design point Xkk (where (where hhik ik depends on depends on the NT-method used)the NT-method used) . .

Forms of the generator Forms of the generator HHk k = (= (hh1k1k, , hh2k2k, …, , …, hhsksk))

1. Lattice-point 1. Lattice-point (LP)(LP) method method: : HHk k = (kn= (kn11, kn, kn22, …, kn, …, knss) mod ) mod NN where (i) nwhere (i) nii (ii) n (ii) ni i < < NN (iii) n(iii) nii ≠ n≠ njj for ifor i≠j (iv) gcd(≠j (iv) gcd(NN,n,nii) = 1 ) = 1

2. Square root sequence 2. Square root sequence (SRS)(SRS) method method:: HHkk = (k√p = (k√p11, k√p, k√p22, … , k√p, … , k√pss ) ) where (pwhere (p11, p, p22, … , p, … , pss) are unique primes) are unique primes

NT-Design Point Generators in CNT-Design Point Generators in Css

2. Square root sequence 2. Square root sequence (SRS)(SRS) method method:: HHkk = (k√p = (k√p11, k√p, k√p22, … , k√p, … , k√pss ) ) where (pwhere (p11, p, p22, … , p, … , pss) are unique primes) are unique primes

3. Powers of the (s+1)3. Powers of the (s+1)st st root root (PR)(PR) method method:: HHkk = (kq, kq= (kq, kq22, kq, kq33, … , kq, … , kqss )) and q = p and q = p 1/(s+1)1/(s+1) for some prime pfor some prime p

4. 4. Cyclotomic field Cyclotomic field (CF)(CF) method method: : HHkk = ( kc(p,1) , kc(p,2), … , kc(p,s) ) = ( kc(p,1) , kc(p,2), … , kc(p,s) ) where c(p,i) = | 2 cos(2where c(p,i) = | 2 cos(2ππi/p) | for some i/p) | for some

prime p ≥ 2s+3prime p ≥ 2s+3

NT-Design Point Generators in CNT-Design Point Generators in Css

Note: For k,m Note: For k,m , , bb00, , bb11 , …, , …, bbrr (<m) such that (<m) such that k = k = bb00 + + bb11mm + + bb22mm22 + … + + … +bbrrmmrr

Let y(k,m) = ∑ ( Let y(k,m) = ∑ ( bbii / m/ mi+1i+1 ), which is called the ), which is called the radical inverse of kradical inverse of k with base m.with base m.

55. . Halton-setHalton-set (H1)(H1) method:method: HHkk = ( y(k,p = ( y(k,p11), y(k,p), y(k,p22),…, y(k,p),…, y(k,pss) )) ) where the pwhere the pii are distinct primes are distinct primes 66. . Hammersly-set Hammersly-set (H2) (H2) methodmethod: : HHkk = ( (2k-1)/2 = ( (2k-1)/2NN , y(k,p , y(k,p22),…, y(k,p),…, y(k,pss) )) )

The The kkthth row X row Xkk of NT-design X of NT-design X ((k = 1,…,k = 1,…,N N ))

LPLP methodmethod::

XXkk = ( 2 = ( 2HHkk--1111 ss ) / 2 ) / 2NN

SRS, PR, SRS, PR, andand CF CF methodsmethods

XXkk = ({ = ({HHkk}) = ({kn}) = ({kn11},}, {kn{kn22}, }, … , … , {kn{knss})})

where {kwhere {knnii} is the fractional part of } is the fractional part of kknnii

H1H1 andand H2H2 methodsmethods: : XXkk = = HHkk

Example: LP-method (Example: LP-method (N N =21, s=2)=21, s=2)

Example: LP-method (Example: LP-method (N N =21, s=2)=21, s=2)

What is a “good” NTD What is a “good” NTD generator?generator?

We want the points generated by the We want the points generated by the design generator to be uniformly design generator to be uniformly “scattered” in cube “scattered” in cube CCs s ..

To determine the degree of To determine the degree of “uniformity of scatter”, we need an “uniformity of scatter”, we need an assessment criterion.assessment criterion.

Two Assessment CriteriaTwo Assessment CriteriaLet uLet u11, u, u22, … , u, … , uRR be a random sample of vectors be a random sample of vectors from from CCss. . (evaluation set).(evaluation set).Let d(Let d(xx, , xxDD) = the distance between any ) = the distance between any xx CCss

and its nearest design point and its nearest design point xxDD. .

Mean-squared distance:Mean-squared distance: msd(X)msd(X) = (1/R) = (1/R) ∑ ∑ d(ud(uii, , xxDD))22

Maximum-distance:Maximum-distance: md(X)md(X) = max = maxii [d(u [d(uii, , xxDD)] for i=1, 2,…, R)] for i=1, 2,…, R

Small msd(X) and md(X) imply points in Small msd(X) and md(X) imply points in CCs s tend to be tend to be “close” to the design.“close” to the design.

Example revisited: Example revisited: N N =21 , s=2=21 , s=2(LP-method NT-designs, R=15000 points)(LP-method NT-designs, R=15000 points)

(n1,n2) √msd(X) md(X) *** (1,13) .0954 .2221 (1,8) .0970 .2413 (1,5) .0979 .2191 (1,17) .0981 .2212 *** (1,4) .0991 .2627 (1,16) .0997 .2602 (1,19) .1470 .4124 (1,10) .1474 .4126 *** (1,2) .1626 .4519 (1,11) .1634 .4436 *** (1,20) .2450 .6701

3. Number-theoretic mixture 3. Number-theoretic mixture designs (NTMDs)designs (NTMDs)

Fang and Yang (2000) provide a mapping G Fang and Yang (2000) provide a mapping G of points in of points in CCq-1 q-1 into into TTqq(a,b).(a,b).

Reconsider the 21-point, 2-factor NT-Reconsider the 21-point, 2-factor NT-designs. designs. Suppose we want to generate a 21-point NTMD Suppose we want to generate a 21-point NTMD

such that such that

.1 ≤ .1 ≤ xx1 1 ≤ .7 0≤ ≤ .7 0≤ xx2 2 ≤ .8 .1 ≤ ≤ .8 .1 ≤ xx3 3 ≤ .6≤ .6 After applying G to the NT- design points in After applying G to the NT- design points in CC2 2 , ,

we get three-component NTMDs in we get three-component NTMDs in TT33(a,b) (a,b)

Application of map G (Application of map G (CC2 2 into into TT33(a,b))(a,b))

Application of map G (Application of map G (CC2 2 into into TT33(a,b))(a,b))

Generating NTMDs in Generating NTMDs in TTqq(a,b)(a,b) (no multiple component constraints)(no multiple component constraints)

Generate NT-designs in Generate NT-designs in CCq-1q-1 from a set from a set of generatorsof generators

Apply map G to each NT-design X to Apply map G to each NT-design X to generate a NTMD T in generate a NTMD T in TTqq(a,b)(a,b)

Using an evaluation set in Using an evaluation set in TTqq(a,b), (a,b), calculate msd(T) or md(T) for each Tcalculate msd(T) or md(T) for each T

Select the NTMD with the smallest Select the NTMD with the smallest criterion valuecriterion value

4.4. The High-Dimensional Multiple-The High-Dimensional Multiple-Component Constraint (MCC) Component Constraint (MCC)

ProblemProblem

For high-dimensional mixture problems, For high-dimensional mixture problems, there are often MCCs: Cthere are often MCCs: C ii ≤≤ ∑ A∑ Aiixxii ≤ D ≤ Di i

Many points in a NTMD will not satisfy Many points in a NTMD will not satisfy all multiple-component constraintsall multiple-component constraints

We need a method to generate NTMDs We need a method to generate NTMDs of specified size of specified size NN that satisfies that satisfies allall constraintsconstraints

GeneratingGenerating N N -point NTMDs with -point NTMDs with MCCsMCCs

1.1. Generate NTMDs in Generate NTMDs in TTqq(a,b) of size (a,b) of size N N * > * > N N using one or more of the six NT-methods.using one or more of the six NT-methods.

2.2. Remove points that do not satisfy the Remove points that do not satisfy the MCCs yielding a NTMD T*MCCs yielding a NTMD T*

3.3. Consider only those NTMD T* designs that Consider only those NTMD T* designs that contain exactly contain exactly NN points points

4.4. Generate an evaluation set satisfying all Generate an evaluation set satisfying all constraintsconstraints

5.5. Calculate msd(T*) and/or md(T*) for each Calculate msd(T*) and/or md(T*) for each modified NTMD and comparemodified NTMD and compare

Generation of NTMD T* with MCC: Generation of NTMD T* with MCC: xx11- x- x2 2

≥ ≥ 00

Generation of NTMD T* with MCC: Generation of NTMD T* with MCC: xx11- x- x2 2

≥ ≥ 00

5. Eight Component MCC 5. Eight Component MCC Example: Example: Koons (Technometrics 1989)Koons (Technometrics 1989)

xx11: Earthy hematite ore : Earthy hematite ore 00 ≤≤ x x1 1 ≤ .45≤ .45

xx22: Specular hematite ore : Specular hematite ore 00 ≤ ≤ xx2 2 ≤ .90≤ .90

xx33: Flue dust : Flue dust 00 ≤ ≤ xx3 3 ≤ .35≤ .35

xx44: BOF slag : BOF slag 00 ≤ ≤ xx4 4 ≤ .20≤ .20

xx55: Mill scale : Mill scale 0 0 ≤ ≤ xx5 5 ≤ .30≤ .30

xx66: Dolomite : Dolomite .04 .04 ≤≤ xx66 ≤ .08≤ .08

xx77: Limestone : Limestone .06 .06 ≤≤ xx7 7 ≤ .12≤ .12

xx88: Coke : Coke .029 .029 ≤≤ xx88 ≤ .072≤ .072

Eight Component MCC Eight Component MCC Example Example (cont.) (cont.)

5 Multiple Component Constraints5 Multiple Component Constraints

1.1. 0 0 ≤ -≤ -xx1 1 + .5x+ .5x2 2

2.2. xx33 + x + x44 + x + x5 5 ≤ .35≤ .35

3.3. 0 ≤ 0 ≤ xx11 + x + x22 - - xx33 - x - x44 - x - x55

4.4. .46 .46 ≤≤ .6x .6x11 + .6x + .6x22 + .35x + .35x33 + .2x + .2x44 + .7x + .7x55

5.5. .043 ≤ .043 ≤ .17.17 xx33 + .85x + .85x8 8 ≤ .085≤ .085

20-Point, 8-component NTMDs20-Point, 8-component NTMDs

Number of designs evaluated Number of designs evaluated

1.1. LP method: 1630LP method: 1630

2.2. SRS method: 120SRS method: 120

3.3. PR method: 168PR method: 168

4.4. CF method: 133 CF method: 133

5.5. H1 method: 120H1 method: 120

6.6. H2 method: 84H2 method: 84

Evaluation set : Evaluation set : 100,000100,000 points points

√√msd values for the 20 Best msd values for the 20 Best DesignsDesigns

Method: 1=LP 2=SRS 3=PR 4=CF 5=H1 Method: 1=LP 2=SRS 3=PR 4=CF 5=H1 6=H26=H2

√√msd values msd values (enlarged plot)(enlarged plot)Method: 1=LP 2=SRS 3=PR 5=H1Method: 1=LP 2=SRS 3=PR 5=H1

md values for the 20 Best Designsmd values for the 20 Best DesignsMethod: 1=LP 2=SRS 3=PR 4=CF 5=H1 Method: 1=LP 2=SRS 3=PR 4=CF 5=H1

6=H26=H2

md values md values (enlarged plot)(enlarged plot)Method: 1=LP 2=SRS 3=PR 5=H1Method: 1=LP 2=SRS 3=PR 5=H1

Best Designs for Each MethodBest Designs for Each Method

Method √msd Method md 2. SRS .06719 3. PR .14942 1. LP .06724 1. LP .15309 3. PR .06741 5. H1 .15436 5. H1 .06762 2. SRS .15481 4. CF .07060 4. CF .16522 6. H2 .09567 6. H2 .23176

6. Final Comments6. Final Comments

NTMD approach can provide designs NTMD approach can provide designs in high-dimensional constrained in high-dimensional constrained regions with MCCs.regions with MCCs.

Other assessment criteria may be Other assessment criteria may be developed for the mixture problem.developed for the mixture problem.

May be able to tweak NTMD points to May be able to tweak NTMD points to improve msd(T*) or md(T*).improve msd(T*) or md(T*).

Appendix: From Fang and Yang (2000)Appendix: From Fang and Yang (2000)

Let G(u,d,Φ,Δ,k) = Δ { 1 - [ u(1-Φ)k + (1-u)(1-d)k ]1/k } Let Δk = 1 – ( uk+1 + … + uq ) and Δq = 1

dk = max{ak /Δk , 1- (b1+…+bk-1)/Δk}

Φk= max{bk /Δk , 1- (a1+…+ak-1)/Δk}

where a=(a1, …, aq) and b=(b1, …, bq) are the lower and upper component limits

If u2,…uq is (q -1)-tuple of Unif(0,1) deviates, then

( y1, y2, … , yq ) is a random sample from the uniform

distribution on Tq(a,b) where

yk = G(uk, dk, Φk, Δk, k-1) k =q, q -1,…,2

y1 = 1 – ( y2 + … + yq ).

Selected References:Selected References:

1. Fang, K.-T. & Wang Y. (1994) Number Theoretic Methods in Statistics, Chapman and Hall, London.

2. Fang, K.-T. & Yang, Z.-H. (2000) “On Uniform Design of Experiments with Restricted Mixtures and Generation of Uniform Distribution on Some Domains.”, Stat. and Prob. Letters, 46: 113-120.

WebsiteWebsite

This PowerPoint presentation can be This PowerPoint presentation can be found at my website:found at my website:

www.math.montana.edu/~jobo/ppt/www.math.montana.edu/~jobo/ppt/index.htmlindex.html