Space-Filling Designs for Computer...

Space-Filling Designs for Computer Experiments

V. Roshan Joseph

1

Supported by ARO W911NF-14-1-0024

Computer model

System

Mathematicalmodel

Computermodel

• Computer model is a numerical implementation of the mathematical model.

2

Computer experiments

Computer model outputsinputs ..

.

.

• Expensive black-box code• Deterministic outputs• Complex relationships

Experiment

3

Applications

• Approximation • Sensitivity analysis• Calibration• Optimization• Uncertainty quantification

4

Uncertainty QuantificationUncertainty

sources

Input Output

Propagation of uncertainty

45

Gul, E., Joseph, V. R., Yan, H., and Melkote, S. N. (2015). “Uncertainty Quantification in Machining Simulations Using In Situ Emulator,” Under Review.

Review of Space-Filling Designs

• No need to worry about– Replication– Randomization– Blocking

• What type of designs?– Fractional factorial designs, orthogonal

arrays,…? No, use space-filling designs!

6

Space-Filling Designs

• Definition: designs that fill the space!

• What is the meaning of filling the space?– Maximin distance– Minimax distance– Uniform

7

Maximin distance design

• Johnson, Moore, and Ylvisaker (1991)

8

Minimax distance design

• Johnson, Moore, and Ylvisaker (1991)

9

10

Consider the simple case of four-run design in two factors (n=4, p=2).

Maximin distance design. Minimax distance design.

Examples of Maximin and Minimax Designs

Latin hypercube design

• McKay, Conover, Beckman (1979)

11

Latin hypercube design

12

Maximin Latin hypercube design

• Morris and Mitchell (1995): Maximindistance design within the class of Latin hypercube designs.

where D is an LHD.

13

Uniform design

• Fang (1980)

• Good for approximating integrals by sample averages.

14

Maximum Projection Designs

Joseph, V. R., Gul, E., and Ba, S. (2015). “Maximum Projection Designs for Computer Experiments,” Biometrika, 102, 371-380.

15

MmLHD• MmLHD (20,2)

16

MmLHD• A two-dimensional projection of MmLHD (20,10)

17

MmLHD

• Ensures good space-filling in p dimensions and uniform one-dimensional projections, but their projections in 2,…,p-1 dimensions can be poor.

18

Improvements to MmLHD

• Draguljic, Santner, Dean (2012)

• Criterion is computationally expensive.

19

Uniform Design

• Hickernell (1998) proposed CL2 criterion that ensures projections to all subspaces.

• But is uniformity important in computer experiments?

20

Generalized LHD

• Dette and Pepelyshev (2010): placing more points in the boundaries than around the center can minimize the prediction errors from GP modeling.

21

MaxPro criterion• Weighted Euclidean distance:

• Modify the Morris-Mitchell criterion to

22

Bayesian criterion

• We don’t know about 𝜃𝜃 before the experiment!

• Prior:

• Then, the criterion becomes

23

MaxPro criterion

• MaxPro criterion:

24

LHD property

•• MaxPro design must have n distinct levels

for each factor.• LHD requirement is automatically enforced

in the criterion!

25

Design construction algorithm

• Minimizing MaxPro objective function is not easy!– np number of variables– many local minima

26

R Package: MaxProJMP 12 (under FFF)

ExamplesMaxProLHD(20,2) and MaxPro(20,2)

27

Numerical comparisons• n=100, p=10• Designs:

– MaxPro– MaxProLHD– MmLHD– UD– GLHD

• Criteria:– Maximin– miniMax– CL2

28

Minimum distance (larger-the-better)

29

GP-based criteria

30

An optimality result

• Prior:

• MaxPro design is good in terms of Entropy, condition number, prediction variance, etc.

31

Applications

32

3

Solid End Milling Process• Simulation on computer model of Production Module

software

33

UQ for Solid End Milling

34

MaxPro LHD

35

After Inverse probability transform

36

In-Situ Emulator

37

𝑓𝑓 𝒖𝒖, 𝑡𝑡 = 𝑒𝑒𝑒𝑒𝑒𝑒{log 𝑓𝑓 𝒖𝒖𝟎𝟎, 𝑡𝑡 + �̂�𝛿𝑡𝑡 (𝒖𝒖)}

�̂�𝛿𝑡𝑡(u) = 𝒓𝒓 𝒖𝒖 ′𝑹𝑹−1(𝒘𝒘𝑡𝑡 − 𝒘𝒘0,𝑡𝑡)

𝑅𝑅𝑡𝑡(𝒖𝒖𝑖𝑖 − 𝒖𝒖𝑖𝑖) = 𝑅𝑅𝑡𝑡,𝒖𝒖 𝒖𝒖𝑖𝑖 − 𝒖𝒖𝑗𝑗 − 𝑅𝑅𝑡𝑡,𝒖𝒖 𝒖𝒖𝑖𝑖 − 𝒖𝒖0 − 𝑅𝑅𝑡𝑡,𝒖𝒖 𝒖𝒖𝑗𝑗 − 𝒖𝒖0 + 1,

𝑅𝑅𝑡𝑡,𝒖𝒖 𝒖𝒖𝑖𝑖 − 𝒖𝒖𝑗𝑗 = exp −∑𝑘𝑘=1𝑝𝑝 𝜃𝜃𝑡𝑡,𝑘𝑘 𝑢𝑢𝑖𝑖𝑘𝑘 − 𝑢𝑢𝑗𝑗𝑘𝑘

2.

where

Output with 95% confidence region

38

Computational time

39

UD (n=2000) In-Situ EmulatorCurve cutting 1 day 8 mins5-axis cutting 9 days 1 hour

References• V. Roshan Joseph (2016). “Space-Filling Designs for Computer

Experiments: A Review,” (with discussions and rejoinder), Quality Engineering, 28, 28-44.

• V. Roshan Joseph, Gul, E., and Ba, S. (2015). “Maximum Projection Designs for Computer Experiments,” Biometrika, 102, 371-380.

• Gul, E., V. Roshan Joseph, Yan, H., and Melkote, S. N. “Uncertainty Quantification in Machining Simulations Using In Situ Emulator,” Under review.

40

roshan@gatech.edu