Adaptive Response Surface Methodology a-RSM - 2000 - Global Optimization - Kriging - Juillen - Technical Report

Pour action : M. Tourbier 64230 TCR RUC 4 31 M. Biaggio 64230 TCR RUC 4 45 M. Egee 64230 TCR RUC 4 29 M. Buchou 64230 TCR RUC 4 30 M. Blain 64260 TCR DIA 0 02 M. Caquot 12250 TCR PLU 1 10 M. Awade 64130 TCR LAB 0 35 M. Arai Nissan M. Torigaki Nissan Pour info : M. Guimier* 64230 TCR RUC 4 47 M. Hebrard* 64260 TCR RUC 4 24 M. Moussy* 64140 TCR LAB 0 50 M. Dornez* 65301 TCR AVA 1 08 M. Bonnet* 64260 TCR RUC 4 05 M. Bonte* 64230 TCR RUC 4 45 M. Lory* 64230 TCR RUC 4 29 M. Nedelec* 64230 TCR RUC 4 30 Interne : M. Demonsant 64230 TCR RUC 4 31 Pour archivage : Mme Hery 64230 * : people who has the tiny version of the document. The technical report (61 pages) is available on request.

DR / OPTIMISATION PRODUIT PROCESS Groupe Qualit Statistique et Optimisation

L. JUILLEN

Tel : (01.34.9)5.74.60 Fax : (01.34.9)5.77.73

Service 64230 e-mail: [email protected]

API : TCR RUC 4 31

H10.400 Optimisation de pices mcaniques

Le 08/10/2000

Technical Report n64230/H1/2000.040

Thme Theoretical reference manual for Adaptive Response Surface Methodology (A-RSM) software Objectif Your information Attente Remarks and questions

Summary ConclusionsThis technical report contains the description of a new global optimization strategy (A-RSM) build to solve engineering design problems in a numerical simulation context : - we are dealing with expensive black-box finite element computations, - computer model functions are sometimes numerically noisy or have a small number digits precision, - the global number of numerical experiments is hardly limited by designers lead time, - we need methods that make search more global than existing methods, to achieve a better solution. Our Matlab software code can be easily linked with external finite element solvers and is available from the author. We have already used it with some success on several real-world design optimization problems : - Mass minimization of a crash energy absorbtion device for car/truck crash compatibility purposes (TR n:

64230/H1/2000.041) - Optimization of the paint thickness distribution for an electrostatic paint process (TR n: 64230/H1/2000.060) - Calibration of CFD Wave models (TR n: 64230/H1/2000.056) In the near future, we will continue to apply our method to engineering design problems that would come up to us. We will embed our algorithm in the open software architecture iSight, that will become the main tool used for Renault and Nissan s optimization applications (TR n64230/H1/2000.057). Lionel Juillen

Page 1

Rsum - Conclusions Cette note technique contient la description dune nouvelle mthode doptimisation globale (A-RSM) conue pour rsoudre des problmes de conception dans un contexte li la simulation numrique : - nous avons souvent faire des simulations lments finis trs couteuses et de type bote noire - les modles numriques possdent souvent du bruit numrique et leur prcision nest que de quelques dcimales, - le nombre maximum de simulations numriques auquel nous avons droit est trs limit par les dlais trs courts des

concepteurs qui veulent un temps de rponse trs bref, - nous avons besoin de mthodes qui explorent dune manire plus globale lespace des paramtres de conception

que des mthodes classiques, dans le but dobtenir de plus grands gains. Notre logiciel crit en langage Matlab peut facilement tre interfac avec des logiciels de calcul par lments finis, et est disponible sur demande auprs de lauteur. Nous lavons dj utilis avec succs pour des problmes doptimisation industriels : - minimisation de la masse dun dispositif anti-encastrement (DPEA) pour RVI (Note technique n:

64230/H1/2000.041) - optimisation de la distribution dpaisseur de peinture pour un procd de dpose lectrostatique (Note technique :

n: 64230/H1/2000.060) - Identification de modles numriques en mcanique des fluides (Note technique n: 64230/H1/2000.056) Nous projetons de continuer appliquer notre mthode dautres problmes de conception qui nous seraient soumis. Nous projetons dintgrer la mthodologie A-RSM dans le logiciel iSight, qui va bientt devenir loutil principal pour Renault et Nissan, pour les applications doptimisation (note n: 64230/H1/2000.057). Lionel Juillen

Page 2

Introduction In the automotive industry, as well as many others, there is a growing emphasis on designing products using mathematical and/or computer models. Computer models make easier the exploration of alternative designs and reduce the need for expensive prototypes. This approach is often made difficult by the fact that computer models runtimes are very long. Designing optimization algorithms that take into account such practical constraint with a maximum efficiency is a great challenge. In a such context, we can remark that:

- we are dealing very often with expensive black-box finite element solvers - computer model functions are sometimes numerically noisy or have a small number digits precision - function gradients are generally not available - the global number of numerical experiments is limited by designers lead time - we need methods that make search more global than classical methods (descent ones for example), to achieve a better solution

In this technical report, a new approach based on an adaptive response surface methodology (A-RSM) (a bayesian procedure for statisticians) has been designed and tuned for engineering optimization applications with deterministic computer models. This research work was mainly inspired by many researchers : Trosset M.W., Torczon V., Schonlau M., Welch W.J., Koehler J.R., Owen A.B., Jones D.R. and Sacks J., and many others. Several real-world design problems have been recently solved with our A-RSM algorithm. The results obtained by Mr. Lidon (GECI company) to design an automotive mechanical part are described in technical report number 64230/H1/2000.041. The design of a electrostatic paint process hast just been done (technical report 64230/H1/2000.060). Our methodology is also embedded in a software tool used to calibrate some CFD Wave models (technical report 64230/H1/2000.056). The basic idea of our optimization algorithm is to build iteratively cheaper surrogate models of the expensive black-box solvers involved in the simulation of the physical phenomenums, and to find at each iteration the optimum of a simplified optimization problem. The surrogates are numerical approximation whose approximation quality get higher with the number of numerical experiments. A similar idea is also used by Yves Tourbier at Renault (Reseach Division Sce 64230), within his StatOpt Project for Diesel engine calibration purposes. In our optimization procedure, kriging metamodels are used to approximate wealthy function outputs. We start this report by fastly describing general metamodeling (a metamodel is a model of a model) techniques, and specially kriging statistical model and the way to compute its parameters is also investigated. In the second part we describe the adaptive procedure and defining a merit function that balances the search between a local and a more global one. Then, we describe three main ways to use the A-RSM software : non-iterative and iterative building of kriging metamodels, global unconstrained optimization and global constrained optimization.

1) Metamodeling techniques Building approximations of simulation codes involves the following steps : - choosing an experimental design to sample the computer code - choosing a model to represent the data - fitting the model to the observed data - validating the model (Optional but risky if not done) There are a lot of variety of options for these steps as it can be seen on the next figure (taken from [3])

In our case, the model class implemented in A-RSM software is kriging one. Plans of experiments can be classical ones (Factorial, Central Composite, Orthogonal arrays) or space filling designs (random, latin hypercube or maximin designs) 2) Kriging metamodels Kriging is based on a semi-parametric model which allows much more flexibility than parametric models, since no specific model structure has to be used. Kriging postulates a combination of a polynomial model (linear regression parametric model) and departures (non-parametric) of the following form:

y(x) = f(x) + Z(x) (response = linear model + departure)

where : y(x) is the unknown function of interest, f(x) is a known polynomial function of x, and Z(x) is the realization of a gaussian stochastic process with mean zero, variance , and non-zero covariance.

2

The f(x) term is similar to the polynomial model in a response surface, providing a global model of the design space (a trend). In many cases f(x) is simply taken to be a constant term, but in A-RSM procedure, the user can take f(x) as a linear, linear with interactions, pure quadratic or complete quadratic polynomials. While f(x) globally approximates the design space, Z(x) creates localized deviations so that the kriging model interpolates the ns sampled data points. We suppose Z(x) depends on the distances between points too.

Page 4

The covariance matrix of Z(x) which dictates the local deviations follows:

Cov[Z(xi),Z(xj)] = R([R(x2 i,xj)]), where R is the correlation matrix, and R(xi,xj) is the correlation function between any two of the ns sampled data points xi and xj. R is a ns x ns symmetric, positive definite matrix, with ones along the diagonal. The correlation function R(xi,xj) is specified by the user and quantifies how quickly and smoothly the function moves from point xj to point xi and determines how the metamodel fits the data. For A-RSM, R(xi,xj) has been choosed to be :

=

= dv kp

kk

n

k

dji exxR

1

.),(

ndv is the number of design variables, ( )kk p, are the unknown correlation parameters used to fit the model, and dk = xki- xkj which is the distance between the kth components of sample points xi and xj. pk=2 is a common choice for R(xi,xj). There are others correlation functions forms, see reference [3] for more details. Once a correlation function has been selected, predicted estimates, , of the response, y(x), at untried values of x are given by :

)( xy

).().().()( 1 FyVxrxfxy tt += , where y is a vector of length ns which contains the values of the response at each sample point. We have : [ ]tnt sxxRxxRxxRxr ),(,),,(),,()( 21 L= , with and RV z .

2= [ ] [ ]),( jiij xxRR = We define matrix as , F

=

)(

)( 1

st

t

xf

xfF M

we have : ).()( xfxf t= and ).().()( 1 FyRxrxZ t =

The mean-square error (MSE) for the prediction (or process variance) is

[ ]

=)()(

.0

.)()()( 22xrxf

VFF

xrxfxt

ttz

As is an interpolator, we have conditions satisfied for every sampled points. )( xy 0)(2 =kxThe kriging metamodel belongs to the Best Linear Unbiased Predictor family (BLUP). The prediction depends on the parameters )( xy k and pk in the covariance or correlation function R. Assuming that the stochastic process Z(x) is gaussian, k and pk parameters can be estimated by maximum likelihood, together with and z . We have to identify the metamodels parameters that will give us the Best prediction by maximizing the likelihood ([14]). To achieve this goal, we have to maximize the following function :

MLE = , )))ln(det()ln(.( 2 Rn zs + with respect to k and pk.

s

t

z nFyRFy )..(.).(

12 =

, and yRFFRF t ...)..( 111 =

z , and correspond to the maximum likelihood estimators of z and respectively. The optimization problem of maximizing the likelihood function is a hard global one.

Page 5

In our case, we have decided to approximate the solution by a genetic algorithm (other global optimization algorithm could be used, Alienor algorithm for example (see reference [17])) Exactly strictly Solving likelihood optimization problem is not necessary. We found that a quite good solution is enough to build a correct kriging metamodel. Multiple values of the square root of the mean square error at a location x can provide an approximate bound on the actual prediction error of the kriging metamodel at x. Mean square error is high when prediction points are far away from any experimental design points. This distance notion is measured in terms of the weighted distance in the correlation function. For example, )x(MSE.2)x(y is an envelope that gives 95% of confidence on the model. This really means that for location x, if the unknown funtion is truly generated by a random function with the postulated trend ( ) and correlation function R(x,x).(xf t j), then approximatively 95% of the kriging models ),( kk p that go through the observed design points would lie in that envelope (see reference [10]). Of course, the real unknown function will not be a random function, but we can make the strong hypothesis that approximated error function will not be far from the real one in the case of continuous and differentiable functions. We see kriging approximating models as a way to expand a polynomial functions basis with an exponential one, dependant of the datas (sampling points). To be careful, we must insert these ideas and hypotheses in an adaptive context, and that is the object of the A-RSM procedure.

Page 6

3) A-RSM Software Basic principles As we said in the introduction, A-RSM is an iterative procedure whose basic algorithm is described in the following flow chart:

4 Initial Plan of experiments 4 Build an approximation interpolating the initial

experiments. 4 for i = 1 to maximum number of experiments

+ Make a new experiment at a location that minimize a merit function : weighted combination of the surrogate model and error approximation + Build an approximation that interpolate the augmented plan of experiments + Update the relative weights of the merit function to balance between reducing approximation error and minimizing the statistical model (kriging).

4 end loop i 3.1) Merit Function To balance between a global search and local search, at each iteration we are minimizing a merit function that represents a trade-off between increasing the model precision and minimizing the objective approximation function. Our choice is :

)(2.2)(.1)( xMSEwxywx =

What we want is to obtain the smallest value of the objective at a cost less or equal to our global budget (of function evaluations). At the beginning of the procedure, we need to improve drastically the quality of our model, thats why we give a value to w2 greater than w1. Then at the end of each iteration, we decrease w2 and increase w1, to give gradually more importance to minimizing y(x). Initial w1 is equal to 0 and initial w2 is equal to 1. We have a maximum number of experiments limited to Nf_Maxi function evaluations. 3.2) Unidimensionnal example We want to minimize the univariate function f(x) on the range [-3.1,2.1] :

)3.()2.()2.(.0001.0)( 22 ++= xxxxxf (you can find the graph of this function on the next picture (figure 1)).

Page 7

-3 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5x 10

-3 Fonction reelle

Figure 1 : graph of the example function

The purpose of this example, is to show quickly on a simple problem, how the A-RSM methodology works. We can see that this function admits three local optima. The global one is located in the interval [-1,-0.5]. Our objective is to find the global optimum location with a budget limited to ten function evaluations. As we suppose, we do not have a-priori knowledge on the objective function, we have decided to start the optimization with only two experiments (x1 = 0.5 and x2 = 1). For this example, we choose pk=2. On the next pages are shown the metamodels (in blue) build at each iteration, with the real function (in red) and the envelop (in green).

Page 8

Figure 2 : first DOE 2 experiments

We begin with very small information about the function

Figure 3 : first iteration nexp = 3

We found the third point to be far from the firsts. It makes the kriging metamodel more interesting even if we still lack information to achieve a good approximation.

Page 9

Figure 4 : 2nd iteration nexp = 4

Figure 5 : 3rd iteration nexp = 5

We found the right basin of attraction for the global optimum of the function.

Page 10

Figure 6 : 4th iteration nexp = 6

Now, points begin to clusterize.

Figure 7 : iteration 5 nexp = 7

Page 11

Figure 8 : 6th iteration nexp = 8

Figure 9 : 7th and last iteration nexp = 10

Remarks on the A-RSM execution : We can see on the last figures that as the number of experiments increases, the quality of the approximation gets better. We can remark, as it is stated in the A-RSM algorithm definition, the points added at each iteration tends to clusterize. It is seen with the last three iterations. It is a

Page 12

drawback : the last metamodel is not especially the best one (as compared with the true function in red color), but it is the price we have sometimes to pay to seek the optimum location. With this simple example, we remark that the A-RSM heuristic finds fastly the basin of attraction of the global minimizer of the objective function, and even with a small number of experiments (10) the global optimum search has been successfull. The main purpose of this small example is to show how A-RSM algorithm works and underlying idea : a global search becoming progressively a local one. This basic principle is the same for multidimensionnal optimization problems. 4) Building a Plan of experiments (Matlab API : Read Appendix B) We need an initial plan of experiments for the first phase of the optimization procedure. It helps us to make a first simplified global exploration of the design space and to build the trend of the initial approximation model. There are many ways to achieve this task. Some of them can be taken from the classical theory of design of experiments (cf. [18]) : full and fractional factorial designs, Central Composite Designs, Orthogonal Arrays and Optimal designs (A-, D-, G-, etc.). 4.1) Full Factorial sampling A good picture worths better than a lot of words, you can see on the following drawings (figure 10) two full factorial plans for respectively two and three design parameters with five and three levels.

Figure 10 : examples of full factorial samplings

4.2) Latin Hypercube sampling A Latin Hypercube plan belongs to the space-filling family of plan of experiments. It is a matrix of ns rows and ndv columns. Each column contains the levels 1,2,,ns randomly permuted, and the ndv are matched at random to form a Latin Hypercube. These designs are the earliest space filling experimental design intended for computer experiments. Due to their randomness nature, it is quite impossible to obtain the same Latin Hypercubes even with the same parameters. The basic algorithm to generate a Latin hypercube sampling is as follows (taken from [19]) : each of the ndv parameters (or factors) is associated with a random value between 0 and 1. A sample of size ns is formed by selecting the ith realization of the jth factor as :

s

jjj n

uiv

+= 1)( , where :

Page 13

- (.).(.),....,(.), 21 k are permutations of the integers 1,,ns, sampled randomly, independently, and with replacement from the set of n! possible permuations; and )(ij is the ith element of the jth permutation.

- The uj are sampled independently from [0,1]. On the following figures, you can see examples of Latin Hypercube samplings for 2 and 4 design parameters respectively (figures 11 and 12).

2 1.5 1 0.5 0 0.5 1 1.5 22

1.5

1

0.5

0

0.5

1

1.5

2Plan initial....

Figure 11 : Latin Hypercube Sampling on [-2.048,2.048]2

2 0 22

0

2

2 0 22

0

2

x1

x2

2 0 22

0

2

x2

x1

2 0 22

0

2

x1

x3

2 0 22

0

2

x3

x1

2 0 22

0

2

x1

x4

2 0 22

0

2

x4

x1

2 0 22

0

2

2 0 22

0

2

x2

x3

2 0 22

0

2

x3

x2

2 0 22

0

2

x2

x4

2 0 22

0

2

x4

x2

2 0 22

0

2

2 0 22

0

2

x3

x4

2 0 22

0

2

x4

x3

2 0 22

0

2Plan initial....

Figure 12 : Latin Hypercube Sampling on [-2.048,2.048]4

Remark : Latin Hypercube samplings have stratification properties similar to Orthogonal Arrays : it means and we can see it for the 4D example (above), that the projections on subspaces show a nice spreading of the point set. It can be useful for example in the case we want to use it to reduce the number of factors. 4.3) MaxiMin design sampling (for mathematical formulations, read Appendix A) This sampling design type is caracterized by the fact that we want to maximize the minimum distance (defined by a vectorial norm) between any two sample points, thus spreading the points out as much as possible in the design space. Our motivation to examine this kind of sampling is that several authors ([3],[10]) has proven that the quality of the kriging models is improving as the experiments fill the parameter space.

Page 14

On the following figures, here are examples of maximin designs in 2- and 4-dimensional spaces (figures 13 and 14).

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 13 : Maximin design 2 parameters, 20 experiments

0 0.5 10

0.5

1

0 0.5 10

0.5

1

x1

x2

0 0.5 10

0.5

1

x2

x1

0 0.5 10

0.5

1

x1

x3

0 0.5 10

0.5

1

x3

x1

0 0.5 10

0.5

1

x1

x4

0 0.5 10

0.5

1

x4

x1

0 0.5 10

0.5

1

0 0.5 10

0.5

1

x2

x3

0 0.5 10

0.5

1

x3

x2

0 0.5 10

0.5

1

x2

x4

0 0.5 10

0.5

1

x4

x2

0 0.5 10

0.5

1

0 0.5 10

0.5

1

x3

x4

0 0.5 10

0.5

1

x4

x3

0 0.5 10

0.5

1

Figure 14 : Maximin design 4 parameters, 40 experiments

(symetric with respect to the diagonal)

Page 15

5) Some ways to use A-RSM software (Matlab API : Appendix A) In this section we will look more precisely at different ways to use our A-RSM software tools. The first one, is to build approximation models of expansive numerical models and use these models for several manipulations (visualization, integration, etc.). The other way is to optimize an objective function with or without constraints. We make a Matlab code (ARSMexample.m in Appendix D) showing many different ways to use A-RSM software. In the following paragraphs, we used this piece of Matlab code to generate the results and the pictures. A-RSM can be used to build approximation models and solving global unconstrained optimization problems. Constrained ones can be solved by the mean of a penalization function. We will use as main example the Rosenbrock function, it has a quite bad well known behavior. For visualization convenience, we restrict our examples to 2 dimensional space problems. 5.1) Building response surface 5.1.1 - One-shot construction The plan of experiments is limited to twenty-five points. On the following figures, we tried a Maximin design, a random design, a latin Hypercube design and a full factorial plan with 5 levels per factor. For each trials, we drawn the final metamodel, the quadratic trend, the Mean Square Error and the True Error between the kriging metamodel and the analytical Rosenbrock function. With a very refined grid, we plot metamodel versus real function on figures called scatter plot. If the kriging approximation is good, all points should be close to the line y=x. In this study, we are showing the effects of the choice of the initial plan of experiments (maximin design, latin hypercube design, random design and full factorial design) on the quality of the approximation. We used A-RSM too, to build adaptively an approximation model by minimizing the approximation error at each step.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

x1

x2

500

1000

1500

2000

2500

3000

3500

Figure 15 : maximin design Rosenbrock function

Page 16

5.1.1.1) Maximin design

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

3500

Figure 16 : maximin design kriging metamodel

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Tendance du Metamodele

-500

0

500

1000

1500

2000

2500

Figure 17 :maximin design quadratic trend

The point pattern is almost symetric, so the trend is found symetric too with respect to the vertical axis.

Page 17

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2MSE metamodele

2

4

6

8

10

12

14

16

18

20

22

Figure 18 :maximin design Mean Square Error (MSE)

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2abs(Erreur absolue)

10

20

30

40

50

60

Figure 19 : Maximin design True absolute Error

We can see with figures 18 and 19, that Mean Square error prediction is a quite good approximation of the true difference between the kriging metamodel and the Rosenbrock function. Mean square error prediction gives us the area of maximum error (absolute values).

Page 18

Figure 20 : Maximin design - scatter plot

Scatter plot confirms the fact that the approximation obtained is fairly good. All the points of a refined grid are close to the first diagonal line. 5.1.1.2) Latin Hypercube design

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

Figure 21 : latin hypercube sampling kriging metamodel

As the pattern point is less regular and spread, the approximation is not as good as maximin designs.

Page 19

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


0

500

1000

1500

2000

Figure 22 : latin hypercube sampling quadratic trend

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2MSE metamodele

50

100

150

200

250

Figure 23 : latin hypercube sampling MSE

Page 20

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


200

400

600

800

1000

1200

1400

1600

Figure 24 : latin hypercube sampling True Absolute Error

Mean square error prediction gives us the general trend of the true error.

Figure 25 : Latin Hypercube sampling - scatter plot

The metamodel is good for a lot of points, and underestimates the Rosenbrock function on some areas.

Page 21

5.1.1.3) Random design

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

0

500

1000

1500

2000

2500

Figure 26 : random sampling kriging metamodel

Random design does not possess the geometric properties of latin hypercube sample and maximin designs, so the kriging metamodel is not as good as the others.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


0

500

1000

1500

2000

2500

Figure 27 : random sampling quadratic trend

The point pattern is not symetric, so the quadratic trend too.

Page 22

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2MSE metamodele

50

100

150

200

250

Figure 28 : random sampling MSE

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


200

400

600

800

1000

1200

1400

1600

1800

Figure 29 : random sampling True absolute Error

Mean square error prediction is bad. It is close to 0, near the experiments, but we dont really find a close shape of the true error.

Page 23

Figure 30 : random sampling - scatter plot

The kriging metamodel underestimates and overestimates the true function, on large portions of the design parameter space. 5.1.1.4) Full factorial design

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

3500

Figure 31 : Full factorial design (5 levels) kriging metamodel

Page 24

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


-500

0

500

1000

1500

2000

2500

Figure 32 : Full factorial design (5 levels) quadratic trend

The point pattern is symetric, so trend and metamodel show the same structure.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2MSE metamodele

50

100

150

200

250

300

350

400

450

500

Figure 33 : Full factorial design (5 levels) MSE

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


50

100

150

200

250

300

350

Figure 34 : Full factorial design (5 levels) True Absolute Error

Page 25

The mean square error prediction is biased by the regularity of the point pattern. But it successes in finding some band structure in the true error.

Figure 35 : Full factorial design (5 levels) scatter plot

The approximation obtained is fairly good. We dont overestimate too much. We dont underestimate too much too. 5.1.2) Adaptive Construction : In this section, we present results obtained by using A-RSM software to sequentially improve the quality (reducing the maximum mean square error) of the kriging metamodel. We will start with a small number of experiments (10). Our global budget will be limited to 25 experiments, and we will let A-RSM methodology build the best metamodel. Like in the previous sections, we take a full quadratic trend.

Page 26

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

0

500

1000

1500

2000

2500

3000

3500

Figure 36 : maximin design (10) kriging metamodel

Ten points is not enough to build a good approximation of the Rosenbrock function !

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2MSE metamodele

50

100

150

200

250

300

350

400

450

Figure 37 : maximin design (10) MSE

Page 27

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


200

400

600

800

1000

1200

1400

Figure 38 : maximin design (10) True absolute error

Figure 39 : maximin design (10) : scatter plot

It confirms that our starting approximation is not a good one. We need to improve it !

Page 28

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

3500

Figure 40 : A-RSM (25) kriging metamodel

The final result looks good. A lot of points has been added on the borders of the design space. It means that to build a good approximation, it must be first good on the borders.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2MSE metamodele

10

20

30

40

50

60

Figure 41 : A-RSM (25) - MSE

Page 29

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5


10

20

30

40

50

60

70

80

Figure 42 : A-RSM (25) True absolute error

The final Mean square error prediction looks like the true error distribution and with approximatively the same order of magnitude.

Figure 43 : A-RSM (25) scatter plot

We have a good final approximation.

Page 30

5.1.3) Conclusions on building kriging approximations We examined several one-shot methods wich differ only on the design of experiment type and an adaptive method to build an approximation of a black-box function. For one-shot methods, using Maximin designs and Latin Hypercube designs seems to be the best choices. Full factorial design is not bad, but it becomes too expensive for a great number of parameters and levels. Maximin designs are better than Latin Hypercube sampling but are considerably harder to find (look at appendix A). If you are in a hurry, Latin Hypercube sampling is the best compromise between speed and quality of approximation. Adaptive construction was found to be nearly as good as one-shot maximin design method. But the sequential process was a bit slower. Adaptive construction is the best choice when we dont have the choice. When we are obliged to start with a small amount of experiments, or if we begin with an existing sampling. 5.2) Unconstrained global optimization Our A-RSM software tries to solve directly the global unconstrained optimization problem, given a limited budget of experiments. Again, the 2D Rosenbrock function will be used. We will start with a small number of experiments (10). Our global budget will be limited to 25 experiments

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

0

500

1000

1500

2000

2500

3000

3500

Figure 44 : initial metamodel (10 maximin design)

We tried A-RSM methodoly to minimize Rosenbrock function with an amount of 25, 30, 35 and 40 experiments.

Page 31

With 25 experiments, we obtained the following final metamodel (figure 45) :

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

3500

Figure 45 : A-RSM final metamodel budget : 25

With 30 experiments, we obtained (figure 46) :

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

3500

Figure 46 : A-RSM final metamodel (budget = 30)

Page 32

With a budget of 35 experiments :

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

500

1000

1500

2000

2500

3000

3500

Figure 47 : A-RSM final metamodel budget = 35

Points tends to be located along the narrow valley. And with a budget of 40 experiments :

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

0

500

1000

1500

2000

2500

3000

3500

Figure 48 : A-RSM final metamodel (budget = 40)

Points are more and more in the valley, and they tend to make a cluster in the area of the optimum ([ 1 1]).

Page 33

Obtained results are collected in the next spreadsheet :

Budget Minimum obtained Xoptimum 25 0.14210075272397 1.26878085463878

1.63623544636559 30 0.12302666156750 0.85731909196360

0.70295404984306 35 0.10461430741770 1.30212529286577

1.70707783859490 40 0.00229023035194 1.01304450410719

1.02165474414081

25 30 35 400

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

Budget

Min

imum

val

ue o

btai

ned

Convergence

Figure 49 : Convergence of the algorithm

From appendix C, we already know that the true optimum of the Rosenbrock function is located at [ 1 1 ] and that its optimal value is 0. In this case, we can see that as we increase the budget, the A-RSM procedure best point gets closer to the optimal location (figure 49). Now we compare the relative efficiency of our method to a classical SQP method (Sequential Quadratic Programming from Matlab Optimization Toolbox) by limiting the maximum number of function evaluations of the SQP algorithm. Obtained results are collected in the next spreadsheet :

Budget Minimum obtained Xoptimum 25 0.63237333845480 0.20542085573013

0.04538727644189 30 0.54577720628813 0.29421444849716

0.06473464492223 35 0.33547639050801 0.44037028611572

0.17899582685641 40 0.24058283370509 0.51641572310708

0.25848210157977 (starting point = [-2 1], f = 901);

Page 34

We can now draw on the same figure (figure 50), for each method, the results obtained versus the cost of search (Number of evaluation functions only).

20 30 40 50 60 70 80 90 100 110 1200

0.1

0.2

0.3

0.4

0.5

0.6

0.7Unconstrained Rosenbrock function SQP=Blue A-RSM=green

Global budget

Opt

imum

val

ue o

btai

ned

Figure 50: SQP vs A-RSM on unconstrained Rosenbrock function For the SQP method, we reported the last point very close to the solution of the problem. We did not reported CPU time, because we suppose evaluation function time is very long (it is not the case for the Rosenbrock function, but imagine crash finite element analyses instead of it). A-RSM method seems to converge faster than SQP method, but we do not know if this method would reach the same level of precision with the same amount of function evaluations. We can partially conclude that A-RSM is better when the number of function evaluations is rather limited. 5.3) Constrained global optimization Solving the general constrained global optimization problem with our software needs to transform it into an unconstrained one. Several basic techniques exists. We will show the most simple one wich involves the use of a penalization function (or merit function) :

Page 35

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

x1

x2

1000

2000

3000

4000

5000

6000

7000

Figure 51 : True penalized function with initial maximin design (10)

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

1000

2000

3000

4000

5000

6000

7000

Figure 52 : A-RSM final metamodel obtained 25 experiments

Page 36

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

0

1000

2000

3000

4000

5000

6000

7000

Figure 53 : A-RSM final metamodel 30 experiments

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

1000

2000

3000

4000

5000

6000

7000


Page 37

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

1000

2000

3000

4000

5000

6000

7000


-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Metamodele Final

1000

2000

3000

4000

5000

6000

7000

Figure 56 : A-RSM final metamodel - 50 experiments

Page 38

Minimum obtained Xoptimum Norm(Xoptimum)

25 0.86015889910006 0.72523220920637 0.43738063285649

0.84691414870003

30 0.14738770748001 0.66357357061054 0.45882446797913

0.80675261141762

35 0.44168559739016 0.75368953233007 0.62977446547076

0.98217299316510

40 0.08312147663122 0.71215485959505 0.50553159647533

0.87334228059779

50 0.09290510254356 0.69812568228276 0.48316402184951

0.84901527681931

The solution of the problem is the point that minimize the Rosenbrock function on a unity radius circle. So the distance of the solution from the origin (norm(Xoptimum)) must be close to 1.

25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Constrained Rosenbrock function A-RSM

Global budget

Opt

imum

val

ue o

btai

ned

Figure 57 : Minimum value obtained for the constrained Rosenbrock optimization problem

25 30 35 40 45 500

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2Constrained Rosenbrock function A-RSM

Global budget

Dis

tanc

e to

the

con

stra

int

boun

dary

Figure 58 : Distance to the constraint boundary

Page 39

We can see that we get closer to the solution as we increase the budget, but the convergence to the minimum is not monotone. We solved a transformed version of the constrained problem via a penalization function that helps us to combine two opposite objectives. We have to minimize the Rosenbrock function and minimize the constraint violation together. As the functions are highly nonlinear, we can see a strange behavior of the convergence of the algorithm on figures 57 and 58 : when the real objective function is minimized, the distance to the boundary is increased. We suggest that the convergence is not monotonous, and may be if we would have made additionnal optimizations with greater budgets (e.g.: 60, 70,), we would have seen the Distance to the constraint boundary becoming closer to 0. We can compare the relative efficiency of our method to a classical SQP method (from Matlab Optimization Toolbox) by limiting the maximum number of function evaluations of the SQP algorithm. Obtained results are collected in the next spreadsheet :

Budget Minimum obtained Xoptimum Norm(Xoptimum) 25 0.64561595718386 0.19796274012057

0.03433930735380 0.20091897497644

30 0.62484498672076 0.22291646955880 0.03520514705707

0.22567931845845

35 0.55402471171105 0.30302374026579 0.06569891490627

0.31006408141629

40 0.46207156712246 0.37159444144307 0.11216370573736

0.38815348226716

50 0.13865390401953 0.62812620386744 0.39263521634420

0.74074620559175

(starting point = [-2 1], f =2428.86 );

We can now draw on the same figure (figure 59), for each method, the results obtained versus the cost of search (Number of evaluation functions only).

20 30 40 50 60 70 80 90 100 1100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Constrained Rosenbrock function SQP=Blue A-RSM=green

Global budget

Opt

imum

val

ue o

btai

ned

Figure 59: A-RSM vs SQP Constrained Rosenbrock function Page 40

We did not reported CPU time, because we suppose evaluation function time is very long. The final result obtained by SQP method is drawn on the picture. A-RSM has a faster convergence in the [30,..,60] budget range for this problem, than SQP algorithm. The convergence of A-RSM is not monotonous (but its theory does not assume it) and as the budget is increasing (after 50), the convergence would be as slow as the SQP method. 6) Conclusions In this report we gave many examples (appendix D) to show how to use A-RSM Matlab software and its components (DOE, visualization, optimization, approximation). We tried to present a new global optimization method dedicated to a certain class of problems whose properties are : - small to moderate number of design parameters (

Bibliography [1] Jones D.R., Schonlau M., Welch W.J. Efficient Global Optimization of expensive black-box functions Journal of Global Optimization, 1998 [2] Schonlau M. Computer experiments and Global Optimization PhD thesis, University of Waterloo, 1997 [3] Simpson T. A concept exploration method for product family design PhD thesis, Georgia Institute of Technology, 1998 [4] Torczon V., Trosset M.W. Using approximations to accelerate engineering design optimization 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and Optimization pp 738-748, 1998 [5] Torczon V., Trosset M.W. Numerical optimization using computer experiments Technical report 97-02, Department of Computational & Applied Mathematics, Rice University 1997 [6] Trosset M.W. Optimization on a limited budget Proceedings of the section on physical and engineering sciences, American Statistical Association, 1998 [7] Booker A.J. Examples of surrogate modelling of computer simulations 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and Optimization pp 118-128, 1998 [8] Powell M.J.D. Recent research at Cambridge on radial basis functions Research Report DAMTP 1998/NA05 Cambridge University, 1998 [9] Guttman H.M. A radial basis function for global optimization International workshop on Global Optimization, Florence, Italy, 1 October 1999 [10] Koehler J.R., Owen A.B. Computer experiments in: Ghosh, Rao, Eds., Handbook of Statistics, Volume 13, Elsevier Science, 1996

[11] Barthelemy P. , Haftka R.T. Approximation concepts for optimum structural design A review Structural Optimization, Vol 5, pp 129-144, 1993 [12] Cox D.D., John S. SDO : a statistical method for global optimization in: Alexandrov, Hussaini, Eds., Multidisciplinary Design Optimization: State of the Art, SIAM 1997 [13] Sacks J., Welch W.J., Mitchell T.J., Wynn H.P. Design and analysis of computer experiments Statistical Science, Vol 4, pp 409-435, 1989 [14] Walter E., Pronzato L. Identification de modles paramtriques partir de donnes exprimentales Masson, 1994 [15] Costa J.P., Pronzato L., Thierry E. A comparison between kriging and radial basis function networks for nonlinear prediction in Proc. NSIP99 (IEEE EURASIP Workshop on nonlinear Signal and Image Processing), paper N:155 - 1999 http://www.i3s.unice.fr/~pronzato/biblio.html [16] Zhigljavsky A.A. Theory of global random search Kluwer Academic Publishers, 1991 [17] Delbauve A. La transformation rductrice ALIENOR. Application loptimisation globale et aux calculs dintgrales multiples Rapport de stage effectu chez Renault (DR Service 64230) - Universit dOrsay - 2000 [18] Myers R.H., Montgomery D.C. Response Surface Methodology Process and Product Optimization using designed experiments John Wiley & Sons 1995 [19] Gentle J.E. Random number generation and Monte Carlo methods Springer-Verlag 1998 [20] Trosset M.W. Approximate Maximin distance designs Proceedings of the Statistical Computing Section, ASA - 1999

Appendix A : MaxiMin Designs Mathematical Formulations

We suppose we have ndv factors and ns sampling points. The global number of variables nvar for this problem is equal to ndv times ns. With dist(xi,xj) the distance beetween points xi and xj Basic Maximin Formulation:

Min ( Max(dist(xi,xj) ),

subject to : vlb xi,xj vub

This problem is a nonsmooth one and is very difficult to solve due to the nonconvexity of the objective function. It needs algorithms like simulated annealing, genetic algorithms or random searches. Formulation 1 ([20]): We add an additional artificial variable r.

Maximize r (or minimize r),

subject to : 2r - dist(xi,xj) 0 vlb xi,xj vub

This formulation gives us a smooth objective function but with additional complexity from an extra parameter and a lot of nonlinear constraints. The number of these constraints is equal to :

2)1.( varvar nn

So, it grows rapidly with ndv and ns. We suppose that there are a lot of constraints active at the optimum and that the feasible domain may not ne convex, so standard optimization algorithms based on KKT optimality conditions and Newton iterations may be trapped in local optima. Michael Trossets Formulation ([20]):

Minimize >

ji

jiij xx )( with )11log()( 2ttij +=

subject to : vlb xi,xj vub

This formulation gives us an unconstrained optimization problem with the same number of variables as Formulation 1. It avoids nonlinear constraints, even if the objective function stays strongly nonlinear. The objective function used here can be seen as an approximation (or relaxation) of the normL . From a practical point of view, we take in the range [8,15].

Appendix B : Matlab Software API Page 44

Sampling points generation procedures : Full Factorial plan : The Matlab procedure call to build a full factorial plan of experiments is :

[plan,tab] = LJfullfact(Nb_parameters,Nb_levels)

This function returns the variable plan as the full factorial plan with values in [1,1]. The variable tab contains the corresponding levels. Nb_parameters is the number of variables (ndv) Nb_levels is a vector whose component i is the number of level of the parameter i (if it is a scalar, all parameters has the same number of levels). Latin Hypercube Sampling : The Matlab procedure call to build a Latin Hypercube Sampling is :

[Design] = latinhc(Nb_Experiments,Nb_parameters,lower,upper) Design is a Nb_Experiments by Nb_parameters matrix whose lines represent the experiments. Nb_Experiments is the number of points (ns) Nb_parameters is the dimensionnality of the design space (ndv) lower and upper are vectors (size : Nb_Parameters) filled respectively with lower and upper bounds of the design parameters.

Maximin Design : The plan_MAXIMIN2 Matlab function is a general procedure to compute Maximin designs : [plan_maximin,maximin] = plan_MAXIMIN2( ,

Nb_Experiments, Nb_parameters, lower, upper, init, funRestraint, method, penalization, itermax)

plan_maximin is the maximin design matrix Nb_Experiments is the number of points (ns) Nb_parameters is the dimensionnality of the design space (ndv) lower and upper are vectors (size : Nb_parameters) filled respectively with lower and upper bounds of the design parameters. init is a vector of 2 elements that must be set to : [ 1 0 ] (for example : init = [ 1 0 ] ). funRESTRAINT is the name of a Matlab function that will return the values of a constraints set equations (useful only in the case of nonlinearly constrained design space). Set it to [] in the unconstrained case. method : number corresponding to the mathematical optimization algorithm used. We recommend : method = 7 (random search on basic maximin formulation (Appendix B)), method = 12 (constr (Matlab Optimization toolbox SQP) with formulation 1) method = 14 (constr with TROSSET formulation)

penalization : penalization coefficient. Set it to a great value in the case of constrained design space (e.g. : 1e4), otherwise 0. itermax : maximum number of iterations for the used algorithm Remark : Note that the third method is several orders slower than the first twos. A-RSM matlab procedure call description (metam.m):

function [xopt,fxopt,mseopt,... plan_final,y_final,... trend_final,theta_final,... sigma2_final,Rinv_final,... Fn_final,beta_final,bet_final] = ...

metam(plan,y0,vlb,vub,funobj,indf,indg, ..., gradfobj,indgradf,indgradg,... methode,strategie,merite,options, ... funARRET,funRESTRAINT,restart,racine,... varargin);

% INPUT : % % plan : plan of experiments % y0 : values of the function for each experiments % vlb : lower bounds for the design parameters % vub : upper bounds for the design parameters % funobj : name of the matlab function for the objective function evaluation % indf : indf(i) = 1 : evaluation of f(i) is requested % else indf(i)= 0 % indg : indg(i) = 1 : evaluation of g(i) is requested % else indg(i)= 0 (NOT USED ! = indg = []) % gradfobj : NOT USED ! (gradfobj = ) ; % indgradf : NOT USED ! indgradf = [] ; % indgradg : NOT USED ! indgradg = [] ; % methode : 'weight' % strategie : 'pareto' % merite : 1 = metamodel + MSE % 2 = metamodel + relative MSE % 3 = metamodel + Maximin % 4 = metamodel % 5 = MSE % 6 = Maximim distance % % options(1) = maximum number of iterations % options(2) = maximum number of funobj evaluations % options(3) = precision requested for the stopping rule % options(4) = 4 % options(5) = polynomial trend of the kriging metamodel % 0 = constant % 1 = linear without interactions % 2 = linear with interactions % 3 = pure quadratic % 4 = full quadratic % options(6) = choose the number of theta parameters % 0 : one parameter theta , p = 2 % 1 : one parameter theta per dimension, p = 2 % 2 : one parameter theta and one exponant p per dimension % options(7) = Optimization used to maximize the MLE function % 0 : SQP (constr) Matlab Optimization Toolbox % 1 : GAOT genetic algorithm % 2 : Improved Nelder-Mead algorithm type: SIMPS % 9 : ossrs (random search type algorithm) % options(8) = Optimization method used to minimize the merit function % 1 : GAOT genetic algorithm % options(10) : tolerance for the SVD decomposition of the covariance matrix (1e-14)

% options(20) = 0 % options(21) = number of points to draw the real function % options(22) = number of points to draw the kriging metamodel % options(23) = number of contour levels % options(24) = current iteration (set it to 0 if restart not equal to 1) % % restart = 1 : if we need to restart from a precomputed iteration (options(24), else set it to 0 (normal mode) % % % % % OUTPUT : % % xopt : location of the best minimum value found % fxopt : best mimimum value found % mseopt : MSE value at xopt % plan_final : final plan of experiments % y_final : values of the function for plan_final % % the following outputs are used only in the case we need to % rebuild the final kriging metamodel ( look at save_metamodel and % load_metamodel functions ) % % trend_final % theta_final % sigma2_final % Rinv_final % Fn_final % beta_final % bet_final % Basic 2D visualization functions function [error,ifig] = visplan2D2(vlb,vub,plan); % % visualization of a 2D points set % % INPUT : % vlb : lower bounds vector % vub : upper bounds vector % plan : points locations (plan(i,1) = X(i), plan(i,2) = Y(i)) % % OUTPUT : % error : error message % ifig : list of figure pointers % function [error,ifig] = visplan2DC(titre,vlb,vub,plan,ftab,nbc,np,xopt,ix,iy,funfun,varargin); % % Visualization of a 2D points set with contours from an external function % % INPUT : % titre : title string of the figures % vlb : lower bounds vector % vub : upper bounds vector % plan : points locations % ftab : function values vector % nbc : number of contour levels % np : number of points for each dimension for computations of % contour levels % xopt : reference location (generally optimum location) % ix : index parameter number for X % iy : index parameter number for Y % funfun : Matlab function needed for contour levels computation

% varargin : list of additional parameters for the funfun function % % OUTPUT : % error : error message % ifig : list of figure pointers Basic n-D visualization functions : function [error,ifig] = visplanND(titre,vlb,vub,plan); % % Visualization of a n-D point set % % INPUT : % titre : title string of the figures % vlb : lower bounds vector % vub : upper bounds vector % plan : coordinates of points % % OUTPUT : % error : error message % ifig : list of figure pointers % function [error,ifig] = visplanNDC(titre,vlb,vub,plan,ftab,nbc,np,xopt,funfun,varargin); % % Visualization of a n-D point set with contour levels for a specified external % function % % INPUT : % titre : chaine de caractere % vlb : valeur mini des variables, si vlb == [] le sous-prog prend le min % vub : valeur maxi des variables % plan : tableau contenant les coordonnees des points % ftab : tableau de valeurs de la fonction % nbc : nombre de contours % % OUTPUT : % error : % ifig : % Input-Output metamodel functions function [ierr] = save_metamodele(nom_fichier,METAM); % % % INPUT : % nom_fichier : name of the file to save the metamodel in % METAM : structure type variable whose fields are % datas needed to evaluate a kriging metamodel % % OUTPUT : % ierr : error message function [METAM] = load_metamodele(nom_fichier) % % % INPUT : % nom_fichier : datafile to load % % OUTPUT : % METAM : metamodel loaded (structure) Other functions :

function [methode,strategie,merite,options] = metam_options(nvar,nexp) ; % % Initialization of default parameters used by metam function % % INPUT : % nvar : number of parameters % nexp : number of experiments % % OUTPUT : % methode : % strategie : parameters used by metam % merite : % options : function [ftab,conttab,grad_ftab,grad_conttab,erreur] = generic_batch(plan,funobj,gradobj,indf,indg,indgradf,indgradg,varargin); % % Automatic execution of a lot of evaluation functions % This function launches sequentially funobj and gradobj on % a set of points (plan) % % INPUT : % plan : design of experiments % funobj : function to evaluate % gradobj : gradient function % indf : index of the objective function to evaluate % indg : index of the constraint functions to evaluate % indgradf : index of the objective function gradients to evaluate % indgradg : index of the constraint function gradients to evaluate % varargin : additionnal parameters for funobj and/or gradobj functions % % OUTPUT : % ftab : values of the objective functions % conttab : values of the constraints % grad_ftab : values of the objective function gradients % grad_conttab : values of the constraint function gradients % erreur : error message % function [f,tend,noise]= eval_krigmodel(xx) ; % %eval_krigmodel.m % %This function file evaluates the kriging metamodel and returns the values % of the metamodel itself (f), the trend part (tend) and the noise part. % % WARNING ! : xx must be normalized in [0,1] % The following global variables must be initialized before using eval_krigmodel % function % global x y beta Rinv theta Fn trend sigma2 bet metPARAM; % % INPUT : % xx : point where we want to evaluate the metamodel % % OUTPUT : % f : value of the metamodel at xx % tend : trend value at xx % noise : noise value at xx %

Appendix C : Rosenbrock function

F(x) = =

+2

n

1i

21i2

21i2i2

dv

))x1()xx.(100(

[ ] dvn048.2,048.2x There exists only one optimum located at ( ) dvn1,.,1,...,1 , the minimum value of the function is equal to 0. On the following picture, we drawn the Rosenbrock function for 2 and 4 parameters.

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2

-1.5

-1

-0.5

0

0.5

1

1.5

2Fonction reelle

500

1000

1500

2000

2500

3000

3500

Rosenbrock function 2 variables

Rosenbrock function 4 variables

Page 50

We used the SQP method implementation from MATLAB Optimization Toolbox to find the solution and it gave for the 2D Rosenbrock function, with forward finite difference gradients : f-COUNT FUNCTION MAX{g} STEP Procedures 3 2509 0 1 6 461.76 0 1 9 233.829 0 1 13 0.772339 0 0.5 18 0.664555 0 0.25 21 0.642115 0 1 24 0.632373 0 1 27 0.588289 0 1 30 0.545777 0 1 33 0.493587 0 1 36 0.335476 0 1 39 0.240583 0 1 43 0.199519 0 0.5 46 0.0892429 0 1 50 0.0698957 0 0.5 53 0.0403858 0 1 57 0.0327867 0 0.5 60 0.0223813 0 1 63 0.00840244 0 1 66 0.00319637 0 1 69 0.00303542 0 1 72 0.000644356 0 1 75 0.000219762 0 1 Hessian modified 78 2.24883e-005 0 1 Hessian modified 81 1.53686e-006 0 1 Hessian modified 84 4.96639e-009 0 1 Hessian modified 87 3.73017e-012 0 1 Hessian modified twice 104 3.05854e-012 0 -6.1e-005 Hessian modified twice 119 3.05732e-012 0 -6.1e-005 Hessian modified twice Optimization Converged Successfully No Active Constraints Optimum = x = 1.00000030537696 1.00000043858955 Optimum value of the function ans = 3.057315616996783e-012 Number of function evaluations ans = 119

The constrained Rosenbrock function problem is as follows :

Minimize Rosenbrock function F(x) = =

+2

n

1i

21i2

21i2i2

dv

))x1()xx.(100(

Subject to : 01x)x(g2/1n

1i

2i

dv

=

=

With [ ] dvn048.2,048.2x We solve it with the SQP method : f-COUNT FUNCTION MAX{g} STEP Procedures 3 2509 1.23607 1 7 28.3498 -0.475451 0.5 12 3.19575 -0.765158 0.25 15 0.741385 -0.840748 1 18 0.706189 -0.838414 1 21 0.704594 -0.837305 1 Hessian modified 24 0.688777 -0.822903 1 27 0.665528 -0.796984 1 30 0.624792 -0.747467 1 33 0.561365 -0.692827 1 36 0.414019 -0.591794 1 39 0.297447 -0.498391 1 43 0.257299 -0.379071 0.5 46 0.176777 -0.321301 1 49 0.134228 -0.140957 1 52 0.0823377 -0.124507 1 55 0.0491848 -0.00628552 1 58 0.0457367 9.33923e-006 1 61 0.0456748 1.02737e-007 1 Hessian modified 62 0.0456748 1.03162e-012 1 Hessian modified Optimization Converged Successfully Active Constraints: 1 Optimum = x = 0.78641516511815 0.61769829858456 Optimum value of the function ans = 0.04567480871935 Number of function evaluations ans = 62

Page 52

Now we solve the unconstrained formulation of our constrained problem via a penalization function (look at constrained_rosen_fun.m MATLAB code page 57 for the mathematical formulation). Matlab execution gives us : f-COUNT FUNCTION MAX{g} STEP Procedures 3 4036.86 0 1 7 28.3498 0 0.5 12 3.91421 0 0.25 15 0.754423 0 1 18 0.648972 0 1 21 0.647771 0 1 Hessian modified 24 0.645616 0 1 Hessian modified 27 0.639032 0 1 30 0.624845 0 1 33 0.597786 0 1 36 0.554025 0 1 39 0.462072 0 1 42 0.323804 0 1 45 0.244906 0 1 48 0.138654 0 1 52 0.11845 0 0.5 55 0.0831644 0 1 59 0.0620863 0 0.5 64 0.0498578 0 0.25 72 0.0483572 0 0.0313 81 0.0475846 0 0.0156 92 0.0475287 0 0.00391 95 0.0459494 0 1 98 0.0456651 0 1 Hessian modified 102 0.0456601 0 0.5 Hessian modified 105 0.0456601 0 1 Hessian modified 109 0.0456601 0 0.125 Hessian modified Optimization Converged Successfully No Active Optimum =

Constraints

x = 0.78648418351974 0.61780709301725 Optimum value of the function ans = 0.04566005301450 Number of function evaluations: ans = 109

Matlab code for Rosenbrock function :

(rosen_fun.m) function [f,g,no,ni,ne] = rosen_fun(x,indf,indg,varargin); % %Rosenbrock Function % % INPUT : % x : variables % indf : indf(i) = 1 : evaluation of f(i) is requested % else indf(i)= 0 % indg : indg(i) = 1 : evaluation of g(i) else indg(i) = 0 % % OUTPUT : % f : objective functions % g : constraints % no : number of objective functions % ni : number of inequality type restraints % ne : number of equality constraints % no = 1 ; % number of objectives ni = 0 ; % number of inequality restraints ne = 0 ; % number of equality restraints % nvar = length(x) ; % fvec = zeros(1,2*nvar); % if (isempty(indf)|(no==0)); f = []; else if mod(nvar,2) == 1; error('number of variables must be even !'); else for i = 1:(nvar/2) ; fvec(2*i-1)=10*(x(2*i)-x(2*i-1)^2); fvec(2*i)=1-x(2*i-1); end; end; f = sum(fvec.^2) ; end; if (isempty(indg)|((ni+ne)==0)); g = []; else g = 0 ; end;

Matlab code for the Rosenbrock function gradients:

(gradrosen_fun.m) function [df,dg,varargout] = gradrosen_fun(fun,x,indgradf,indgradg,varargin); % %Rosenbrock function Gradients % % INPUT : % fun : objective function Matlab procedure % x : variables % indgradf : indgradf(i) = 1 means compute grad f(i) (look at indf) % indgradg : indgradg(i) = 1 means compute grad g(i) (look at indg) % % OUTPUT : % df : objective functions gradients with respect to x % dg : constraints gradients % nvar = length(x) ; % no = 1 ; ni = 0 ; ne = 0 ; % fvec = zeros(1,2*nvar); if (isempty(indgradf)); df = []; else if mod(nvar,2) == 1; error('number of variables must be even !'); else dfvec = zeros(2*nvar,nvar); for i = 1:(nvar/2) ; fvec(2*i-1)=10*(x(2*i)-x(2*i-1)^2); fvec(2*i)=1-x(2*i-1); dfvec(2*i-1,2*i-1)= dfvec(2*i-1,2*i-1)+2*fvec(2*i-1)*(-20*x(2*i-1)); dfvec(2*i-1,2*i) = dfvec(2*i-1,2*i) + 2*fvec(2*i-1)*10 ; dfvec(2*i,2*i-1) = dfvec(2*i,2*i-1) + 2*fvec(2*i)*(-1) ; dfvec(2*i,2*i) = dfvec(2*i,2*i) + 0 ; end; for i = 1:nvar ; df(i,:) = sum(dfvec(:,i)); end; end; end; % if (isempty(indgradg)); dg = []; else dg = 0 ; end; %

Matlab code for the Unconstrained version of the Constrained Rosenbrock optimization problem

(constrained_rosen_fun.m) The constrained optimization problem :

Minimize Rosenbrock function F(x) = =

+2

n

1i

21i2

21i2i2

dv

))x1()xx.(100(

Subject to : 01x)x(g2/1n

1i

2i

dv

=

=

With [ ] dvn048.2,048.2x Is changed to :

Minimize ,0.(max(1000)x(F)x( += g(x)))2With [ ] dvn048.2,048.2x

function [f,g,no,ni,ne] = constrained_rosen_fun(x,indf,indg,varargin); % no = 1 ; ni = 0 ; ne = 0 ; % nvar = length(x) ; % fvec = zeros(1,2*nvar); % if (isempty(indf)|(no==0)); f = []; else if mod(nvar,2) == 1; error('le nombre de variables doit etre pair !'); else for i = 1:(nvar/2) ; fvec(2*i-1)=10*(x(2*i)-x(2*i-1)^2); fvec(2*i)=1-x(2*i-1); end; end; g = norm(x) - 1 ; f = sum(fvec.^2) + 1000*(max(g,0))^2; end; if (isempty(indg)|((ni+ne)==0)); g = []; else g = 0 ; end;

Page 56

Appendix D : A-RSM examples (ARSMexample.m)

% % HOW TO USE A-RSM !!!! % % ARSMexample.m % format long ; warning off; % prompt={'Number of parameters',... 'Name of the Matlab black box function',... 'Name of the session log file',... 'Initial number of experiments',... 'Maximum number of experiments',... 'Maximum number of iterations',... 'Trend',... 'Stopping rule value',... 'Number of points per parameters for drawing the real function',... 'Number of contour levels' } ; def={'2','rosen_fun','rosen_function.out','10','25','15','4','0.001','100','50'}; titre='Data Input'; lineNo=1; answer=inputdlg(prompt,titre,lineNo,def); nvar = str2num(answer{1}); funobj = answer{2}; logfile = answer{3}; nexp = str2num(answer{4}); nexp_maxi = str2num(answer{5}); itmaxi = str2num(answer{6}); % Trend % 0 = constant, 1 = pure linear, 2 = linear with interactions, 3 = pure quadratic, 4 = full quadratic trend = str2num(answer{7}); precision = str2num(answer{8}); np = str2num(answer{9}); nbc = str2num(answer{10}); if(isempty(np)) np1 = 100 ; else np1 = np ; end; % diary(logfile); % % boundary constraints for the parameters % vu = 2.048 ; %vu = 10 ; vl = -vu ; low_var = vl*ones(1,nvar); up_var = vu*ones(1,nvar); % % Selection of the initial plan of experiment type % plan_str = {'maximin','latinHS','random','full_factorial_5levels'}; [s,v] = listdlg('PromptString','Choose a plan:',... 'SelectionMode','single',... 'ListString',plan_str); choix_plan = plan_str{s} ; % if(strcmp(choix_plan,'maximin')); disp('MaxiMin design'); init =[ 1 100 ]; mplan = 12 ; p0 = 1000 ; iterMX = 1 ; [plan,maximin_dist] = plan_MAXIMIN2(nexp,nvar,low_var,up_var,... init,'',mplan,p0,iterMX); elseif (strcmp(choix_plan,'latinHS')); disp('Latin Hypercube Sampling'); [plan] = latinhc(nexp,nvar,low_var,up_var); elseif (strcmp(choix_plan,'random'));

disp('Random design'); [plan] = LJunifrnd(low_var,up_var,nexp,nvar); elseif (strcmp(choix_plan,'full_factorial_5levels')); disp('Full Factorial design - 5 levels per factors'); nb_nivo = 5 ; [plan,index_exp] = LJfullfact(nvar,nb_nivo); [nexp,nvar] = size(plan) ; for j = 1:nvar ; plan(:,j) = ( (plan(:,j)+1).*((up_var(j)-low_var(j))/2) )+low_var(j); end; elseif(strcmp(plan_str{s},'')); disp('Latin Hypercube Sampling'); [plan] = latinhc(nexp,nvar,low_var,up_var); end; % % visualization of the sampling points % nfig = [] ; if (nvar == 2); [err,ifig]=visplan2D2(low_var,up_var,plan) ; else [err,ifig]=visplanND('',low_var,up_var,plan) ; end; nfig = [ nfig ifig ]; % gradobj = [] ; indf = 1 ; indg = []; indgradf = [] ; indgradg = [] ; y = generic_batch(plan,funobj,gradobj,indf,indg,indgradf,indgradg) ; % % we visualize the shape of the function near its optimum % % you have to change it for another function x_optimum = ones(1,nvar); % optimum location for Rosenbrock function if(np>0); titre = [ 'Function ' funobj ]; if (nvar > 2); [err,ifig] = visplanNDC(titre,low_var,up_var,plan,y,nbc,np,x_optimum,funobj,1,[]); else [err,ifig] = visplan2DC(titre,low_var,up_var,plan,y,nbc,np,x_optimum,1,2,funobj,1,[]); end; end; nfig = [ nfig ifig ]; % % Part 1 - Building kriging metamodel with an initial plan of experiments % disp('Building kriging metamodel with an initial plan of experiments'); indf = 1 ; indg = []; gradfobj = ''; indgradf = []; indgradg = []; methode = 'weight'; strategie = 'pareto' ; merite = 5 ; % options(20 = 1 == trace debug pendant les iterations options = [ 0 nexp_maxi precision 2 trend 1 2 1 1e-3 1e-14 10 0 0 0 0 0 0 0 0 2 np np1 nbc 0 ]; funARRET = '' ; % no stopping rule funRESTRAINT = '' ; % no restraints restart = 0 ; racine = 'ARSM_example_part1' ; % [xopt,fxopt,msefinal,... planfinal,yfinal,... trendfinal,thetafinal,... sigma2final,rinvfinal,... Fnfinal,betafinal,betfinal] = ... metam(plan,y,low_var,up_var,funobj,... indf,indg,gradfobj,indgradf,indgradg,... methode,strategie,merite,options,... funARRET,funRESTRAINT,restart,racine); % % saving the results in a structure % METAM(1).xopt = xopt ; METAM(1).fxopt = fxopt ; METAM(1).mse = msefinal ; METAM(1).plan = planfinal ; METAM(1).y = yfinal ; METAM(1).trend = trendfinal ; METAM(1).theta = thetafinal ; METAM(1).sigma2 = sigma2final ; METAM(1).rinv = rinvfinal ; METAM(1).fn = Fnfinal ; METAM(1).beta = betafinal ; METAM(1).bet = betfinal ; METAM(1).metPARAM = options(6) ; METAM(1).low_var = low_var ;

METAM(1).up_var = up_var ; %%keyboard % % Part 2 - Building kriging metamodel by increasing the quality of an initial plan of experiments % disp('Building kriging metamodel by increasing the quality of an initial plan of experiments'); indf = 1 ; indg = []; gradfobj = ''; indgradf = []; indgradg = []; methode = 'weight'; strategie = 'pareto' ; merite = 5 ; options = [ itmaxi nexp_maxi precision 2 trend 1 2 1 1e-3 1e-14 10 0 0 0 0 0 0 0 0 2 np np1 nbc 0 ]; funARRET = '' ; % no stopping rule funRESTRAINT = '' ; % no restraints restart = 0 ; racine = 'ARSM_example_part2' ; % [xopt,fxopt,msefinal,... planfinal,yfinal,... trendfinal,thetafinal,... sigma2final,rinvfinal,... Fnfinal,betafinal,betfinal] = ... metam(plan,y,low_var,up_var,funobj,... indf,indg,gradfobj,indgradf,indgradg,... methode,strategie,merite,options,... funARRET,funRESTRAINT,restart,racine); METAM(2).xopt = xopt ; METAM(2).fxopt = fxopt ; METAM(2).mse = msefinal ; METAM(2).plan = planfinal ; METAM(2).y = yfinal ; METAM(2).trend = trendfinal ; METAM(2).theta = thetafinal ; METAM(2).sigma2 = sigma2final ; METAM(2).rinv = rinvfinal ; METAM(2).fn = Fnfinal ; METAM(2).beta = betafinal ; METAM(2).bet = betfinal ; METAM(2).metPARAM = options(6) ; METAM(2).low_var = low_var ; METAM(2).up_var = up_var ; keyboard % % Part 3 - unconstrained global optimization with a global limited budget % disp('unconstrained global optimization with a global limited budget'); indf = 1 ; indg = []; gradfobj = ''; indgradf = []; indgradg = []; methode = 'weight'; strategie = 'pareto' ; merite = 1 ; options = [ itmaxi nexp_maxi precision 2 trend 1 2 1 1e-3 1e-14 10 0 0 0 0 0 0 0 0 2 np np1 nbc 0 ]; funARRET = '' ; % no stopping rule funRESTRAINT = '' ; % no restraints restart = 0 ; racine = 'ARSM_example_part3' ; % [xopt,fxopt,msefinal,... planfinal,yfinal,... trendfinal,thetafinal,... sigma2final,rinvfinal,... Fnfinal,betafinal,betfinal] = ... metam(plan,y,low_var,up_var,funobj,... indf,indg,gradfobj,indgradf,indgradg,... methode,strategie,merite,options,... funARRET,funRESTRAINT,restart,racine); METAM(3).xopt = xopt ; METAM(3).fxopt = fxopt ; METAM(3).mse = msefinal ; METAM(3).plan = planfinal ; METAM(3).y = yfinal ; METAM(3).trend = trendfinal ; METAM(3).theta = thetafinal ; METAM(3).sigma2 = sigma2final ; METAM(3).rinv = rinvfinal ; METAM(3).fn = Fnfinal ; METAM(3).beta = betafinal ; METAM(3).bet = betfinal ; METAM(3).metPARAM = options(6) ; METAM(3).low_var = low_var ; METAM(3).up_var = up_var ;

keyboard % % Part 4 - constrained global optimization with a global limited budget % % the constrained formulation is changed to an unconstrained one via a penalization function % funobj = 'constrained_rosen_fun' ; gradobj = [] ; indf = 1 ; indg = []; indgradf = [] ; indgradg = [] ; y2 = generic_batch(plan,funobj,gradobj,indf,indg,indgradf,indgradg) ; % % we visualize the shape of the function near its optimum % % you have to change it for another function x_optimum = ones(1,nvar); % optimum location for Rosenbrock function if(np>0); titre = [ 'Function ' funobj ]; if (nvar > 2); [err,ifig] = visplanNDC(titre,low_var,up_var,plan,y2,nbc,np,x_optimum,funobj,1,[]); else [err,ifig] = visplan2DC(titre,low_var,up_var,plan,y2,nbc,np,x_optimum,1,2,funobj,1,[]); end; end; nfig = [ nfig ifig ]; disp('constrained global optimization with a global limited budget'); funobj = 'constrained_rosen_fun' ; indf = 1 ; indg = []; gradfobj = ''; indgradf = []; indgradg = []; methode = 'weight'; strategie = 'pareto' ; merite = 1 ; options = [ itmaxi nexp_maxi precision 2 trend 1 2 1 1e-3 1e-14 10 0 0 0 0 0 0 0 0 2 np np1 nbc 0 ]; funARRET = '' ; % no stopping rule funRESTRAINT = '' ; % no restraints restart = 0 ; racine = 'ARSM_example_part4' ; % [xopt,fxopt,msefinal,... planfinal,yfinal,... trendfinal,thetafinal,... sigma2final,rinvfinal,... Fnfinal,betafinal,betfinal] = ... metam(plan,y2,low_var,up_var,funobj,... indf,indg,gradfobj,indgradf,indgradg,... methode,strategie,merite,options,... funARRET,funRESTRAINT,restart,racine); METAM(4).xopt = xopt ; METAM(4).fxopt = fxopt ; METAM(4).mse = msefinal ; METAM(4).plan = planfinal ; METAM(4).y = yfinal ; METAM(4).trend = trendfinal ; METAM(4).theta = thetafinal ; METAM(4).sigma2 = sigma2final ; METAM(4).rinv = rinvfinal ; METAM(4).fn = Fnfinal ; METAM(4).beta = betafinal ; METAM(4).bet = betfinal ; METAM(4).metPARAM = options(6) ; METAM(4).low_var = low_var ; METAM(4).up_var = up_var ; keyboard % % Part 5 - saving metamodels..... % disp('SAVING KRIGING METAMODELS !'); ficsave = 'ARSMmetamodels' ; save_metamodele(ficsave,METAM) ; % disp('LOADING KRIGING METAMODELS !'); [NEW_METAM]=load_metamodele(ficsave); nb_metamodels = length(NEW_METAM) ; % number of metamodels in the structure % % Part 6 - use of pre-built kriging metamodels % global x y beta Rinv theta Fn trend sigma2 bet metPARAM; % % loop over all the metamodels % for ii = 1:nb_metamodels; [nn,ll] = size(NEW_METAM(ii).plan);

as = ones(nn,1)*NEW_METAM(ii).low_var; bs = ones(nn,1)*NEW_METAM(ii).up_var ; x= (NEW_METAM(ii).plan-as)./(bs-as); y = NEW_METAM(ii).y ; beta = NEW_METAM(ii).beta ; Rinv = NEW_METAM(ii).rinv ; theta= NEW_METAM(ii).theta ; Fn = NEW_METAM(ii).fn ; trend= NEW_METAM(ii).trend ; sigma2= NEW_METAM(ii).sigma2 ; bet= NEW_METAM(ii).bet ; metPARAM= NEW_METAM(ii).metPARAM ; vlb1 = NEW_METAM(ii).low_var ; vub1 = NEW_METAM(ii).up_var ; % % Visualization of contour levels % xdum = ones(1,ll) ; % xdum = [ 1 1 .... 1 ]; nbc = 50 ; % number of contours np = 100 ; % number of points per dimension for the plots [err,infig] = visplanNDC(titre,vlb1,vub1,NEW_METAM(ii).plan,y,nbc,np,xdum,'eval_krigmodel'); % % evaluation of the metamodel and the MSE at one point (xdum) % xdum_normalized = (xdum-vlb1)./(vub1-vlb1) ; [krig_xdum,trend_xdum,noise_xdum ] = eval_krigmodel(xdum_normalized); % % Minimization of the metamodel ii % % we save it in its own file global METAM_FILE ; % METAM_FILE = strcat('ARSMmetamodel_',num2str(ii)) ; % save_metamodele(METAM_FILE,NEW_METAM(ii)) ; % options = foptions ; x0 = 0.5*(vlb1+vub1); indf = 1 ; indg = []; [xopt_metam,options,lambda,hess] = constr('metamodele_fun',x0,options,vlb1,vub1,'',... indf,indg); xopt_metam end; % % The End % diary off ;

2) Kriging metamodels3\) A-RSM Software Basic principles3.1) Merit Function4) Building a Plan of experiments (Matlab API : Read Appendix B)4.1) Full Factorial sampling4.2) Latin Hypercube sampling4.3) MaxiMin design sampling (for mathematical formulations, read Appendix A)

5) Some ways to use A-RSM software (Matlab API : Appendix A)5.1) Building response surface5.2) Unconstrained global optimization5.3) Constrained global optimization

6) Conclusions

BibliographyInput-Output metamodel functions

Documents

Adaptive Response Surface Methodology a-RSM - 2000 - Global Optimization - Kriging - Juillen - Technical Report