Computers & Industrial Engineering 62 (2012) 927–935

Contents lists available at SciVerse ScienceDirect

Computers & Industrial Engineering

journal homepage: www.elsevier .com/ locate/caie

Responses modeling and optimization criteria impact on the optimizationof multiple quality characteristics

Nuno Costa a,c,⇑, João Lourenço a,b, Zulema Lopes Pereira c

a IPS–Setubal College of Technology, Campus do IPS, Estefanilha, 2910-761 Setúbal, Portugalb INESC-ID, Rua Alves Redol 9, 1000-029 Lisboa, Portugalc UNIDEMI/DEMI, Faculdade de Ciências e Tecnologia-Universidade Nova de Lisboa, 2829-516 Caparica, Portugal

a r t i c l e i n f o

Article history:Received 8 August 2010Received in revised form 14 October 2011Accepted 14 December 2011Available online 22 December 2011

Keywords:DesirabilityLoss functionOLSRobustnessSURVariance

0360-8352/$ - see front matter � 2011 Elsevier Ltd. Adoi:10.1016/j.cie.2011.12.015

⇑ Corresponding author at: IPS–Setubal College ofEstefanilha, 2910-761 Setúbal, Portugal. Tel.: +3265721869.

E-mail addresses: nuno.costa@estsetubal.ips.pt (N.ubal.ips.pt (J. Lourenço), zlp@fct.unl.pt (Z.L. Pereira).

a b s t r a c t

Responses modeling and optimization criteria impact on the optimization results were investigated. TheOrdinary Least Squares and Seemingly Unrelated Regression techniques were illustrated in two examplesfrom the literature and the performance of three optimization criteria evaluated. In contrast to thestandard practice, compromise solutions were evaluated in terms of bias and robustness usingoptimization performance measures. The results of both examples show that responses modelingstrongly impacts on the optimization results, while there is no significant difference between criteriaperformance. The Seemingly Unrelated Regression technique proved to be useful for modeling correlatedresponses. Otherwise, this technique can lead to results in close agreement to those obtained withmodels fitted with the OLS technique.

� 2011 Elsevier Ltd. All rights reserved.

1. Introduction technique and ‘‘standard’’ second order models are fitted to

The variety and quantity of methods available in the literature todeal with optimization of the mean and variance or standarddeviation of multiple quality characteristics (responses) are large.Within the framework of the Response Surface Methodology(RSM), which was comprehensibly exposed by Myers, Montgomery,and Anderson-Cook (2009) and has been increasingly used insimulated and real-life problems (Ilzarbe, Álvarez, Viles, & Tanco,2008), most authors have focused on what to optimize, developingor improving objective functions, in contrast to those who haveproposed new optimization algorithms. In general, those methodsprovide to the user or Decision-Maker (DM) one way to assignweights or priorities to responses. This is a critical issue inmultiresponse optimization because preference parameters aredependent on the particular DM and the context of the problem.Thus, methods that require a minimum amount of preferenceinformation from the DM are especially appealing and their perfor-mance must be evaluated to justify or not their recommendation forsolving real-life problems. Another critical issue that has beenunderestimated is the responses modeling. Most case studiesreported in the literature employ the Ordinary Least Squares (OLS)

ll rights reserved.

Technology, Campus do IPS,51 265790000; fax: +351

Costa), joao.lourenco@estset

responses. However, for problems where significant responsescorrelation exists, which are not unusual in practice, the OLStechnique is not appropriate. Shaibu and Cho (2009) and Goethalsand Cho (2011a) also show that higher-order polynomial modelsmay be more effective in finding better compromise solutions thanthe commonly-used quadratic model.

The objective of this article is to investigate the impact of theresponses modeling and objective functions designed with basis indifferent approaches on the results of multiresponse optimizationproblems developed under the RSM framework. In particular, theSeemingly Unrelated Regression (SUR) is presented as alternativeto OLS technique and the solutions generated with a compromiseprogramming-based method compared with those of twoalternative optimization criteria.

The remainder of the article is organized as follows: Section 2provides a review on the literature; the proposed optimizationcriterion is presented in Section 3; Section 4 overviews the SURand OLS techniques; Section 5 uses two examples from theliterature to evaluate the impact of methods and regressiontechniques on the results; Section 6 discusses the results andSection 7 presents the conclusions.

2. Literature review

Design and conduct experiments are not trivial tasks, especiallyfor non-statistician practitioners who do not have the required

928 N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935

background in the subject. There are technical issues that need tobe carefully managed in addition to the non-technical ones thatare thoroughly discussed by Tanco, Viles, Ilzarbe, and Alvarez(2009). For example, choosing an inappropriate experimentaldesign will surely compromise the results or conclusions. To selectan appropriate modeling technique and fit models to responses areadditional difficulties in multiresponse optimization problems.

Several regression techniques can be used to fit models toresponses. Fogliatto and Susan (2000) summarized the assumptionsfor using four response modeling techniques and give a tutorial onthem. What can be stated from their article is the following:

– For uncorrelated responses: OLS is suitable when the responses’variance is homogeneous, while the Generalized LeastSquares (GLS) is suitable when the responses’ variance isnon-homogeneous;

– For correlated responses: Multivariate Regression (MVR) andSUR are suitable when the responses’ variance is homogeneous.

Shah, Montgomery, and Carlyle (2004) also reviewed the OLSand SUR techniques and presented the proof of their equivalencyin two particular situations, namely, for zero correlations amongall model errors and the same model form for each response withthe same set of design variables. These authors also showed thatthe SUR technique can lead to more precise estimates of theregression coefficients than the OLS technique when responsesare correlated. The OLS is the most-frequently used technique forestimating the response models in problems developed under theRSM framework (Khuri & Mukhopadhyay, 2011), even whenresponses are correlated. Moreover, authors rarely fit models ofdegree higher than two to responses, although diagnostic checksfor models adequacy are usually performed using meaningfulstatistics.

The R-square and adjusted R-square have been used to assessthe quality of description for models fitted with the OLS technique.In practice, high values for R-square and adjusted R-square, sayhigher than 0.9, are desirable. With the SUR technique has beenused the system R-square (Jitthavech, 2010). The predictedR-square and prediction error sum of squares (PRESS), whose valuemust be as low as possible, have been used to assess the quality ofpredictions of models fitted with the OLS technique. These andother statistics are reviewed, for example, by Shaibu and Cho(2009) and Goethals and Cho (2012). The models adequacy check-ing is not limited to these statistics so the reader is referred toMyers et al. (2009) for more details on this issue.

As concerns the optimization criteria, a large variety ofalternatives have been put forward in the literature. The desirabil-ity-based and loss function-based methods are very popular amongpractitioners. The desirability-based methods are easy to use, easy tounderstand, and flexible enough for incorporating the DM’spreferences. Moreover, the most popular desirability-based method,the Derringer and Suich method (1980) with the modificationintroduced by Derringer (1994), is available in several data analysissoftware packages. However, the analyst needs to specify values tofour types of shape parameters (weights) to use the method whenthe optimization problem includes responses of different types,namely: (1) Nominal-The-Best (NTB) – the value of the estimatedresponse is expected to achieve a particular target value;Smaller-The-Better (STB) – the value of the estimated response isexpected to be smaller than an upper bound; Larger-The-Better(LTB) – the value of the estimated response is expected to be largerthan a lower bound. This is not a simple task and it impacts on theoptimal variable settings. Moreover, the composite function valuedoes not provide a clear interpretation except the principle thateither a higher or lower value is preferred, depending on how thecomposite function is defined. For a tutorial on desirability-based

methods the reader is referred to Costa, Lourenço, and Pereira(2011).

Loss function-based methods have also been widely used inpractice, namely those that consider the responses’ variance leveland explore the information on responses correlation. Thesemethods allow the assignment of priorities to individual responsesand the results are expressed in monetary units, which are appealingfeatures. However, different response scales, relative variabilities,and relative costs are difficult to take into account in thespecification of cost coefficients. This is a critical task that seriouslyimpacts on the final results because inaccurate specification ofcost coefficients leads to less favorable compromise solutions(higher expected losses). For an extensive review on lossfunction-based methods the reader is referred to Murphy, Tsui,and Allen (2005).

Other methods that have been successful illustrated in theliterature are those based on Goal Programming (Kazemzadeh,Bashiri, Atkinson, & Noorossana, 2008), Physical Programming(Messac, 2000); Probability-based (Peterson, Miró-Quesada, & DelCastillo, 2009), Lexicographic Weighted Tchebycheff (Shin & Cho,2009), and Non-parametric models (Besseris, 2009).

Besides the theoretical complexity of some methods andunavailability of the algorithms employed, one premise inherentto those which require preference information from the DM is thathe/she is capable of assigning priorities or relative importance toresponses in order to find a satisfactory solution. The fundamentalsignificance of the weights or priorities to responses has beenstudied by Jones (2011), Marler and Arora (2010), Jeong, Kim,and Chang (2005), who also proposed procedures to extract valuesrepresenting the DM’s preferences. However, to the best of ourknowledge, a general procedure for multiresponse optimizationproblems, which ensures that the ‘‘best’’ compromise solution(responses as close as possible to their target values and withminimum variability around them) is achieved, has not been foundyet. In general, those preference parameters do not reflectproportionally the relative importance of the responses, and maynot be promptly available or easily defined. In practice, theymerely locate points in the domain when are varied. Thus, the taskof assigning priorities to responses is, in general, performedthrough a trial-and-error procedure that may be a source of frus-tration and significant inefficiency, particularly when either thenumber of responses or control factors is large. Recently, Lee,Kim, and Köksalan (2011) proposed a posterior preferencearticulation approach that initially finds a set of compromisesolutions without preference information from the DM in advance,and then uses an interactive selection procedure based on pairwisecomparisons to facilitate the selection of the most satisfactorysolution among the set initially generated. However, they concludethat ways of improving the set of generated compromise solutionsand efficiency of the selection method require further research. Theintricate problem of assigning priorities to responses remains anopen research field, and it is known that practitioners prefer meth-ods that are easy to understand and use in addition to effectiveness(Ilzarbe et al., 2008). Thus, a compromise programming-basedmethod (hereafter denoted as the CP method) that requires aminimum amount of subjective information from the DM wasselected from the literature and its working abilities evaluated.This method is presented in the next section.

3. Optimization criterion

Compromise Programming is a mathematical programmingtechnique which has proven to be extremely powerful inincorporating and resolving conflicting objectives concurrentlyfor locating efficient solutions in convex and non-convex response

N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935 929

surfaces (Messac, Sundararaj, Tappeta, & Renaud, 2000; Shin & Cho,2009). This technique can be generalized into the metric

Min Lp ¼ MinXn

i¼1

kiðyiðxÞ � uiÞ½ �p !1=p

ð1Þ

where u is the utopia (ideal) point, p is the parameter that definesthe type of metric, and ki represents the priority or the relativeworth of the ith response.

Costa and Pereira (2010) proposed an easy-to-use criterion builton the Compromise Programming technique that is defined asfollows:

MinimizeXn

i¼1

jyi � hijUi � Li

� �pi

ð2Þ

where hi represents the target value for the ith estimated responseðyiÞ at x, pi are user-specified parameters (response’s priorities,pi > 0), and Ui and Li are response specification limits that areavailable for process or product quality control.

In Eq. (2) the exponent 1/p was omitted so the metric behavioris broken. This change does not impact on the final solutionbecause the formulation with and without the root theoreticallyprovides the same solution. Moreover, by setting ki = 1/Ui � Li less(subjective) information is required from the DM and makespossible the combination of responses which by nature havedifferent measuring units, scale and allowable range. Thisresponses normalization procedure is a variant of the so-calledupper-lower-bound approach evaluated by Marler and Arora(2005) who showed that this type of transformation is appropriateto eliminate the potential for the numerical dominance of aparticular response and provide a safeguard against potentialdifficulties for capturing non-dominated solutions (non-inferioror Pareto optimal solutions, where any improvement in oneresponse cannot be done without degrading the value of, at least,another response) along convex and concave response surfaces.Such as Costa and Pereira (2010) affirmed, varying the pi willprovide the required flexibility to manipulate the objectivefunction’s curvature and explore trade-offs among responses soas to obtain satisfactory solutions. This will be illustrated here insituations where robustness is an issue, including mean andvariance responses, whereas previous authors limited the analysisto the optimization of multiple mean responses. It is interesting tonote that in Eq. (2) the target value for responses’ variance may notbe set equal to zero, as it is typically considered in other criteria. Atarget value for standard deviation up to a certain level isacceptable as it may be useful to explore trade-offs amongresponses (Shaibu & Cho, 2009).

To validate or compare the results obtained with this criterion,the Kim and Lin (2006) and Wu and Chyu (2004) methods are usedas benchmark and reviewed in the Appendixes A and B,respectively.

4. SUR and OLS techniques: an overview

The SUR technique is an application of the GLS technique andoften called feasible generalized least squares, because the residualcovariance matrix though unknown can be consistently estimatedfrom the data. With the SUR technique a multiple equationstructure is defined as yi = Xibi + ei for i = 1, 2, . . ., n, where yi is aN � 1 vector of observations on the ith response, Xi is a N � Ki

matrix, bi is a Ki � 1 vector of regression parameters, and ei is anN � 1 vector of random error.

Whenever the error are correlated across the equations(response models), the equations are related and joint estimation,rather than equation-by-equation estimation, leads to more

precise estimates for the regression coefficients and predictionvalues for the dependent variables. The best linear unbiasedestimate of the regression coefficients (b) can be expressed as

b ¼ XTX�IN

� ��1X

� ��1

XTX�IN

� ��1y ð3Þ

where X is a block diagonal matrix,P

is the variance–covariancematrix of the responses, IN is an NxN identity matrix, and� represents the Kronecker product. Zellner (1962) proposed anestimator for

Pbased on the residuals (the difference between

the observed response value and the corresponding estimatedvalue, denoted by eÞ obtained from the OLS estimation defined as

dX ¼ eTi ej=N for i; j ¼ 1;2; . . . ;n: ð4Þ

For the case of OLS technique, the error terms in different equationsare mutually uncorrelated, equations are independently estimated,and a linear unbiased estimate of regression coefficients are definedas

b ¼ XT X� ��1

XT y ð5Þ

For a comprehensive and thorough review on this and other modelingtechniques the reader is referred, for example, to Greene (2008).

5. Examples

To evaluate the response modeling and optimization criteriaimpact on the optimization of multiple responses, the OLS andSUR techniques were used to fit models to responses and theworking abilities of the CP criterion compared with that of otherpopular criteria. Two examples from the literature with differentresponse-types, feasible regions, number of responses andvariables are used.

Three cases are considered in the first example, namely:

(a) Case 1a: the results obtained with the CP method arecompared with those obtained with the ‘‘maximin’’ approachproposed by Kim and Lin (2006), hereafter denoted as the KLmethod, using the response models fitted by these authorswith the OLS.

(b) Case 1b: the response models were fitted with the SURtechnique and, with no loss of generality, only solutionsgenerated with the CP method are presented.

(c) Case 1c: the response models were fitted with the OLStechnique using the same regressors of the SUR models.The solutions were generated with the CP method.

A similar procedure is followed in the second example:

(a) Case 2a: the results of the CP method are compared withthose of the loss function-based method proposed by Wuand Chyu (2004), hereafter denoted as the WC method,using the response models fitted by Wu and Chyu (2004)with the OLS. The improved models fitted by Pal and Gauri(2010) are also used.

(b) Case 2b: the response models were fitted with the SURtechnique and, with no loss of generality, only solutionsgenerated with the CP method are presented.

(c) Case 2c: the response models were fitted with the OLStechnique using the same regressors of the SUR models.The solutions were generated with the CP method.

In both examples the compromise solutions are evaluated interms of bias and robustness, using the measures introduced by

Table 1Responses specifications – Example 1.

Specifications Responses

ðl1; r1Þ ðl2; r2Þ ðl3; r3Þ

Lower bound (3.00, 0.00) (0.10, 0.00) (15.00, 1.00)Target (—–, —–) (—–, —–) (30.00, —–)Upper bound (7.00, 0.10) (0.60, 0.10) (45.00, 2.00)

930 N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935

Costa, Pereira, and Tanco (2010), namely, the cumulative bias(Bcum) and Robustness (Rob). These measures are defined as

Bcum ¼Xn

i¼1

Wi y�i � hi

�� �� ð6Þ

where y�i represents the ith estimated response value at ‘‘optimal’’variable settings, and hi is the corresponding target value; Wi is anormalization parameter defined as W = 1/(U � L) for STB andLTB-type responses, and W = 2/(U � L) for NTB-type responses.

The robustness is assessed by

Rob ¼ trace ury�� �

ð7Þ

where ry� is the variance–covariance matrix of the responsesmodels at optimal variables setting and u is a matrix whosediagonal and non-diagonal elements are defined as uii = 1/(Ui � Li)2

and uij = 1/(Ui � Li) (Uj � Lj), i – j, respectively.As concerns the results of Bcum and Rob, the lower their values

are, the better the compromise solution will be. In practice, Bcum

and Rob take dimensionless values greater than or equal to zero,but zero is the most favorable.

Example 1.

Case 1a Response models fitted with the OLS techniqueTo identify the settings for concentration of surfactant(x1), concentration of salt (x2), and time of stirring (x3)that lead to the optimization of three properties of microbubbles that result from surfactant solutions mixed athigh speeds, namely, the stability (y1), volumetric ratio(y2), and temperature (y3), the mean and standarddeviation models for these properties (responses) werefitted with the OLS as follows:

Table 2Results – Example 1a.

KL1 KL2 CP

l1 ¼ 4:95þ 0:82x1 � 0:45x2 þ 0:00x3 � 0:16x21 þ 0:27x2

2

þ 0:00x23 � 0:11x1x2 þ 0:07x1x3 þ 0:00x2x3

þ 0:00x1x2x3½R2 ¼ 0:950;R2adj ¼ 0:931; PRESS ¼ 1:123�

r1 ¼ 0:06þ 0:00x1 þ 0:11x2 þ 0:06x3 þ 0:12x21 þ 0:00x2

2

þ 0:10x23 þ 0:00x1x2 � 0:10x1x3 þ 0:05x2x3

þ 0:00x1x2x3½R2 ¼ 0:828;R2adj ¼ 0:698; PRESS ¼ 0:228�

l2 ¼ 0:46þ 0:13x1 � 0:06x2 þ 0:05x3 � 0:06x21 þ 0:00x2

2

� 0:03x23 þ 0:00x1x2 þ 0:00x1x3 þ 0:00x2x3

þ 0:00x1x2x3½R2 ¼ 0:892;R2adj ¼ 0:812; PRESS ¼ 0:073�

r2 ¼ 0:02� 0:01x1 þ 0:01x2 � 0:01x3 þ 0:00x21 þ 0:00x2

2

þ 0:02x23 þ 0:00x1x2 � 0:01x1x3 þ 0:02x2x3

þ 0:00x1x2x3½R2 ¼ 0:798;R2adj ¼ 0:647; PRESS ¼ 0:026�

xi (�0.60, �1.00, (�0.23, �0.38, (0.006, �0.235,

�1.00) �0.99) �1.00)

li (5.10, 0.34, 25.41) (4.97, 0.37, 26.39) (5.08, 0.39, 26.35)ri (0.02, 0.06, 3.92) (0.06, 0.05, 1.64) (0.09, 0.04, 1.23)Bcum 4.98 3.05 2.91Rob 15.76 3.34 2.55

Table 3Responses correlation – Example 1.

l1 l2 l3

l3 ¼ 28:75� 1:48x1 þ 0:00x2 þ 2:33x3 � 0:78x21 � 1:18x2

2

þ 0:00x23 þ 0:00x1x2 � 0:71x1x3 þ 0:00x2x3

þ 0:00x1x2x3½R2 ¼ 0:954;R2adj ¼ 0:920; PRESS ¼ 14:683�

r3 ¼ 6:08� 1:53x1 þ 0:49x2 þ 4:85x3 þ 0:00x21 þ 2:26x2

2

þ 0:00x23 þ 0:00x1x2 � 0:65x1x3 þ 0:00x2x3

� 0:67x1x2x3½R2 ¼ 0:943;R2adj ¼ 0:886; PRESS ¼ 43:00�

l1 1.0000 0.8850 �0.4029l2 0.8850 1.0000 �0.1042l3 �0.4029 �0.1042 1.0000

The mean response l1 is of LTB-type, l2 is of STB-type, and l3 isof NTB-type; ri ði ¼ 1;2;3Þ are of STB-type. The constraints for the

input variables are �1 6 xi 6 1 (i = 1,2,3), and the responsesspecifications are presented in Table 1 (Kim & Lin, 2006).

Table 2 displays the results achieved with the CP method andthose presented by Kim and Lin (2006). The solution generatedwith the CP method is better in terms of Bcum and Rob valuesthan the solution denoted by KL1, which corresponds to thesolution obtained with the ‘‘maximin’’ criterion when (k = 0) lineardesirability functions are used and standard deviation responsesare not considered but estimated a posteriori by plugging optimalvariables setting into the standard deviation response models(Kim & Lin, 2000). Moreover, the CP solution is slightly betterthan KL2, which was obtained by Kim and Lin (2006) when lineardesirabilities for the mean and standard deviation responses wereconsidered. In fact, Bcum and Rob values are lower in the CP andKL2 solutions than in KL1 due to the smaller value of r3, which isoutside of the specifications in KL1. This shows that unacceptablesolutions are obtained whenever the mean of the responses isconsidered and the variability of the responses is ignored.Furthermore, note that r3 value is lower in CP than in KL2 andthe CP solution is obtained by keeping responses prioritiesunchanged, with pi values equal to one.Case 1b Response models fitted with the SUR technique

To take into account the correlations among responses,such as displayed in Table 3, the SUR technique wasemployed to fit models for the mean and varianceresponses. The models are as follows:

l1 ¼ 4:9918þ 0:8241x1 � 0:4289x2 � 0:1193x21 þ 0:1771x2

2

r1 ¼ 0:0564þ 0:0959x2 þ 0:0736x3 þ 0:1031x21 þ 0:1236x2

3

� 0:1142x1x3 þ 0:0491x2x3

l2 ¼ 0:4669þ 0:1302x1 � 0:0447x2 þ 0:0320x3 � 0:0719x21

� 0:0387x23 þ 0:0194x1x2x3

r2 ¼ 0:0197� 0:0123x1 þ 0:0111x2 þ 0:0175x23

� 0:0077x1x3 þ 0:0211x2x3 � 0:0181x1x2x3

Table 4Results

p1;2;3

xi

li

ri

Bcum

Rob

a p1;2

Table 5Statistic

R2

R2adj

PRES

Table 6Results

p1;2;3

xi

li

ri

Bcum

Rob

N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935 931

l3 ¼ 28:6361� 1:5357x1 þ 2:3048x3 � 0:8568x22

� 0:9407x23 � 0:7056x1x3 þ 0:3983x2x3

r3¼5:9925�1:4468x1þ5:0503x3þ2:3962x22

�0:1803x1x2�0:7564x1x3�0:3084x2x3�0:8357x1x2x3

R2system ¼ 0:998

The best solution obtained with the CP method is displayed inTable 4, and the improvement in terms of both bias and robustnessis noticeable when compared with those presented in Table 2. Infact, Bcum and Rob values obtained with response models fitted withthe SUR are lower than those obtained with the OLS (see Tables 2and 4). The reduction in the Rob value is significant and confirmsthat the SUR technique is useful for modeling correlated responses.

Case 1c Models fitted with the OLS technique with the sameregressors of Case 1bThe values for quality of descriptions ðR2;R2

adjÞ andquality of predictions (PRESS) of each response modelare displayed in Table 5.The values of the statistics in Table 5 are similar to thoseof response models in Case 1a so the CP method resultsdisplayed in Table 6 do not show, as expected,significant differences, except that the Rob value isslightly lower and less favorable than those of Example1b. This confirms that models fitted with the OLS are notas useful as those fitted with the SUR for optimizingresponses that are significantly correlated.

– Example 1b.

CP

ðl; rÞa (2, 1; 2, 1; 1, 1)(�0.1975, �0.1160, �1)(4.876, 0.372, 25.589)(0.082, 0.040, 1.090)2.681.79

;3ðl;rÞ corresponds to the weights assigned to ðl; rÞ for responses 1, 2 and 3.

s – Example 1c.

Stability Volumetric ratio Temperature

l1 r1 l2 r2 l3 r3

0.953 0.846 0.910 0.808 0.958 0.9460.935 0.730 0.843 0.664 0.926 0.893

S 1.086 0.228 0.081 0.005 13.683 48.93

– Example 1c.

CP

ðl; rÞ� (0.5, 0.5; 2, 0.5; 1.5,2)(0.0207, �0.1546, �1.00)(5.046, 0.392, 25.652)(0.098, 0.038, 1.244)2.972.23

Example 2.

Case 2a Response models fitted with the OLS techniqueA plasma enhanced chemical vapor deposition processstudy was reported by Tong and Su (1997) and revisitedby other authors, namely by Wu and Chyu (2004) andPal and Gauri (2010). Eight variables (x1, . . . ,x8) wereconsidered to optimize the mean and variance of twoquality characteristics, namely, the deposition thickness(y1) and the refractive index (y2). The responses modelsfitted with the OLS are as follows (Wu & Chyu, 2004):

Logl1¼2:834�0:007x1þ0:010x2�0:129x3�0:000x4

þ0:209x5þ0:083x6þ0:069x7�0:210x8

�0:021x3x4�0:076x3x5þ0:156x3x8�0:019x4x5

þ0:006x4x8�0:035x5x8½R2¼0:948;R2adj

¼0:710;PRESS¼0:312�

Logr21 ¼ 4:660� 0:002x1 þ 0:859x2 � 2:051x3 � 1:234x4

þ 1:621x5 þ 0:396x6 þ 0:511x7 � 1:851x8

� 0:127x3x4 � 0:273x3x5 þ 1:404x3x8 þ 0:092x4x5

þ 0:344x4x8 � 0:808x5x8½R2 ¼ 0:896;R2adj

¼ 0:413; PRESS ¼ 33:774�

Logl2 ¼ 0:321þ 0:039x1 � 0:068x2 þ 0:031x3 þ 0:083x4

� 0:164x5 þ 0:016x6 � 0:027x7 þ 0:062x8

þ 0:019x3x4 þ 0:046x3x5 � 0:087x3x8 � 0:015x4x5

� 0:014x4x8 þ 0:064x5x8½R2 ¼ 0:982;R2adj

¼ 0:896; PRESS ¼ 0:069�

Logr22 ¼ �0:918� 0:053x1 þ 0:215x2 þ 0:098x3 � 2:273x4

þ 0:132x5 þ 0:212x6 þ 0:015x7 � 0:269x8

þ 0:196x3x4 � 0:417x3x5 þ 0:212x3x8 þ 0:580x4x5

þ 0:289x4x8 � 0:313x5x8½R2 ¼ 0:968;R2adj

¼ 0:818; PRESS ¼ 3:077�

The estimated mean responses, l1 and l2, are of NTB-type; r1

and r2 are of STB-type. The responses specifications (target, lowerand upper bounds for the mean ðliÞ and standard deviation ðriÞresponses) are displayed in Table 7 where we assume that the upperbounds for r1 and r2 correspond to the respective higher valuesobtained from the experimental runs.

Wu and Chyu (2004) showed that the solutions achieved withthe other methods they tested are not in close agreement. In fact,these authors showed that their loss function-based methodyielded a better solution (a lower total average loss). Assuming anasymmetric loss function, Wu and Chyu (2004) obtained thesolution denoted by WCD in Table 8 for a discrete region wherex1 takes the values 1 and 2, and xi (i = 2,3,4, . . . ,8) takes the values1, 2 and 3. The same solution was found by the CP method keepingall the pi values equal to one, whereas Wu and Chyu used (eight)different cost coefficients for different response targets.

Considering that the region of control factors is measurable,except for x1 and x3, with xi 2 [1,3] for i = 2, 4, . . ., 8, x1 = (1,2) andx3 = (1,2,3), the CP method yielded a solution with marginaldifferences to that of the WC method, denoted by WCc in Table 8,setting the shape factors for the estimated mean and standarddeviation models of the responses 1 and 2 as p1;2ðLog l; Log r2Þ ¼ð1:5;1:5; 2:0;1:0Þ.

Table 7Responses specifications – Example 2.

Specifications Responses

ðl1; r1Þ ðl2; r2Þ

Lower bound (950.00, 0.00) (1.90, 0.00)Target (1000.00, —–) (2.00, —–)Upper bound (1050.00, 292.00) (2.10, 0.09)

Table 8Results – Example 2a.

WCD WCc

xi (2, 1, 1, 1, 3, 1, 3, 2) (1, 1, 2, 1, 2.54, 1.29, 3, 3)li (1007.14, 2.02) (1000.00, 2.00)r2

i(1986.32, 0.0009) (484.27, 0.0003)

Bcum 2.11 1.72Rob 0.15 0.05

Table 11Results – Example 2b.

CPBd and CPRd CPBm and CPRm

p1;2ðLog l; Log r2Þ (1, 1; 1, 1) (1, 1; 2, 2)

xi (2, 3, 3, 1, 1, 2, 3, 3) (2, 2.99, 3, 1, 1, 1.66, 2.85, 2.98)li (1005.29, 2.01) (1000.00, 2.00)r2

i(134.65, 0.0001) (102.42, 0.0001)

Bcum 0.68 0.35Rob 0.01 0.01

Table 9Statistics – PGmodels.

Deposition thickness Refractive index

Log l1 Log r21 Log l2 Log r2

2

R2 0.973 0.965 0.964 0.990

R2adj

0.886 0.849 0.899 0.942

PRESS 0.053 12.19 0.013 1.962

Table 10Results – Example 2a – PGmodels.

CP

xi (1, 1, 3, 1, 1, 2.23, 1, 1.72)li (1000, 2.00)r2

i(467.27, 0.0001)

Bcum 1.63Rob 0.02

932 N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935

Since Wu and Chyu (2004) showed that their methodoutperforms others that have been used in practice, the resultsachieved from the CP method in this example, which are in closeagreement with those of the WC method, gives confidence tosuggest its application to solve real-life problems.

Pal and Gauri (2010) also presented response models fitted withthe OLS to this example. Overall, their models are slightly betterthan those of Wu and Chyu (2004). The statistics for quality ofdescriptions ðR2;R2

adjÞ and quality of predictions (PRESS) of eachresponse model are presented in Table 9.

As concerns the results achieved with the CP method forcontinuous region with the models fitted by previous authors, Table10 shows slightly improvements in responses, namely, lower r2

i

values, which can be justified with the slight improvements in thestatistics of the models.

Case 2b Response models fitted with the SUR techniqueModels fitted with the SUR technique to data of plasmaenhanced chemical vapor deposition process study areas follows:

Logl1 ¼ 2:739� 0:068x1 þ 0:072x6 þ 0:069x7 � 0:056x1x2

þ 0:049x1x7 � 0:017x2x7 � 0:019x2x8 � 0:037x3x6

þ 0:039x3x7 � 0:006x4x7 � 0:031x6x7 þ 0:019x26

Logr21 ¼ 5:115� 2:713x6 þ 0:544x1x3 � 0:564x4x6

þ 0:260x6x7 þ 0:171x6x8 � 0:302x7x8 þ 1:095x26

� 0:378x21x2 þ 0:320x2

1x4 þ 0:036x22x4 � 0:062x2

2x6

þ 0:039x22x7 � 0:121x2

6x1

Logl2 ¼ 0:483� 0:219x2 þ 0:152x3 � 0:151x8 þ 0:082x1x2

� 0:049x1x3 � 0:046x1x8 � 0:035x2x4 � 0:033x3x4

þ 0:061x27

Table 12

Statistics – Example 2c.

Deposition thickness Refractive index

Log l1 Log r21 Log l2 Log r2

2

R2 0.984 0.994 0.989 0.953

R2adj

0.947 0.973 0.976 0.900

PRESS 0.023 0.753 0.002 17.23

Logr22 ¼ �1:192� 0:497x2 þ 0:466x5 � 1:867x6 þ 0:319x7

þ 0:695x26 þ 0:595x1x4 � 0:360x1x6 � 0:189x1x8

� 0:394x4x5

R2system ¼ 0:999

Table 11 displays the solutions with minimum bias and highrobustness for discrete (CPBd and CPRd, respectively) andmeasurable (CPBm and CPRm, respectively) regions obtained withthe CP criterion. It is interesting to note that there is neithersignificant difference between CPBd and CPRd nor between CPBm

and CPRm solutions. Like in the previous case, the results for thediscrete region are worse than those achieved for the measurableregion in terms of Bcum.

In this example the responses are not significantly correlated.The correlation between the mean responses is equal to 0.13.However, the improvement in responses properties (Bcum andRob) using response models fitted with the SUR technique is visible,particularly in terms of r2

1, which may justify the use of thistechnique in situations where significant correlations betweenresponses do not exist.

Case 2c Models fitted with the OLS technique with the sameregressors of Case 2bIn this case the statistics for quality of descriptionsðR2;R2

adjÞ and quality of predictions (PRESS) of eachresponse model are presented in Table 12.These values are slightly better than those of Wu andChyu (2004) and Pal and Gauri (2010), except for thePRESS value of response two (Refractive index). As

Table 13Results – Example 2c.

CPBm and CPRm

p1;2ðLog l; Log r2Þ (1, 1; 2, 2)

xi (2, 3, 3, 1, 3, 1.67, 2.81, 2.98)li (1000.00, 2.00)r2

i(95.38, 0.0002)

Bcum 0.33Rob 0.02

N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935 933

concerns the results displayed in Table 13, it is visible animprovement in terms of r2

1 when they are comparedwith those achieved from the models fitted by Wu andChyu (2004) and Pal and Gauri (2010). Results displayedin Table 13 also confirm that OLS and SUR techniqueslead to the similar results whenever responses are notsignificantly correlated and responses models have thesame regressors.

6. Discussion

An optimal solution for single response optimization problems isdefined easily, whereas a solution for multiresponse optimizationproblems is more a concept than a definition (Marler and Arora,2004). In these problems the responses are usually in conflict, andone cannot expect to find a solution with all responses on-target.In contrast to current practice, the evaluation of compromisesolutions, and consequently of the criteria performance, wasundertaken through optimization measures that consider thedesired response properties. In particular, bias and robustness wereassessed using optimization measures that provide promptfeedback about the utility or worth of the compromise solution,taking into account technical and economic considerations or theDM’s preferences. These measures can be used along with anymethod of practitioner’s interest, and are particularly useful whenan optimization routine is used to generate a large number ofsolutions, because they may serve to discard those solutions withless favorable Bcum and Rob values. In fact, while a large numberof solutions may provide insights into the trade-offs among theresponses, some of them may have a small chance of being chosenas preferential solutions. So they must be promptly identified anddiscarded.

The previous examples do not show significant differencesamong the best solutions obtained from methods that differ interms of their theoretical basis and statistical properties, whichmay help to understand why there is no widely-accepted methodfor optimizing multiple responses in spite of the existence of somemethods that are more appealing than others, namely, as concernsthe difficulty to obtain the required preference information. Whileeasy-to-use is, surely, a relevant aspect for practitioners when theychoose an approach or criterion for multiresponse optimization,the type and amount of preference information required fromthe DM is also decisive for a satisfactory solution to be achieved.

To date, there is not a procedure to guide the analyst in theassignment of weights or priorities to responses and starting pointsto the optimization routine. While the center of the experimentalregion x = (0, 0, 0) is, in general, a good starting point, since thedistances from the center to the boundaries of the experimentalregion are smaller, guidelines about how to choose the weightingcoefficients do not exist. What is known is that optimizationresults change significantly as the weighting coefficients vary. Todetermine beforehand the variations required in the weights orresponse properties and to know which response(s) value willchange and which is the direction and magnitude of that change,so as to produce a solution of interest, is difficult, if at all possible.

Thus, this task terminates, in general, by virtue of time constraintsor sufficient satisfaction of the decision-maker with one of thegenerated solutions.

As Messac and Ismail-Yahaya (2001) showed, aggregateobjective functions must provide the flexibility of changing theircurvature in order to be effective. Nevertheless, this is not a simpletask and becomes harder whenever the number of preferenceparameters and responses increase. In the CP method the curvatureof response surfaces can be easily manipulated and in some casesminimum or no cognitive effort is required from the DM. Inparticular, the solutions for Cases 1a (Table 2), 2a and 2b (for thediscrete region) were obtained keeping the responses prioritiesunchanged (equal to one). In Cases 1b, 1c, 2a, 2b and 2c (for themeasurable region) the curvature of the objective function was ad-justed to capture solution(s) of preference. Nevertheless, in bothexamples the pi value assigned to each response is less than orequal to three (pi 6 3). This is relevant, because this guidelinemay facilitate the task of achieving a preferential solution.Solutions of some cases were found using natural numbers(pi = 1, 2, 3), although to vary the pi values in steps of 0.5 units oreven in steps of 0.25 units is suggested, and have been the bestchoice to achieve satisfactory solutions for problems similar tothose revisited here.

This article illustrates the CP method at problematic situationsoften found in practice, namely, those with variance due to uncon-trollable factors, and evaluates its performance in two exampleswhere responses models were fitted with the OLS and SURtechniques. To have ‘good’ fitting models is determinant, becauseresponses models have a strong impact on the optimization criteriaoutput. The results improvement obtained with the SUR techniquein both examples is not unexpected. In the first example there is asignificant correlation between responses so better responseproperties in Cases 1b were expected. In the second example,where there is no significant correlation between responses, theresults achieved with models fitted with the SUR technique wereequivalent to those achieved with the OLS models that have highquality of predictions and quality of descriptions. Lowest-degreemodels that satisfy modeling technique assumptions, fit the dataadequately, and models with regressor variables lower in numberthan the number of response observations are preferable.Nevertheless, higher-order models supported by meaningfulstatistics must be considered whenever they yield higher R2 andR2

adj values in addition to lower PRESS values in order to enhanceresponses properties. In those cases, other statistics like thevariance inflation factor (VIF), Mallows Cp, and Akaike’s informa-tion criterion (AIC) may also be considered in the models adequacychecking procedure. This is because the VIF is an importantdiagnostic for multicollinearity (significant correlation betweentwo or more predictor variables), Mallows Cp criterion is used todiagnose bias (lack of fit), and AIC measures the tradeoff betweenthe amount of bias and variance in the model.

7. Conclusions

Ordinary Least Squares and Seemingly Unrelated Regressiontechniques were used for modeling responses and results showthat this is a decisive task, rather than the criterion selection, toachieve solutions with desired responses properties. Numeroustypes of errors are possible by modeling responses. Perhaps themost common ones are the omission of relevant variables andthe inclusion of superfluous variables. Models with high qualityof description and quality of prediction lead to better compromisesolutions, which may not be possible with standard (second order)polynomial models. This is illustrated in Example 2, whereresponses were not significantly correlated. In this situation the

934 N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935

Ordinary Least Squares can compete with the Seemingly UnrelatedRegression. However, for situations where responses aresignificantly correlated, the later technique is more useful. TheSUR technique is not included in software packages widely usedby practitioners so its application in problems developed underthe RSM framework has been rare. Since correlated responses arenot unusual in this type of problems, the SUR technique along withthe Costa and Pereira (2010) criterion is an approach that haveproved to be effective. The examples do not cover all possiblescenarios regarding the number of responses and input variables,model’s form and response types, but the results confirm that thiscriterion can yield results, at least, in close agreement with those ofother statistically sound methods, which justifies its use along withthe SUR technique in multiresponse optimization problemsdeveloped in the RSM framework.

Two major topics are identified for future research. The first oneis the simulation of situations where correlation level betweenresponses is varied and the evaluation of the obtained results withthe aforementioned and other regression techniques. The secondone is the investigation on the advantages of transforming theresponses into a smaller number of uncorrelated principalcomponents.

Acknowledgements

The authors are very grateful to the three anonymous reviewersfor their comments and suggestions, which were really valuable forintroducing significant improvements in the paper. The Fundaçãopara a Ciência e Tecnologia (FCT) through the PIDDAC Programfunds (INESC-ID multiannual funding) supported the second author.

Appendix A

A.1. ‘‘Maximin’’ desirability-based method

Kim and Lin (2006) proposed an integrated modeling approachto simultaneously optimize the location and dispersion effects ofmultiple responses, which combines the idea of maximizing theminimum desirability function value for the mean of multipleresponses (Kim & Lin, 2000) and for the mean and standarddeviation of a single response (Kim & Lin, 1998). These authorsproposed

Maximize minimum dl1ðyl1ðxÞÞ; . . . ;dln

ðylnðxÞÞ;

�dr1 ðyr1 ðxÞÞ; . . . ;drn ðyrnðxÞÞ

�ðA:1Þ

where dliand dri

ði ¼ 1; . . . ; nÞ are exponential desirability functionsof the mean and standard deviation responses, respectively. Thedesirability functions are defined as

d ¼expðkÞ�expðkjzjÞ

expðkÞ�1 ; t – 0

1� jzj; t ¼ 0

(ðA:2Þ

where k is a constant (�1 < k <1) and z is a standardized parameterrepresenting the distance of the estimated response from its target(h) in units of the maximum allowable deviation. This parameterdepends on the response type and is defined as

z ¼

y�hymax�h

� �for NTB

y�ymin

ymax�ymin

� �for STB

ymax�yymax�ymin

� �for LTB

8>>>>>><>>>>>>:with ymin

6 y 6 ymax ðA:3Þ

where ymax is the maximum and ymin is the minimum value of theestimated response or specifications limits available for qualitycontrol.

This modeling approach can take into account the model’s pre-dictive ability through the adjustment of the k parameter asfollows:

k0 ¼ kþ ð1� R2Þðkmax � kÞ ðA:4Þ

In this equation, kmax is a sufficient large value of k such that d (z)with kmax is extremely concave and thus has virtually no effect inthe optimization process.

The ‘‘maximin’’ formulation is also robust to the potential depen-dencies among responses, in contrast to the widely used desirabilityfunction approach, the so-called Derringer and Suich’s method.However, it has a drawback. It only concerns with the response thatproduces the lowest degree of satisfaction or is farther from thetarget, and thus useful information associated with all others isignored. As Kim and Lin (2006) noted, this may lead to an unreason-able decision in some cases. As an example, the approach wouldsuggest a solution with satisfaction levels for four responses equalto (0.6, 0.6, 0.6, 0.6) instead of (0.9, 0.9, 0.58, 0.9).

To accommodate specific needs of a given problem situation,and overcome the mentioned drawback, Kim and Lin (2006)extended the above formulation to cope with the followingsituations: (1) responses are alternatives rather than all beingessential; (2) compensation of the ‘‘maximin’’ criterion. The readeris referred to Kim and Lin (2006) for details on these situations.

Appendix B

B.1. Multivariate loss function-based method

Arguing that little attention has been focused on the correlatedmultiple responses with asymmetric loss function, Wu and Chyu(2004) introduced a multivariate loss function-based method thatis an alternative to the popular Pignatiello (1993) method. Theyproposed the simultaneous optimization of process mean (l) andstandard deviation of multiple responses through an expected lossdefined as

E½LðyðxÞ; hÞ� ¼Xn

i¼1

ci ðliðxÞ � hiÞ2 þ r2i ðxÞ

h iþXn

i¼2

�Xi�1

j¼1

cij rijðxÞ þ ðliðxÞ � hiÞðljðxÞ � hjÞ� �

ðB:1Þ

where liðxÞ is the estimated mean of the ith response at x, ci is thecost coefficient for ith response, cij is the correlated cost coefficientfor responses i and j (for i – j), r2

i ðxÞ is the estimated variance forthe ith response at x, and rijðxÞ the estimated covariance for ithand jth responses at x. Wu and Chyu (2004) also have the meritof proposing the formulae for both the loss and correlated costcoefficients. Nevertheless, these coefficients depend on the qualityloss value, which may not be readily available or easily defined.

A major advantage of Eq. (B.1) comes from the modelingversatility, which allows the separate analysis of process means,variances and covariances, making possible to accommodatemultiresponse problems more rigorously and extensively.Minimizing loss functions is a persuasive approach for findingthe best design variable settings in multiresponse problems.However, besides the difficulties already mentioned, existingmethods provide limited flexibility as the same weighting schemeis assigned to bias and variance and this weighted-sum square(loss) functions are incapable of capturing points of interest innon-convex response surfaces (Messac et al., 2000; Messac &Ismail-Yahaya, 2001). These are not interesting features from apractical and theoretical point of view, which justify the increasinguse and development of other methods or approaches.

N. Costa et al. / Computers & Industrial Engineering 62 (2012) 927–935 935

References

Besseris, G. (2009). Multi-response quality improvement by non-parametric DOEmethods. International Journal of Productivity and Quality Management, 4,303–323.

Costa, N., & Pereira, Z. (2010). Multiple response optimization: A global criterion-based method. Journal of Chemometrics, 24, 333–342.

Costa, N., Lourenço, J., & Pereira, Z. (2011). Desirability function approach: A reviewand performance evaluation in adverse conditions. Chemometrics and IntelligentLaboratory Systems, 107, 234–244.

Costa, N., Pereira, Z., & Tanco, M. (2010). Optimization measures for assessingcompromise solutions in multiresponse problems. In World congress onengineering – ICMEEM (pp. 2252–2257).

Derringer, G., & Suich, R. (1980). Simultaneous optimization of several responsevariables. Journal of Quality Technology, 12, 214–218.

Derringer, G. (1994). A balancing act: Optimizing product’s properties. QualityProgress, 51–58.

Fogliatto, F., & Susan, A. (2000). Variance of predicted response as an optimizationcriterion in multiresponse experiments. Quality Engineering, 12, 523–533.

Goethals, P., & Cho, B. (2011a). Using higher precision-based response surfacedesigns to determine the optimal process target. International Journal ofAdvanced Manufacturing Technology, 56, 13–30.

Goethals, P., & Cho, B. (2012). Extending the desirability function to account forvariability measures in univariate and multivariate response experiments.Computers & Industrial Engineering, 62, 457–468.

Greene, W. (2008). Econometric analysis (6th ed.). New Jersey: Pearson Prentice Hall.Ilzarbe, L., Álvarez, M., Viles, E., & Tanco, M. (2008). Practical applications of design

of experiments in the field of engineering: A bibliographical review. Quality andReliability Engineering International, 24, 417–428.

Jeong, I., Kim, K., & Chang, S. (2005). Optimal weighting of bias and variance in dualresponse surface optimization. Journal of Quality Technology, 37, 236–247.

Jitthavech, J. (2010). A goodness-of-fit measure for a system-of-equations model.Songklanakarin Journal of Science and Technology, 32, 519–525.

Jones, D. (2011). A practical weight sensitivity algorithm for goal and multipleobjective programming. European Journal of Operational Research, 213, 238–245.

Kazemzadeh, B., Bashiri, M., Atkinson, A., & Noorossana, R. (2008). A generalframework for multiresponse optimization problems based on goalprogramming. European Journal of Operational Research, 189, 421–429.

Kim, K., & Lin, D. (1998). Dual response surface optimization: A fuzzy modelingapproach. Journal of Quality Technology, 30, 1–10.

Kim, K., & Lin, D. (2000). Simultaneous optimization of multiple responses bymaximining exponential desirability functions. Applied Statistics C, 49, 311–325.

Kim, K.-J., & Lin, D. (2006). Optimization of multiple responses considering bothlocation and dispersion effects. European Journal of Operational Research, 169,133–145.

Lee, D., Kim, K., & Köksalan, M. (2011). A posterior preference articulation approachto multiresponse surface optimization. European Journal of Operational Research,210, 301–309.

Marler, R., & Arora, J. (2004). Survey of multi-objective optimization methods forengineering. Structural Multidisciplinary Optimization, 26, 369–395.

Marler, R., & Arora, J. (2005). Function-transformation methods for multi-objectiveoptimization. Engineering Optimization, 37, 551–570.

Marler, R., & Arora, J. (2010). The weighted sum method for multi-objectiveoptimization: Some insights. Structural and Multidisciplinary Optimization, 41,853–862.

Messac, A. (2000). From dubious construction of objective functions to theapplication of physical programming. AIAA Journal, 38, 155–163.

Messac, A., & Ismail-Yahaya, A. (2001). Required relationship between objectivefunction and Pareto Frontier orders: Practical implications. AIAA Journal, 39,2168–2174.

Messac, A., Sundararaj, G., Tappeta, R., & Renaud, J. (2000). Ability of objectivefunctions to generate points on non-convex Pareto Frontiers. AIAA Journal, 38,1084–1091.

Murphy, T., Tsui, K., & Allen, J. (2005). A review of robust design methods formultiple responses. Research in Engineering Design, 15, 201–215.

Myers, R., Montgomery, D., & Anderson-Cook, C. (2009). Response surfacemethodology: Process and product optimization using designed experiments (3rded.). New Jersey: John Wiley & Sons.

Pal, S., & Gauri, S. (2010). Assessing effectiveness of the various performance metricsfor multi-response optimization using multiple regression. Computers &Industrial Engineering, 59, 976–985.

Peterson, J., Miró-Quesada, G., & Del Castillo, E. (2009). A Bayesian reliabilityapproach to multiple response optimization with seemingly unrelatedregression models. Quality Technology & Quantitative Management, 6, 353–369.

Pignatiello, J. (1993). Strategies for robust multiresponse. IIE Transactions, 25, 5–15.Shah, H., Montgomery, D., & Carlyle, W. (2004). Response surface modeling and

optimization in multiresponse experiments using seemingly unrelatedregressions. Quality Engineering, 16, 387–397.

Shaibu, A., & Cho, B. (2009). Another view of dual response surface modeling andoptimization in robust parameter design. International Journal of AdvancedManufacturing Technology, 41, 631–641.

Shin, S., & Cho, B. (2009). Studies on a biobjective robust design optimizationproblem. IIE Transactions, 41, 957–968.

Tanco, M., Viles, E., Ilzarbe, L., & Alvarez, M. (2009). Implementation of design ofexperiments projects in industry. Applied Stochastic Models in Business andIndustry, 25, 478–505.

Tong, L., & Su, C. (1997). Optimizing multiresponse problems in the Taguchi methodby fuzzy multiple attribute decision making. Quality and Reliability EngineeringInternational, 13, 25–34.

Wu, F., & Chyu, C. (2004). Optimization of robust design for multiple qualitycharacteristics. International Journal of Production Research, 42, 337–354.

Zellner, A. (1962). An efficient method of estimating seemingly unrelated regressionequations and tests for aggregation bias. Journal of American StatisticalAssociation, 57, 348–368.