ZhouEtAl2010_QuadInterpolModels_JGlobOptim

7/31/2019 ZhouEtAl2010_QuadInterpolModels_JGlobOptim

http://slidepdf.com/reader/full/zhouetal2010quadinterpolmodelsjgloboptim 1/12

J Glob OptimDOI 10.1007/s10898-010-9600-2

Constructing composite search directions

with parameters in quadratic interpolation models

Qinghua Zhou · Yan Li · Minghu Ha

Received: 14 August 2010 / Accepted: 21 August 2010© Springer Science+Business Media, LLC. 2010

Abstract In this paper, we introduce the weighted composite search directions to develop

the quadratic approximation methods. The purpose is to make fully use of the information

disclosed by the former steps to construct possibly more promising directions. Firstly, we

obtain these composite directions based on the properties of simplex methods and use them

to construct trust region subproblems. Then, these subproblems are solved in the algorithm

to find solutions of some benchmark optimization problems. The computation results show

that for most tested problems, the improved quadratic approximation methods can obviouslyreduce the number of function evaluations compared with the existing ones. Finally, we con-

clude that the algorithm will perform better if the composite directions approach the previous

steepest descent direction of the sub-simplex so far. We also point out the potential applica-

tions of this improved quadratic interpolation method in business intelligence systems.

Keywords Unconstrained optimization · Trust region method · Quadratic model · Simplex

methods · Business intelligence systems

1 Introduction

In this paper, we consider the problem of minimizing a nonlinear smooth objective function of

several variables when the derivatives of the objective function are unavailable and when no

constraints are specified on the problem’s variables. More formally, we consider the problem

Q. Zhou (B) · Y. Li · M. Ha (B)

College of Mathematics and Computer Science, Hebei University, 071002 Baoding, Hebei Province,People’s Republic of China

e-mail: [email protected]

M. Ha


Y. Li


123



J Glob Optim

min x∈ Rn

f ( x)

where we assume f : Rn → R is a smooth nonlinear function, and that its gradient ∇ f ( x)

can not be computed for any x .There are lots of optimization methods to deal with this kind of minimization problems,

such as simplex methods, pattern search methods, trust region methods, etc. [1–3]. The algo-

rithms discussed here belong to the class of trust region methods. They are iteratively built on

the points of a model of the true objective function, which is cheaper to evaluate and easier

to minimize than the objective function itself. For more details about trust region methods,

please refer to Winfield [4] and [5], Powell [6], Yuan [7] and [8], Du et al. [9], Lucia et al.

[10].

Because there is no information of derivatives to use, the main idea of these methods

is to construct a quadratic approximation model by interpolation, which is supposed to be

a good approximation of the true objective functions in a so-called trust region. Then the

model is solved in the trust region space Bk = { x ∈ Rn : x − xk k } , where k is

the region radius and xk is the current iterate point. A trial point is then computed and

accepted as a new iteration point if the objective function can make sufficient reductions.

The trust region radius is enlarged at the same time. Otherwise, the trust region is reduced

and the quadratic model is solved again. This process is repeated until the algorithm finds the

minimum.

The basic idea of such methods is first introduced by Winfield [4] and [5] during the late

1960s , since then it has received great progress both in the real application and in theory

research. In 2002, Powell [11] published the well known method UOBYQA(Unconstrainedoptimization by quadratic approximation). Its main idea is to construct quadratic models

using Lagrange interpolation function at the given points, and then minimize the model

within a typical trust region. If the achieved function reduction is sufficient, then the new

point would be retained as a new interpolation point. Otherwise, the algorithm will consider

whether the model well approximates the objective function. If the answer is “yes”, the next

step would be another trust region step within the updated subspaces. If the answer is “no”,

the algorithm would revise the model first before reducing the trust region radius. In 2004,

Powell described some new methods in that the quadratic model was not completely deter-

mined by the interpolation points, but also with the use of Frobius norm, see Powell [12]. It is

useful in coping with some high dimension problems. Also in 2004, Han and Liu [13] provedthe global convergence of the UOBYQA for general objective functions. In 2005, Berghen

and Bersini [14] developed a parallel constrained UOBYQA, the experimental results are

very encouraging. In 2007, Zhou [15] proposed a revised version of UOBYQA, where a

composite search direction directed by simplex methods was introduced. The author tried to

use the descent information disclosed by the former two steps. The strategy was proved to

be effective. However, it did not sufficiently use the information disclosed by the simplex

methods. In this paper, as we further investigated the properties of simplex methods, we

develop the quadratic model methods by introducing weighted composite search directions.

Our motive is to fully use the information disclosed by the former successful descent steps.The rest of the paper is organized as follows. In Sect. 2, some basic concepts are briefly

reviewed for constructing quadratic models. Then in Sect. 3, we introduce our improved

quadratic interpolation methods after the analysis of the properties of simplex methods. In

Sect. 4, we describe the complete algorithm and give our conclusions in Sect. 5. Finally, in

Sect. 6, we explain potential applications of this proposed method to deal with optimization

problems in business intelligence.

123



J Glob Optim

2 Basic concepts

In this section, we briefly introduce the basic concepts which will be used in the following

sections. For detailed information, please refer to Powell [11].

Let f ( x), Q( x) denote the objective function and the quadratic approximation modelrespectively, li ( x) the Lagrange interpolation function, let x ∈ Rn be the independent var-

iable. The initial and final values of a trust region radius ρ are denoted by ρbeg and ρend

respectively, and · directed to 2-norm.

One of the main ingredients of a trust region algorithm is the choice of an adequate objec-

tive function model. Here we follow Powell’s proposal in choosing a quadratic model of the

form

Q( x) = cQ + gT Q( x − xb) +1

2( x − xb)T GQ( x − xb), x ∈ Rn, (2.1)

where xb is the initial vector in the case ρ = ρbeg , and whenever ρ is reduced, xb is changedto the vector of variables of the least calculated value of f so far. It is designed to prevent

damage from computer rounding errors in the computation of Q( x), by picking xb so that

x − xb becomes small for relevant vectors x . In addition, gQ is a vector of Rn , while GQ is

a square symmetric matrix of dimension n. However, we will depart from many trust region

algorithms in that gQ and GQ will not be determined by the (possibly approximate) first and

second derivatives of f ( x), but rather by imposing that the model (2.1) interpolates function

values at past points, that is:

Q( xi ) = f ( xi ), i = 1, 2, . . . , m. (2.2)

For the parameters of the quadratic model (2.1), cQ is a real number, gQ is a vector and GQ

is a matrix. Hence, to ensure the quadratic model is entirely determined by the Eq. (2.2), the

linear space of quadratic polynomials from Rn to R has dimension m = 12

(n + 1)(n + 2).

The quadratic model is used in a trust region calculation. Specially, d ∈ Rn is set to an

estimate of the solution of the problem

min Qk ( xk + d )

s.t . d (2.3)

where k is the integer such that f ( xk ) is the least objective function value computed so far.

While is another trust region radius satisfying ρ.Furthermore, the algorithm in [11] needs another trust region subproblem to access the

goodness of the quadratic model, in which it is called the model step. And it is solved by the

following maximize problem.

max |l j ( xk + d )|

s.t . d ρ(2.4)

where l j ( x) is the Lagrange function. For more details of the methods, please refer to Powell

[11] and Zhou [15].

3 Weighted composite search directions

In 2007, Zhou [15] proposed a method by introducing composite search directions which

directed by simplex methods. When the algorithm has already got two successful directions,

say, d 1 and d 2, then a new composite direction of d c = θ1(d 1 + d 2) + θ2d 2 is defined. There

123



J Glob Optim

Fig. 1 The search places of simplex methods and trust region methods

exist two common ways to choose the parameters of θ1 and θ2. One choice is to set θ1 = θ2 =

1, which means to equally use the last former two descent steps to form the new search direc-

tions. The other is to set θ1 =f ( xk −1)− f ( xk )

f ( xk −2)+ f ( xk −1)−2 f ( xk ), θ2 =

f ( xk −2)− f ( xk )

f ( xk −2)+ f ( xk −1)−2 f ( xk ), which

not only uses the descent steps, but also trying to use the function values computed already.

The strategies were proved to be effective and the total number of function evaluations were

reduced to some extent.

To explain the motive of this research, firstly we introduce the basic ideas of Multi-

directional search methods, please see [16] for details. Like most of simplex methods, a

basic character of Multi-directional search is that it can use the local information of the

tested problem efficiently to construct new search directions. The process of the implemen-

tation can be depicted in Fig. 1. Suppose that the algorithm has already got two successfuldescent directions, and the current three iterative points denoted by xk −2, xk −1 and xk sepa-

rately. It is easy to know that xk −2 denotes the point with the largest function value and xk the smallest one. The three points form a 2-dimensional sub-simplex. Following the general

ideas of Multi-directional search methods, the next iterative point is most likely generated in

the reflection and expansion parts of the segments xk −2 xk and xk −1 xk . See Fig. 1, the circle

points in the left figure are the most likely substitutes of the next iterative one. If the above

search steps fail to find better point, then the algorithm will consider the contraction step,

in which the algorithm will find possibly better points on the segments xk −2 xk and xk −1 xk .

This process is then repeated until convergence (hopefully) occurs or the stopping criteria is

satisfied.The middle figure in Fig. 1 is the traditional trust region methods, which search the trust

region denoted by the circle centered at the current iterative point to find the descent direction

by minimizing a quadratic model. While the right one is the improved trust region methods

under investigated, where the search places are not centered at the current xk , but at a point

located at the composite directions generated by the former descent directions, where xk is

in the boundary of the improved trust region. Especially, The classic methods find the next

iterate point by searching a trust region centered at the current best point xk . We want to

search the new subspace centered at a point determined by the current best point xk and the

directions xk −1−

xk −2 and xk −

xk −2. Different choices of the circle center “O” result indifferent search space.

In this proposed procedure, not only the current best point but also the previous iterate

points are considered, the algorithm can make better use of the information which the former

successful steps provide. More specifically, let d 1 = xk −1 − xk −2, d 2 = xk − xk −1, then

we define new search direction d c as d c = θ (d 1 + d 2) + (1 − θ )d 2, where 0 ≤ θ ≤ 1. So,

the new trust region has the form

123



J Glob Optim

d − k θ (d 1 + d 2) + (1 − θ )d 2

θ (d 1 + d 2) + (1 − θ )d 2

k .

Keeping in mind that, at the current sub-simplex, xk −2 and xk are the points with the high-

est and the smallest function value respectively, we guess the direction xk − xk −2 is a more

promising one than the direction xk − xk −1. So, if the composite search direction d c is closeto the direction xk − xk −2, the algorithm would perform better. That is, we choose a higher

weight of the direction xk − xk −2, and at the same time, also consider the effect of another

decent direction xk − xk −1. The process would then make use of the information contained

in the former successful steps as much as possible. The outcomes prove the correctness of

the above supposes.

From (2.1), we have:

Qk ( xk + d ) = Qk ( xk ) + [gk + Gk ( xk − xb)]T d +1

2d T Gk d , d ∈ Rn . (3.1)

Then the new trust region subproblem is as follows,

min Qk ( xk ) + [gk + Gk ( xk − xb)]T d +1

2d T Gk d

s.t .

d − k

θ (d 1 + d 2) + (1 − θ )d 2

θ (d 1 + d 2) + (1 − θ )d 2

k (3.2)

and if we set d̃ = d − k θ (d 1+d 2)+(1−θ )d 2

θ (d 1+d 2)+(1−θ )d 2, then (3.2) converts to:

min C k + hT k d̃ +

1

2

d̃ T Gk d̃

s.t .

d̃ k (3.3)

where

C k = Qk ( xk ) + k [gk + Gk ( xk − xb)]T θ (d 1+d 2)+(1−θ )d 2

θ (d 1+d 2)+(1−θ )d 2+ 1

22

k

θ (d 1+d 2)+(1−θ )d 2

θ (d 1+d 2)+(1−θ )d 2

T

Gk

θ (d 1+d 2)+(1−θ)d 2

θ (d 1+d 2)+(1−θ)d 2

, and hk = gk + Gk ( xk − xb) + k Gk

θ (d 1+d 2)+(1−θ )d 2θ (d 1+d 2)+(1−θ )d 2

.

Note that there is no need to retainC k , because it will be omitted when calculating function

reduction of the quadratic model. While hk is needed to compute, because it will be used in

the KKT condition of the subproblem (3.3) to solve d k . Say, d k should satisfy the equationGQ + θ I

d k = −hQ , where I is a n × n unit matrix, θ is a nonnegative number such that

GQ + θ I is positive definite or semi-definite. For details, see Moré and Sorensen [18], Parlett

[19] or Powell [11].

Furthermore, before adjusting the model itself (model step, see [11,15]), the algorithm

will investigate the opposite direction −d c first to see whether the function evaluation would

reduce when the algorithm can not find a better point. In this situation, the trust region

subproblem has the following form.

min Qk ( xk ) + [gk + Gk ( xk − xb)]T d +1

2d T Gk d

s.t .d + k

θ (d 1 + d 2) + (1 − θ )d 2

θ (d 1 + d 2) + (1 − θ )d 2

k . (3.4)

If the algorithm finds a better point in this opposite direction, then we think that the algorithm

maybe already be close to a minimum of the problem, and then, the next choice would not

continue the new strategy, but perform the traditional trust region method at the best point

computed so far.

123



J Glob Optim

Above we have described our ideas about how to construct more promising search direc-

tions according to the skill of Multi-directional methods. In the next section, we will introduce

the complete algorithm.

4 The improved quadratic interpolation algorithm

The main part of the algorithm is similar with the one in [15], where the author tested two

kind of obvious settings of the parameters and did not consider the different effect caused

by the different weights of the directions. We explain the basic steps of the algorithm for

completeness of the paper. For detailed information, please refer to [11,15]. Specially, a zero

value of the integer variable j indicates that the decision between the alternatives is that d

will be calculated by solving the problem (2.3) or (3.2). The parameter direction equals to

one, zero, and negative one, which denotes that the algorithm solves the subproblem (3.2),

(2.3) and (3.4), respectively.

Algorithm

Step 1: Initialization. Including supply the vector of variables xb ∈ Rn , the parameter ρbeg

and ρend , the value of the weight θ . Set = ρ = ρbeg .

Step 2: Construct the model. Generate the initial set of interpolation points, construct the

quadratic model, remember the current best point.

Step 3: Trust region step. Solve the problem (2.3), (3.2) or (3.4), which decided by the value

of the parameter direction, and which provides the trial step d . Note that there is a branchto Step 5 if d < 1

2ρ happens.

Step 4: Update the model. If the algorithm gets a better point, then revise the quadratic

model, the set of interpolation points, enlarge the trust region radius k . Otherwise, before

reducing the trust region radius ρ, we need to test whether the algorithm needs to proceed to

model step. If the answer is “no’́, then set direction=zero, update ρ, k , go to Step 3.

Step 5: Model step. Solve the subproblem (2.4). If the model step get a better point, then go to

Step 3, otherwise, decrease the radii ρ and k unless ρ > ρend , revise xb, set direction = 0,

go to Step 3. If ρ ≤ ρend , then stop.

5 Numerical results and discussions

In this section, we test the above ideas by constructing different search subspaces. Actually,

let θ = 0.1, 0.2, 0.3, 0.4, 0.6, 0.7, 0.8, 0.9 separately, then we test the effect of different

weights of the two descent directions as so far. We guess that the algorithm would perform

better when the composite search direction closes to the direction xk − xk −2. That means

the parameter θ should be chosen as large as possible. At the mean time, the effect of the

other descent direction should not be ignored. The computational results have proved our

conjecture. Remember that we do not test the setting of θ =

0.5, for it is actually the case of θ1 = θ2 = 1, which is just the same as ALG1 in [15]. All the tested problems are taken from

Moré et al. [17] with three kinds of initial points, say, x0, 10 x0 and 100 x0, which correspond

to the three cases separately that the distance between the initial points and the minimum is

nearing, moderate and large.

Specially, for the given initial point x0, let the initial trust region radius ρbeg = 0.2 x0

except that it is set to 0.2 if it would satisfy x0 = 0. The stopping condition is ρend = 10−8.

123



J Glob Optim

Table 1 Number of wins

x0 10 x0 100 x0

Wins Balances Wins Balances Wins Balances

UOBYQA 5 7 5 7 4 8

ALG(θ = 0.1) 6 7 6 7 6 8

UOBYQA 8 8 5 5 3 8

ALG(θ = 0.2) 4 8 8 5 7 8

UOBYQA 6 7 2 5 4 9

ALG(θ = 0.3) 5 7 11 5 5 9

UOBYQA 6 5 3 5 3 8

ALG(θ = 0.4) 7 5 10 5 7 8

UOBYQA 5 6 4 5 4 9

ALG(θ = 0.6) 7 6 9 5 5 9

UOBYQA 3 8 3 5 3 9

ALG(θ = 0.7) 7 8 10 5 6 9

UOBYQA 5 5 3 5 3 8

ALG(θ = 0.8) 8 5 10 5 7 8

UOBYQA 7 5 3 5 4 8

ALG(θ = 0.9) 6 5 10 5 6 8

And if the number of calculating values of objective function exceeds 5000, we also stop the

algorithm.

In most of the cases, our procedure performs better than the well known algorithm

UOBYQA in Powell [11]. We report the detailed computational results in Table 3, 4 and

5 for the case of θ = 0.8 which performs the best compared with algorithm UOBYQA in

Powell [11] and algorithm ALG1 in Zhou [15], while others are omitted for the reason of

similarity in results. Note that, the number of function evaluations is one of most often used

and important criteria in accessing the performance of algorithms. In Table 1, we report the

computational statistic behaviors of UOBYQA and ALG with different θ . Generally, we say

that an algorithm wins if the number of function evaluations required to solve a test prob-lem is less than the one required by another algorithm. For example, in the case of x0, the

numbers of function evaluations required for solving problem 2 are 715 and 692 by algo-

rithm UOBYQA and ALG(θ = 0.8) respectively. Then we say the algorithm ALG(θ = 0.8)

wins because it decreases by 3.2% of the number of function evaluations compares with the

one of UOBYQA. The numbers for solving problem 6 by both algorithms are very similar

(77 and 75, respectively). It is not quite convincing to say that ALG(θ = 0.8) wins in this

aspect alone. But considering the achieved accuracy of the final function value, which is

10−23 for UOBYQA, and 10−25 for ALG(θ = 0.8), we can also say ALG(θ = 0.8) wins.

But, if the precision of the final solution which it achieves is much worse than the other,we do not classify it as the “wins”. Some examples of this case are given later. Further, the

column “balances” means the number of test problems that both of the algorithms perform

almost very similarly on the number of function evaluations.

For convenience to compare and summarize the behavior of the different choices of the

parameter θ , which denotes the different weight of the search directions, we also depict their

performance by column diagram in Figs. 2, 3, and 4.

123



J Glob Optim

1 2 3 40

1

2

3

4

5

6

7

8

θ=0.1 θ=0.2 θ= 0.3 θ= 0.4

i n

t h e

c a s e

o f x 0

1 2 3 40

1

2

3

4

5

6

7

8

i n

t h e

c a s e

o f x 0

θ=0.6 θ=0.7 θ=0.8 θ=0.9

Fig. 2 The statistic outcomes at the initial point x0

1 2 3 40

2

4

6

8

10

12

i n

t h e

c a s e

o f 1 0 x 0

θ=0.1 θ=0.2 θ=0.3 θ=0.4

1 2 3 40

1

2

3

4

5

6

7

8

9

10

θ=0.6 θ=0.7 θ=0.8 θ=0.9

i n

t h e

c a s e

o f 1 0 x 0

Fig. 3 The statistic outcomes at the initial point 10 x0

In each case, the first column of the three denotes the number of wins of algorithm

UOBYQA, the second column denotes that of our algorithm, and the third column

means the number of balances. From Figs. 2, 3, and 4, we can easily know that in most

of cases, the weighted composite search methods perform better than UOBYQA does. Espe-

cially in the case of 10 x0, the number of wins is much more than that of the classical method.

The advantage of the weighted composite search methods is very obvious. furthermore, in

the case of initial points x0 and 10 x0, the larger we set the parameter θ , the better the algo-

rithm performs. In the case of 100 x0, their performances are comparative. That is to say, ingeneral the algorithm with weighted composite search directions could find the solutions of

the tested problems more efficiently when the composite directions approach to the direction

xk − xk −2, which is the steepest one in the sub-simplex.

Noticethatthecaseofsetting θ = 0.5isactuallythesameoneasALG1inZhou [15], where

the correspondent setting is θ1 = θ2 = 1, we compare the different outcomes with ALG1 fur-

ther. We think that although the setting θ1 = θ2 = 1 is the most natural one of all the composite

123



J Glob Optim

1 2 3 40

1

2

3

4

5

6

7

8

9

θ=0.1 θ=0.2 θ=0.3 θ=0.4

i n

t h e

c a s e

o f 1 0 0

x 0

1 2 3 40

1

2

3

4

5

6

7

8

9

θ=0.6 θ=0.7 θ=0.8 θ=0.9

i n

t h e

c a s e

o f 1 0 0

x 0

Fig. 4 The statistic outcomes at the initial point 100 x0

Table 2 Number of wins

(with ALG1 in [15])x0 10 x0 100 x0

Wins Balances Wins Balances Wins Balances

ALG1 3 11 4 9 1 15

ALG(θ = 0.8) 4 11 5 9 2 15

search directions, it would not be the most effective. In fact, θ = 0.8 is the most effective

setting, which also proved our conjecture that if the composite search directions approach

the steepest descent direction of the sub-simplex, then it will perform better. Actually, the

composite direction d c = 0.8 ∗ (d 1 + d 2) + 0.2 ∗ d 2 = 0.8 ∗ ( xk − xk −2) + 0.2 ∗ ( xk − xk −1).

In Table 2, we list the number of wins again. The details of the different algorithms are

expressed in Tables 3, 4, and 5. Notice that for problem 13, in the case of 100 x0, although

the number of function evaluation is 84, which is greater than 79 required by ALG1, the

accuracy of the minimum achieves 10−20, while it is 10−3 for ALG1. The differences are

obvious. So, in this situation, we say the algorithm with weight θ = 0.8 wins. It is the same

situation for problem 6 in the case of initial point 10 x0. And we can easily see the differenceof them.

Finally, we give the details of the algorithms UOBYQA in Powell [11], ALG1 in Zhou

[15] and the weighted composite search direction (θ = 0.8) with all the three kind of initial

points. Specially, the column “P” stands for the number of the problem, and “N” the number

of variables. “nf” denotes the number of function evaluations, “f” the final value which the

algorithm achieved. “OF” appeared in the column “N” is the acronym for Over Flow, and

“Fail” denotes the algorithm stops because the quadratic model can not find any better point

and the trust region radius ρ can not be reduced anymore. Further, the values of f ( x) for the

last two cases denoted by “×

”.

6 Possible application in business intelligence systems

Business intelligence refers to computer-based techniques whose common functions are

usually reporting, data mining, business performance management, benchmarks, etc. It often

123



J Glob Optim

Table 3 The outcomes at the initial point x0

P N UOBYQA ALG1 ALG(θ = 0.8)

nf f nf f nf f

1 3 99 3.47799D−21 109 3.97736D−23 121 5.59604D−21

2 6 715 1.00483D−26 708 4.76818D−23 692 1.36717D−22

3 3 32 1.12793D−08 32 1.12793D−08 32 1.12793D−08

4 2 OF × OF × OF ×

5 3 122 1.43266D−23 119 4.72199D−25 119 4.72199D−25

6 3 77 2.11433D−23 Fail × 75 2.87857D−25

7 3 59 4.71400D−01 52 4.71400D−01 52 4.71400D−01

8 3 333 1.51793D−05 483 1.51793D−05 422 1.51793D−05

9 3 OF × Fail × Fail ×

10 2 275 1.35528D−20 288 0 330 1.77494D−30

11 4 OF × 124 8.58222D+04 133 8.58222D+04

12 3 OF × OF × OF ×

13 3 48 5.13264D−21 Fail × Fail ×

14 3 167 3.98997D+00 160 3.98997D+00 160 3.98997D+00

15 3 64 2.39103D+00 72 2.39103D+00 63 2.39103D+00

16 2 43 3.25509D−27 45 2.33060D−25 45 2.33060D−25

17 4 464 2.98127D−20 445 2.80724D−22 445 2.80724D−22

18 4 89 1.73019D−19 89 1.73019D−19 89 1.73019D−19

Table 4 The outcomes at the initial point 10 x0


nf f nf f nf f

1 3 140 4.17632D−23 131 4.31183D−19 131 4.31183D−19

2 6 4384 1.30490D−23 3078 1.62021D−24 3647 3.26072D−24

3 3 86 1.12793D−08 109 1.12793D−08 97 1.12793D−08

4 2 OF × OF × OF ×

5 3 OF × Fail × Fail ×

6 3 110 1.14757D−25 92 1.32948D−24 93 6.21487D−28

7 3 143 4.71400D−01 142 4.71400D−01 131 4.71400D−01

8 3 406 1.51793D−05 672 1.51793D−05 517 1.51793D−05

9 3 OF × 490 3.19813D−06 OF ×

10 2 281 1.97215D−31 271 7.88861D−31 222 1.97215D−31

11 4 232 8.58222D+04 OF × OF ×

12 3 OF × OF × OF ×

13 3 100 2.57369D−03 80 2.81801D−20 94 5.76492D−21

14 3 389 3.98997D+00 247 3.98997D+00 247 3.98997D+00

15 3 122 2.39103D+00 116 2.39103D+00 119 2.39103D+00

16 2 215 5.54668D−29 175 3.88514D−29 175 3.88514D−29

17 4 501 3.54357D−22 472 6.44567D−23 472 6.44567D−23

18 4 OF × OF × OF ×

123



J Glob Optim

Table 5 The outcomes at the initial point 100 x0


nf f nf f nf f

1 3 137 2.14571D−23 135 2.06254D−21 141 9.65256D−22

2 6 OF × 5000 4.91227D−03 5000 4..91227D−03

3 3 OF × OF × Fail ×

4 2 OF × OF × OF ×

5 3 Fail × Fail × Fail ×

6 3 217 6.90253D−31 186 3.19861D−21 172 4.70010D−22

7 3 211 4.71400D−01 193 4.71400D−01 200 4.71400D−01

8 3 OF × OF × OF ×

9 3 OF × OF × Fail ×

10 2 289 0 243 3.15544D−30 243 3.15544D−30

11 4 OF × 343 8.58222D+04 343 8.58222D+04

12 3 OF × OF × OF ×

13 3 OF × 79 2.57369D−03 84 2.28905D−20

14 3 OF × 594 3.98997D+00 594 3.98997D+00

15 3 172 2.39103D+00 189 2.39103D+00 189 2.39103D+00

16 2 OF × OF × OF ×

17 4 389 1.46138D−22 484 1.16019D−24 484 1.16019D−24

18 4 OF × OF × OF ×

aims to support better business decision-making. That is, given a business problem, the deci-

sion makers often try to identify the optimal decision from all possible choices. In this sense,

optimization algorithm is certainly one of the most important and widely used tools for the

processes of decision-making in business intelligence [20].

When the objective of the decision-making process can be defined as a function of some

decision variables, it can be solved by some optimization methods under certain constraints of

feasible decisions. Unfortunately, due to the complexity of business problems, the objective

functions of such decision-making processes often do not have explicit expressions. There-fore, it is very difficult to get any information about the derivatives of the objective function.

In this situation, our algorithm which only requires function values to search the optimal

solution can still feasible and effective.

In the following, we give a more clear description on how to use this proposed algorithm

to search the optimal (or approximately optimal)decision. Here we assume that a database is

available for the given business problem, which is in the form of 2-dimension table. These

data are described by some features (i.e., factors) and corresponding decisions. Then each

record of the data with n features can be regarded as a point in n-dimension space, and its

decision as the function value of this point. In this case, the values of the decision variables(i.e., the feature values) are discrete and we only have the information of the function values

of them. A most intuitive way to deal with the problem is that, we still search the optimal

solution in the current trust region at a given initial point, where the improved quadratic

interpolation models are used to obtain a better substitute of current point. Then, the feasible

point of closest to this substitute should be found and as the next iteration point, where the

feasible point is the true discrete point of a database. If there are more than one feasible points

123



J Glob Optim

which satisfy the least distance from the substitute, the one that has the least function value

would be chosen as the next iteration point. The process is repeated until the stopping criteria

are satisfied. Here the stopping criteria could be the same as those of the above algorithms

in Sect. 4, where one of them is controlled by trust region radius ρend . To summarize, our

proposed quadratic interpolation method has potential applications in business intelligencesystems.

Acknowledgments We cordially thank two anonymous referees for their valuable comments which lead

to the improvement of this paper. This work is supported by National Natural Science Foundation of China

(Grant No: 60773062, 60903088) and NaturalScience Foundation of Hebei Province(Grant No: A2010000188).

References

1. Garcia-Palomares, U.M., Gonzalez-Castaño, F.J., Burguillo-Rial, J.C.: A combined global & local search

approach to global optimization. J. Glob. Optim. 34, 409–426 (2006)

2. Vaz, A.I.F., Vicente, L.N.: A particle swarm pattern search method for bound constrained global optimi-

zation. J. Glob. Optim. 39, 197–219 (2007)

3. Pardalos, P.M., Resende, M. (eds.): Handbook of Applied Optimization. Oxford University Press,

Oxford (2002)

4. Winfield, D.: Function and functional optimization by interpolation in data tables. Ph.D Thesis, Hardvard

University, Cambridge, USA (1969)

5. Winfield, D.: Function minimization by interpolation in a data table. J. Inst. Math. Appl. 12,

339–347 (1973)

6. Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L.,

Ritter, K. (eds.) Nonlinear Programming, Academic Press, New York (1970)

7. Yuan, Y.X.: Conditions for convergence of trust region algorithms for nonsmooth optimization. Math.Program. 31, 220–228 (1985)

8. Yuan, Y.X.: On the superlinear convergence of a trust region algorithm fornonsmooth optimization. Math.

Program. 31, 269–285 (1985)

9. Du, J.Z., Pardalos, P.M., Wu, W.L.: Mathematical Theory of Optimization. Kluwer, Dordrecht (2001)

10. Lucia, A., Dimaggio, P.A., Depa, P.: A geometric terrain methodology for global optimization. J. Glob.

Optim. 29, 297–314 (2004)

11. Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Pro-

gram. 92, 555–582 (2002)

12. Powell, M.J.D.: On the use of quadratic models in unconstrained minimization without derivatives. Optim.

Methods Softw. 19, 399–411 (2004)

13. Han, L.X., Liu, G.H.: On the convergence of the UOBYQA method. J. Appl. Math. Comput. 16

(1–2), 125–142 (2004)14. Berghen, F.V., Bersini, H.: CONDOR, a new parallel, constrained extension of Powell’s UOBYQA algo-

rithm: experimental results and comparison with the DFO algorithm. J. Comput. Appl. Math. 181(1),

157–175 (2005)

15. Zhou, Q.H.: On the use of simplex methods in constructing quadratic models. Sci. China Ser. A

Math. 50(7), 913–924 (2007)

16. Torczon, V.: Multi-directional search: a direct search algorithm for parallel machines. Ph.D Thesis, Rice

University, Houston, TX, USA (1989)

17. Moré , J.J., Garbow, B.S., Hillstorm, K.E.: Testing unconstrained optimization software. ACM Trans.

Math. Softw. 7, 17–41 (1981)

18. Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)

19. Parlett, B.N.: The Symmetric Eigenvalue Problem. Prentice-Hall, Englewood Cliffs (1980)

20. Vercellis, C.: Business Intelligence. Wiley, New York (2009)

123

Documents

ZhouEtAl2010_QuadInterpolModels_JGlobOptim