Lecture3-2HookeJeeves

8/7/2019 Lecture3-2HookeJeeves

1/12

1

Smooth, Unconstrained Nonlinear

Optimization without Gradients

Hooke Jeeves

6/16/05


2/12

2

Hooke Jeeves or Pattern Search

Zero order

No derivatives

No line searches Works in discontinuous domain

No proof of convergence

Tool when other tools fails

References: Evolution and Optimum Seeking by Hans-Paul Schwefel

Mark Johnson code handout

Characteristics


3/12

3

Hooke JeevesWith downhill simplex is the simplest algorithm in iSIGHT

Has essentially no formal diagnostics for outputs.

The algorithm is an unconstrained optimization algorithm

which can also be used in constrained situations.

Expected number of iterations =

StepSizeReductionFactor ** n < Termination Step Size

StepSizeReductionFactor between 0 1. Default .5

The larger the value the slower the convergence.


4/12

4


5/12

5


6/12

6

Hooke Jeeves AlgorithmTermination step size = e

Step size reduction = rho

Step 0: InitializationChoose a starting point, an accuracy bound e > 0,

and initial step lengths (current value * rho).

If current value = 0.0 make step length rho

situation)original(back toesxxreplaceotherwise

(success)2Steptogo),F(x)xF(if

direction)negativeinstep(discretee2s-xxreplaceotherwise

l)first trial(successfu2Steptogo),F(x)xF(if

direction)positiveinstep(discreteesxxConstruct

moveyExplorator:1Step

i

(k)

i

1)-i(k,

i

(k)

i

1)-i(k,

i

(k)

i

1)-i(k,

d!d

d

d!d

d

!d


7/12

7i

(k)

i

1)-i(k,

i

(k)

i

1)-i(k,

i

(k)

i

1)-i(k,

n)1,-(ki

n)(k,i

(k)i

1)(ki

n)1,-(kn)(k,1,0)(k

(k,0)1,0)(k(k,0)n)(k,

i)(k,

esxxreplaceother ise

6steptogo)(x)x(I

e2s-xxreplaceother ise

6steptogo)(x)x(I

esxxonstruct

ionextrapolata ter nxploratio:5tep

ar)somovepatterntheocontrolsuccessnoisThere:( bserve

1isetand1kkincrease

n1,iallor)x-sign(xssand

tion)(extrapolax-2xet x

moveattern:4tep

9steptogoandxset x),(x)(xI

.directionsallinailureor totalTest:3tep

1.steptogoand1iiincreasen,iI

xet x

coordinatenexts itch toandetention:2tep

d!d

d

d!d

d

!d

!!

!!

!

!u

!

d!


8/12


9/12

9


10/12

10


11/12

11

Lab Rerun the Spring_Start.desc file using

Hooke_Jeeves with the default step size.

Does it reach the same optimum? How many function calls did it take?

Is this more or less efficient then SteepestDescent?

On the next slide, label the X1 Step Size, X2 StepSize and algorithm step number next to each rowfor first 7 run counters.


12/12

12

Spring Hooke Initial Steps

Documents

Lecture3-2HookeJeeves