8/7/2019 Lecture3-2HookeJeeves
1/12
1
Smooth, Unconstrained Nonlinear
Optimization without Gradients
Hooke Jeeves
6/16/05
8/7/2019 Lecture3-2HookeJeeves
2/12
2
Hooke Jeeves or Pattern Search
Zero order
No derivatives
No line searches Works in discontinuous domain
No proof of convergence
Tool when other tools fails
References: Evolution and Optimum Seeking by Hans-Paul Schwefel
Mark Johnson code handout
Characteristics
8/7/2019 Lecture3-2HookeJeeves
3/12
3
Hooke JeevesWith downhill simplex is the simplest algorithm in iSIGHT
Has essentially no formal diagnostics for outputs.
The algorithm is an unconstrained optimization algorithm
which can also be used in constrained situations.
Expected number of iterations =
StepSizeReductionFactor ** n < Termination Step Size
StepSizeReductionFactor between 0 1. Default .5
The larger the value the slower the convergence.
8/7/2019 Lecture3-2HookeJeeves
4/12
4
8/7/2019 Lecture3-2HookeJeeves
5/12
5
8/7/2019 Lecture3-2HookeJeeves
6/12
6
Hooke Jeeves AlgorithmTermination step size = e
Step size reduction = rho
Step 0: InitializationChoose a starting point, an accuracy bound e > 0,
and initial step lengths (current value * rho).
If current value = 0.0 make step length rho
situation)original(back toesxxreplaceotherwise
(success)2Steptogo),F(x)xF(if
direction)negativeinstep(discretee2s-xxreplaceotherwise
l)first trial(successfu2Steptogo),F(x)xF(if
direction)positiveinstep(discreteesxxConstruct
moveyExplorator:1Step
i
(k)
i
1)-i(k,
i
(k)
i
1)-i(k,
i
(k)
i
1)-i(k,
d!d
d
d!d
d
!d
8/7/2019 Lecture3-2HookeJeeves
7/12
7i
(k)
i
1)-i(k,
i
(k)
i
1)-i(k,
i
(k)
i
1)-i(k,
n)1,-(ki
n)(k,i
(k)i
1)(ki
n)1,-(kn)(k,1,0)(k
(k,0)1,0)(k(k,0)n)(k,
i)(k,
esxxreplaceother ise
6steptogo)(x)x(I
e2s-xxreplaceother ise
6steptogo)(x)x(I
esxxonstruct
ionextrapolata ter nxploratio:5tep
ar)somovepatterntheocontrolsuccessnoisThere:( bserve
1isetand1kkincrease
n1,iallor)x-sign(xssand
tion)(extrapolax-2xet x
moveattern:4tep
9steptogoandxset x),(x)(xI
.directionsallinailureor totalTest:3tep
1.steptogoand1iiincreasen,iI
xet x
coordinatenexts itch toandetention:2tep
d!d
d
d!d
d
!d
!!
!!
!
!u
!
d!
8/7/2019 Lecture3-2HookeJeeves
8/12
8/7/2019 Lecture3-2HookeJeeves
9/12
9
8/7/2019 Lecture3-2HookeJeeves
10/12
10
8/7/2019 Lecture3-2HookeJeeves
11/12
11
Lab Rerun the Spring_Start.desc file using
Hooke_Jeeves with the default step size.
Does it reach the same optimum? How many function calls did it take?
Is this more or less efficient then SteepestDescent?
On the next slide, label the X1 Step Size, X2 StepSize and algorithm step number next to each rowfor first 7 run counters.
8/7/2019 Lecture3-2HookeJeeves
12/12
12
Spring Hooke Initial Steps
Recommended