ethz_lecture6

7/30/2019 ethz_lecture6

http://slidepdf.com/reader/full/ethzlecture6 1/34

Engineering Optimization – Concepts and Applications

Engineering Optimization

Concepts and Applications

Fred van KeulenMatthijs Langelaar

CLA H21.1

[email protected]

http://www.tudelft.nl/




Recap / overview

Optimization problem

Definition

Checking

Negative null form Model

Special topics

Sensitivity analysis

Topology optimization

Linear / convex problems

Solution methods

Optimality criteria

Optimization algorithms

Unconstrained problems

Optimality criteria

Optimization algorithms

Constrained problems





Summary optimality conditions

● Conditions for local minimum of unconstrained problem:

– First Order Necessity Condition:

– Second Order Sufficiency Condition: H positive definite

0 f

● For convex f in convex feasible domain:

condition for global minimum:

– Sufficiency Condition: 0 f



http://slidepdf.com/reader/full/ethzlecture6 4/34Engineering Optimization – Concepts and Applications

Stationary point nature summary

Definiteness H Nature x*

Positive d. Minimum

Positive semi-d. Valley

Indefinite Saddlepoint

Negative semi-d. Ridge

Negative d. Maximum

HyyT

i

0

0

0

0

0




Complex eigenvalues?

● Question: what is the nature of a stationary point when

H has complex eigenvalues?

● Answer: this situation never occurs, because H is

symmetric by definition. Symmetric matrices have real

eigenvalues (spectral theory).




Nature of stationary points

● Nature of initial position depends on load (buckling):

F

k 1

k 2

l

2

5.9,10

2,6

21

k k

l F

21 coscos l l dz

Fdz k k 2

222

1121

21

0

1222

2111

cossin

cossin

Fl k

Fl k

21221

21211

coscossinsin

sinsincoscos

Fl k Fl

Fl Fl k

0

0

Fl k

Fl k

2

1

0

0

l

k

l

k F crit

21 ,min

75.4 crit F




Nature of stationary points (2)

75.3 F 75.4 F 75.5 F 75.6 F 75.7 F

2

1




Unconstrained optimization

algorithms● Single-variable methods

– 0th order (involving only f )

– 1st order (involving f and f ’ )

– 2nd order (involving f , f ’ and f ” )

● Multiple variable methods





Why optimization algorithms?

● Optimality conditions often cannot be used:

– Function not explicitly known (e.g. simulation)

– Conditions cannot be solved analytically

● Example: 122

21

2

1̀21

x xe xe x x x x f

Stationary points:

021

021

12

12

221

2

2

211

1

x x

x x

e xe x x x

f

e x xe x x f

f 0





0th order methods: pro/con

● Strengths:

– No derivatives needed

– Work also for

discontinuous / non-

differentiable functions

– Easy to program

– Robust

● Weaknesses:

– (Usually) less efficient

than higher order

methods (many function

evaluations)





Minimization with one variable

● Why?

– Simplest case: good starting point

– Used in multi-variable methods during line search

x x xt s

x f x

..

)(min● Setting:

f

x

Model

Optimizer

Iterative process:





Termination criteria

● Stop optimization iterations when:

– Solution is sufficiently accurate (check optimality criteria)

– Progress becomes too slow:

– Maximum resources have been spent

– The solution diverges

– Cycling occurs

f k k xk k x f x f x x )()(, 11

xa xb





Brute-force approach

● Simple approach: exhaustive search

● Disadvantage: rather inefficient

f

x

L0

0

2

1n L L

n

n points:

Final interval size = Ln





Basic strategy of 0th order

methods for single-variable case1. Find interval [a0, b0] that contains the minimum

(bracketing)

2. Iteratively reduce the size of the interval [ak , b

k ]

(sectioning)

3. Approximate the minimum by the minimum of a simple

interpolation function over the interval [a N , b N ]

● Sectioning methods:

– Dichotomous search

– Fibonacci method

– Golden section method





Bracketing the minimum

f

x

x1 x2 = x1+ x3 = x2+g x4 = x3+g2

[a0, b0]

Starting point x1, stepsize , expansion parameter g: user-defined





Unimodality

● Bracketing and sectioning methods work best for

unimodal functions:

“An unimodal function consists of exactly one

monotonically increasing and decreasing part”





L0

a0 b0

Dichotomous searchMain Entry: di·chot·o·mous Pronunciation: dI-'kät-&-m&s also d&-

Function: adjective

: dividing into two parts

● Conceptually simple

idea:

– Try to split interval in half in each step

d << L0 L0 /2

: f f





Dichotomous search (2)

● Interval size after 1 step (2 evaluations):

d 012

1 L L

● Interval size after m steps (2m evaluations):

mmm

L L

2

11

2

0 d

● Proper choice for d :

m

mm

mm

L L L L L

21010102

0

ideal

0ideal

d

L0





Dichotomous search (3)

● Example: m = 10

10242

0

10

0ideal

10

L L L

1024010

0

ideal

10 L L d

0

10 log L

Lm

Idealinterval

reduction

10240

0 Ld

m





Sectioning - Fibonacci

● Situation:

minimum

bracketed

between x1

and x3 : x1 x3

Fibonacci,

1180?-1250?

x2

● Test new points and reduce interval

x4 x4

● Optimal point placement?





Optimal sectioning

● Fibonacci method: optimal sectioning method

● Given:

– Initial interval [a0, b0]

– Predefined total number of evaluations N, or:

– Desired final interval size





Fibonacci sectioning - basic idea

● Start at final interval and use symmetry and maximum

interval reduction:

I N-1 = 2 I N

I N-2 = 3 I N

I N-3 = 5 I N

I N-4 = 8 I N

I N-5 = 13 I N

d << I N

I N

N j j N I F I 1 Fibonacci number j F

12 k k k I I I j N j N j N I I I 12





Sectioning – Golden Section

● For large N , Fibonacci fraction b converges to golden

section ratio f (0.618034…):

N

N

F

F 1

● Golden section method

uses this constant interval

reduction ratio f

f

f 1

1





Sectioning - Golden Section

● Origin of golden section: I 1

I 3 = f I 2

I 2 = f I 1

I 2 = f I 1321 I I I

1

2

11 I I I f f

2

5101 2,1

2 f f f

618034.02

15

f

● Final interval: 1 I I N

N f





0

10 log L

Lm

Ideal dichotomous

interval reduction

Fibonacci

Golden

section

Evaluations

Comparison sectioning methods

N

Dichotomous 12

Golden section 9

Fibonacci 8

(Exhaustive 99)

Example:

reduction to 2% of

original interval:

● Conclusion: Golden section simple and near-optimal





Quadratic interpolation

● Three points of the bracket define interpolating quadratic

function:

cbxax x f

2

)(

~

ai bi ● New point evaluated at

minimum of parabola:a

b xbax f new

202'

~

xnew

● For minimum: a > 0!

● Shift xnew when very close to existing point

ai+1 bi+1















Cubic interpolation

● Similar to quadratic interpolation, but with 2 points and

derivative information:

d cxbxax x f 23

)(

~

ai bi





Bisection method

● Optimality conditions: minimum at stationary point

Root finding of f ’

● Similar to sectioning methods, but uses derivative:

f f ’





Secant method

● Also based on root finding of f ’

● Uses linear interpolation

f ’















Newton’s method

● Again, root finding of f ’

● Basis: Taylor approximation of f ’:2

'( ) '( ) ''( ) ( ) f x h f x f x h o h Linear approximation

)("

)('

x f

x f h

● New guess:)(")('

1

k

k k k k k

x f x f xh x x





Newton’s method

● Best convergence of all methods:

● Unless it diverges

xk

f’

xk+1

xk+2

f’

xk

xk+1 xk+2





Summary single variable methods

● Bracketing +

Dichotomous sectioning

Fibonacci sectioning

Golden ratio sectioning

Quadratic interpolation

Cubic interpolation

Bisection method

Secant method

Newton method

● And many, many more!

0th order

1st order

2nd order

In practice:additional “tricks”needed to dealwith:

Multimodality

Strongfluctuations

Round-off errors

Divergence


Documents

ethz_lecture6