Block 4 Nonlinear Systems Lesson 12 – Nonlinear Optimization

Is It Not the Best of All Possible Worlds?

An engineer who forgot to optimize

The Goal of this Lesson

Goal: To make this “best of all possible nonlinear worlds” - a little better!

Right on.

"It is demonstrable," said he, "that things cannot be otherwise than as they are; for as all things have been created for some end, they must necessarily be created for the best end.”

Candide by Voltaire

What do Operations Researchers (OR) do?

OR is concerned with optimal decision-making and modeling of deterministic and probabilistic systems that originate from real life. These applications, which occur in government, business, engineering, economics, and the natural and social sciences are largely characterized by the need to allocate scarce resources.

(Hillier & Lieberman)

The General Optimization Problem

1 2

1 2

Max/Min ( , ,..., )subj to :

( , ,..., ) , 1, 2,...,

n

i n i

f x x x

g x x x b i m£ì üï ï= =í ý

ï ï³î þwhere f, g1, …,gm are real-valued functions

The Single Variable Problem

open interval:

closed interval:

/ ( )Max Min f x where x

/ ( )Max Min f x where a x b

The Real Problem

localmax

localmin

unbounded

x

f(x)

a bclosed interval

globalmin

globalmax

Local Minimum

local min: x’ is a local minimum (maximum) if for an arbitrarysmall neighborhood, N, about x’, f(x’) () f(x) for all x in N.

x

f(x)

x’N

x’N

Global Minimum

global min: x* is a global minimum if f(x*) f(x) for all x such that a x b.

x

f(x)

a bx*

Global Maximum

global max: x* is a global maximum if f(x*) f(x) for all x such that a x b.

x

f(x)

x*a b

x

x

x

f(x)

+ -+

( )d f xdx

2

2

( )d f xdx

concave convex

stationary point stationary

point

Animated

Our very first nonlinear optimization problem

2 4

3

2

22

2

( ) 2

' 4 4 0

4 1 0; 0, 1

''( ) 4 12

''(0) 4 0 minimum''(1) 8 0 maximum''( 1) 8 0 maximum

y f x x xdyy x xdx

x x x

d y f x xdxfff

22

2

2

2

''( ) 4 12 0

12 4

1/ 3; .57735

d y f x xdxx

x x

-9-8-7-6-5-4-3-2-1012

-3 -2 -1 0 1 2 3

Global Minimum – Convex Functions

If f(x) is a convex function if and only if2

2

( ) 0;

( *)then if 0, * is the global minimum

d f x for xdx

df x xdx

x

f(x)

Global Maximum – Concave Functions

If f(x) is a concave function if and only if2

2

( ) 0;

( *)then if 0, * is the global maximum

d f x for xdx

df x xdx

x

f(x)

An Unbounded Function

2

( ) 100ln( )100 100'( ) ; ''( ) 0 therefore concave

f x x

f x f xx x

-250

-200

-150

-100

-50

0

50

100

150

200

0 1 2 3 4 5

The Single Variable Problem on the Open Interval

necessary condition for global solution:

f(x) is bounded and

sufficient condition:

for all x:

( ) 0d f xdx

2

2

2

2

( ) 0 for a min (convex)

( ) 0 for a max (concave)

d f xdx

d f xdx

A Bounded Example2

2 2

2

( ) 100ln( ) 2 ; 0100'( ) 4 0; 4 100; 25; 5

100''( ) 4 0

f x x x x

f x x x x xx

f xx

concave function

-300

-250

-200

-150

-100

-50

0

50

100

150

0 5 10 15 20

A word problem

A pipeline from the port in NYC to St. Louis, a distance of 1000 miles, is to be constructed by the Leak E. Oil Company with automatic shutoff values installed every x miles in the event of a leak. Environmentalists have estimated that such a pipeline is likely to have two major leaks during its lifetime. The cost of a valve is $500 and the cost of a cleanup in the event of a leak is $2500 per pipeline mile of oil spilled. How far apart should the valves be placed?

f(x) = 2 (2500) x + 500 (1000) / x0 x 1000

Animated

f(x) = 2 (2500) x + 500 (1000) / x

2

2

*

( ) 500,000'( ) 5000 0

500,000 100500

10miles

d f x f xdx x

x

x

2

2 3

( ) 2(500,000)''( ) 0 0d f x f x for xdx x

therefore f(x) is convex and x* is a global minimumAnimated

A word problem (continued)

The Single Variable Problem on the Closed Interval

( )Max f x where a x b

define a stationary point as any point x’ such that '( ') 0f x

find ' ' '1 2max ( ), ( ), ( ),..., ( ), ( )kx

f a f x f x f x f b

This looks too easy. There must be more to it.

Our very next example problem

4 3 21( ) 3 13 24 20.1 ; 1 64

f x x x x x x

3 2'( ) 9 26 24 0f x x x x

( 2) ( 3) ( 4) 02,3,4

x x xx

I bet that canbe factored!

4 3 21( ) 3 13 24 20.14

f x x x x x

3 2'( ) 9 26 24 0f x x x x 2''( ) 3 18 26f x x x

x f(x) f”(x)1 6.252 4 2 local/global min3 4.25 -1 local max4 4 2 local/global min6 20 global max

( 2) ( 3) ( 4) 02,3,4

x x xx

Our very next example problem (continued)

Another exampleFor a particular government 12-year health care program for the elderly, the number of people in thousands receiving direct benefits as a function of the number of years, t, after the start of the program is given by

For what value of t does the maximum number receive benefits?

326 32 0 12

3tn t t t

My health benefits

will expire soon!

The Answer

t = 0 (n=0),t= 4 (53/3),t = 8 (n = 42.67),t = 12 (n = 96)

32

2

2

2

2

24

2

28

6 323

12 32 0

4 8 0; 4,8

2 12;

4 0

4 0

t

t

tn t t

dn t tdtt t t

d n tdtd ndt

d ndt

local max

local min f

n

Multi -Variable Optimization

i.e. going from one to two

2-Variable Function with a Maximum

z = f(x,y)

2-Variable Function with both Maxima and Minimaz = f(x,y)

2-Variable Function with a Saddle Pointz = f(x,y)

The General Problem

1 2 1 2Max/Min ( , ,..., ) , ,...,n nf x x x x x x

necessary conditions:

1 2( , ,..., ) 0 for all jn

j

f x x xx

sufficient conditions:f(x1,x2,…,xn) is convex for a minimum

f(x1,x2,…,xn) is concave for a maximum

Recall Taylor’s Series Approximation in 2-variables?

00 0 0 0 0 0

0

0 0 0 0 00 0

0 0 0 0 0

( , ) ( , ) ( , ) ( , )

( , ) ( , )1( , ) ( , )2

x y

xx xy

yx yy

x xf x y f x y f x y f x y

y y

f x y f x y x xx x y y

f x y f x y y y

higher order terms

I sure do!

2-Variable Problemsufficient conditions:

0 0

0 0

( , ) 0 for a local min( , ) 0 for a local max

xx

xx

f x yf x y

20 0 0 0 0 0( , ) ( , ) ( , ) 0xx yy xyf x y f x y f x y

and

saddlepoint

20 0 0 0 0 0( , ) ( , ) ( , ) 0xx yy xyf x y f x y f x y

A 2-variable exampleMax f(x,y) = 100 – (x – 4)2 – 2 (y – 2)2

2( 4) 0 4

4( 2) 0 2

f x xxf y yy

necessary conditions:

2

2 2

22 2 2

2 0

4 ; 0

8 0

fx

f fy x y

f f fx y x y

sufficient conditions:

concave function

f(x,y) = 2x3 – 2x2 – 10x + y3 – 3y2 + 20

2

2

6 4 10 0

3 6 0

f x xxf y yy

2(3x – 5) (x + 1) = 0x = 5/3, -1

3y (y – 2) = 0y = 0, 2

Not Another Example? A Cubic no less!

…and it has four

solutions!

(x*,y*) = (5/3,0), (5/3, 2), (-1,0), (-1,2)

2

2

6 4 10

3 6

f x xxf y yy

2

2

2

2

2

12 4

6 6

0

f xx

f yy

fx y

x y

5/3 0 16 -6 saddle pt5/3 2 16 6 local min-1 0 -16 -6 local max-1 2 -16 6 saddle pt

2

2

fx

2

2

fy

Not Another Example (continued)

2 2

2

2 2

2

f fx x y

f fy x y

A special container must be constructed to transport 40cubic yards of material. The transportation cost is onedollar per round trip. It costs $10 per square yard to constructthe sides, $30 per square yard to construct the bottom of the container and $20 dollars to construct the ends. It has no top and no salvage value. It must be rectangular in shape and only one can be made. Find the dimensions which will minimize the construction and transportation costs.

I need a box, quick!

A Logistics Design Problem

The Formulation

let x = the length, y = the width, and z = the height

then volume = xyz and transportation cost = $1 [40 / (xyz)]cost of bottom = $30 xycost of sides = $10 xzcost of ends = $20 yz

40( , , ) 30 10 20f x y z xy xz yzxyz

The necessary conditions

40( , , ) 30 10 20f x y z xy xz yzxyz

2

2

2

( , , ) 40 30 10 0

( , , ) 40 30 20 0

( , , ) 40 10 20 0

f x y z y zx x yz

f x y z x zy xy z

f x y z x yz xyz

.56097761.12195511.6829321

yxz

Is the function convex?

2 2 2

2 3 2 3 2 3

2 2

2 2 2 2

2

2 2

( , , ) 80 ( , , ) 80 ( , , ) 80; ;

( , , ) 40 ( , , ) 4030; 10

( , , ) 40 20

f x y z f x y z f x y zx x yz y xy z z xyz

f x y z f x y zx y x y z x z x yz

f x y zy z xy z

I see, all 9 2nd partials must be

analyzed.

They show us how to do that in MSC 523. I am going to sign up today!

Power Plant Location

city nbr linesCincinnati 7Dayton 4Columbus 10Toledo 4Cleveland 12Youngstown 3

DPL desires to construct a nuclear power plant in Ohiothat will provide electrical power to the cities shownbelow. Also shown are the number of transmission linesrequired to meet each city’s demands for additional electricity. The problem is to locate the power plant sothat the total transmission loss is minimized.

The Great State of Ohio

x

y

(11,35)

(3,9)

(6,15) (15,18)

(31,30)

(24,34)

Euclidean Distances

x

y

(a,b)

(x,y)

(x – a)

(y – b) h

h2 = (x – a)2 + (y – b)2

2 2h x a y b

Animated

The Formulation

let x = the x-coordinate of power station y = the y-coordinate of power station(xi,yi) = coordinate of ith city wi = number of transmission lines to ith city

6

2 2

1

min ( , ) i i ii

f x y w x x y y

Euclideandistancesquared

The Solution – necessary conditions

6

1

6

1

2 ( ) 0

2 ( ) 0

i ii

i ii

fw x x

xf

w y yy

=

=

¶ = - =¶¶ = - =¶

åå

6 6

1 1

6 6

1 1

6

16

1

0i i ii i

i i ii i

i ii

ii

w x w x

x w w x

w xx

w

= =

= =

=

=

- =

=

=

å åå å

åå

6

16

1

i ii

ii

w yy

w

=

=

=åå

6

2 2

1

min ( , ) i i ii

f x y w x x y y

The Solution – sufficient conditions

6

1

6

1

2 ( ) 0

2 ( ) 0

i ii

i ii

fw x x

xf

w y yy

=

=

¶ = - =¶¶ = - =¶

åå

2 6

21

2 26

21

2 0

2 0 0

ii

ii

fw

xf f

wy x y

=

=

¶ = >¶¶ ¶= > =¶ ¶ ¶

åå

convex function

Why it is everywhere

convex. Truly you have found

the global minimum.

Power Plant Location

city nbr lines (wi) location wi * xi wi * yi

Cincinnati 7 3, 9 21 63Dayton 4 6, 15 24 60Columbus 10 15, 18 150 180Toledo 4 11, 35 44 140Cleveland 12 24, 34 288 408Youngstown 3 31, 30 93 90Totals 40 620 941

x* = 620 / 40 = 15.5 y* = 941 / 40 = 23.525

The Great State of Ohio

x

y

Waldo, OhioRoute 23Marion County

(15.5,23.5)

Much ado about Waldo Waldo is a village located in Marion County, Ohio. As of

the 2000 census, the village had a total population of 332. Waldo is known in the central Ohio region for excellent

fried baloney sandwiches from the G&R Tavern. Waldo is also home to several vineyards.

According to the United States Census Bureau, the village has a total area of 1.7 km² (0.6 mi²). 1.7 km² (0.6 mi²) of it is land and none of the area is covered with water.