Upload
tranbao
View
235
Download
0
Embed Size (px)
Citation preview
Matlab-based OptimizationBasic Capabilities
Gene Cliff (AOE/ICAM - [email protected] )3:00pm - 4:45pm, Monday, 4 February 2013
.......... FDI ..........
AOE: Department of Aerospace and Ocean EngineeringICAM: Interdisciplinary Center for Applied Mathematics
1 / 28
Matlab-based Optimization
Introduction & function functions ⇐
fminbnd
fminsearch
lsqnonneg
fzero
2 / 28
INTRO:
Basic Matlab provides several functions for elementaryminimization and zero-finding problems.
The Optimization Toolbox provides techniques for advancedproblems, and a gui interface.
The Optimization Toolbox is included with theVT concurrent license(http://www2.ita.vt.edu/software/department/products/mathworks/matlab annual concurrent/index.html),and with the VT student license(http://www2.ita.vt.edu/software/student/products/mathworks/matlab/index.html )
but not with the VT annual stand alone license(http://www2.ita.vt.edu/software/department/products/mathworks/matlab annual standalone/index.html).
Type ver 〈enter〉 at the Matlab prompt to see the list oftoolboxes available with your license.
In these notes we discuss optimization and zero-finding routines inbasic Matlab. The Optimization Toolbox will be discussed in thenext lecture. 3 / 28
Basic Matlab: linear least-squares
Given an m × n matrix A, a vector b ∈ IRm with m > n.
With more equations than unknowns we do not expect to findan x ∈ IRn such that Ax = b.
Instead we seek a least squares solution, i.e. an x thatminimizes ‖Ax − b‖2.
Such an x can be computed by mldivide (matrix left-divide)x = A\b or x = mldivide(A, b)
If m = n and A is nonsingular, A\b is similar to inv(A)*b,though it is not computed that way.
Other procedures (e.g. minres, gmres) are available if A issparse or if A ∗ x is the output of a function, e.g. my Ax(x).
In Matlab inv(A) is rarely a good idea.
4 / 28
Basic Matlab: function functions
Matlab uses this terminology to indicate that the arguments ofsome functions may refer to other functions.
A function handle is a Matlab value that provides a means ofcalling a function indirectly. You can pass function handles in callsto other functions (often called function functions). You can alsostore function handles in data structures for later use. A functionhandle is one of the standard Matlab data types.
S = functions(funhandle)returns, in the Matlab structure S, the function name, type,filename, and other information for the function handle stored inthe variable funhandle. Note that functions is intended fordebugging; do not use it your coding.
5 / 28
Matlab: named & local functions
A named function is defined in .m file; the name on the functionline should agree with the file name (which takes precedence).
local functions (or scoped functions) are included in the same fileas a named function and are visible only to it.
f u n c t i o n x ou t = eg named ( x i n )% Simple example o f l o c a l ( sub−) f u n c t i o n s% We can make use o f l o c a l 1 and l o c a l 2% which a r e not v i s i b l e to o th e r f u n c t i o n s
x ou t = l o c a l 1 ( x i n ) + l o c a l 2 ( x i n ) ;% S = f u n c t i o n s ( @ l o c a l 1 )end% Sub f u n c t i o n s go he r ef u n c t i o n out = l o c a l 1 ( i n )% Square the e n t r i e s
out = i n . ˆ 2 ;endf un c t i o n out = l o c a l 2 ( i n )% Mu l t i p l y by 2
out = 2∗ i n ;end
Note that each function/subfunction requires an end statement.
6 / 28
Matlab: nested functions
nested functions are included in the same file as a named functionand are visible only to it. These functions are included before theend statement of the host function.
f u n c t i o n x ou t = eg ne s t e d ( x i n )% Simple example o f ne s t ed ( sub−) f u n c t i o n s% We can make use o f l o c a l 1 and l o c a l 2% which a r e not v i s i b l e to o th e r f u n c t i o n s%% Va r i a b l e s w i t h i n the scope o f t h i s f u n c t i o n a r e a v a i l a b l e to the ne s t ed f u n c t i o n s
temp = cos ( x i n ) ;x ou t = l o c a l 1 ( x i n ) + l o c a l 2 ( x i n ) ;
% S = f u n c t i o n s ( @ l o c a l 1 )% ne s t ed f u n c t i o n s a r e ’ i n s i d e ’ the c o n t a i n i n g f u n c t i o n s ’ end ’
f u n c t i o n out = l o c a l 1 ( i n )% Square the e n t r i e sout = i n .ˆ2 + temp ; % ’ temp ’ i s a v a i l a b l e to ne s t ed f u n c t i o n s
end % end f o r l o c a l 1f u n c t i o n out = l o c a l 2 ( i n )
% Mu l t i p l y by 2out = 2∗ i n ;
end % end f o r l o c a l 2
end % end f o r e g n e s t e d
7 / 28
Matlab: a simple named function - my func
A simple named function
funct ion y = my func ( x , param )%Simple pa r ame t e r i z e d s c a l a r f u n c t i o n
y = −x .∗ cos ( param∗x ) ;end
Examine with functions()
>> S = functions(@my_func)
S =
function: ’my_func’
type: ’simple’
file: [1x70 char]
>> S.file
ans =
/Users/ecliff/Documents/classes/fdi_matlab_2013/optimization/sample_code/my_func.m
8 / 28
Matlab: anonymous functions
anonymous functions are defined locally and do not exist in a file.
To assign a handle to an anonymous function use the ‘@ ’symbol
% This can be a f u n c t i o n or a s c r i p t%% Va r i a b l e s d e f i n e d i n t h i s code can be used% as arguments i n d e f i n i n g an anonymous f u n c t i o n
paramete r = s t u f f ;
%% Def i n e the ( hand l e to the ) anonymous f u n c t i o nf u n hnd l = @( x ) my func ( x , paramete r ) ;
%% Use the anonymous f u n c t i o ntemp = f e v a l ( f un hnd l , 3 . 2 ) ;
feval is a built-in function whose first argument is a function
9 / 28
Matlab: using functions() again
param = 2;fun_hndl = @(x) my_func(x, param);param = 4;S = functions(fun_hndl)S =
function: ’@(x)my_func(x,param)’type: ’anonymous’file: ’’
workspace: {[1x1 struct]}fields(S.workspace{1})ans =
’param’S.workspace{1}.paramans =
2
10 / 28
Matlab-based Optimization
Introduction & function functions
fminbnd ⇐
fminsearch
lsqnonneg
fzero
11 / 28
Basic Matlab: fminbnd
fminbnd computes the minimum of a scalar-valued function of ascalar variable within a given interval .
>> param = 2;>> fun_hndl = @(x) my_func(x, param);>> opt = optimset(’fminbnd’);>> opt = optimset(opt, ’Display’, ’iter’);>> xs = fminbnd(fun_hndl, 0, 1, opt);Func-count x f(x) Procedure
1 0.381966 -0.275826 initial2 0.618034 -0.203032 golden.... 3 lines omitted6 0.430208 -0.280548 parabolic7 0.430161 -0.280548 parabolic8 0.430127 -0.280548 parabolic
Optimization terminated:the current x satisfies the terminationcriteria using OPTIONS.TolX of 1.000000e-04
12 / 28
Basic Matlab: available arguments for fminbnd
fminbnd can be invoked with additional output argumentsdoc fminbndfor details
Syntax
x = fminbnd(fun,x1,x2)x = fminbnd(fun,x1,x2,options)[x,fval] = fminbnd(...)[x,fval,exitflag] = fminbnd(...)[x,fval,exitflag,output] = fminbnd(...)
13 / 28
Matlab-based Optimization
Introduction & function functions
fminbnd
fminsearch ⇐
lsqnonneg
fzero
14 / 28
Basic Matlab: fminsearch
fminsearch computes a candidate for the minimum of ascalar-valued function of a vector variable from agiven starting point .
With x ∈ IRn the procedure is based on a simplex of (n + 1)points, and proceeds to produce a test point to replace theworst point in the simplex.
The Matlab algorithm begins by evaluating the function atx0 and the n perturbed points xı = x0 + δ ∗ eı, withδ = 0.05‖x0‖. The points are re-labeled tox , = 1, ..., n + 1 so that f (x) ≤ f (x+1), i.e. in ascendingorder.
Compute x , the average over the set {xk | k = 1, .., n}.
15 / 28
Basic Matlab: fminsearch (cont’d)
The algorithm then proceeds as described in the following figures:
16 / 28
fminsearch for Rosenbrock’s example
% Sc r i p t to d i s p l a y use o f fm i n s e a r ch on the Rosenbrock v a l l e y f u n c t i o n
%% E .M. C l i f f% I n t e r d i s c i p l i n a r y Cente r f o r App l i ed Mathematics% V i r g i n i a Tech , B lacksburg , VA 24061% e c l i f f @ v t . edu%% FDI Shor t c ou r s e on Matlab−based Opt im i z a t i on − b a s i c c a p a b i l i t i e s
% Example u s i n g the Rosenbrock v a l l e y f u n c t i o n% The f i r s t term i n the sum i s z e r o on the pa r abo l a x (2)=x ( 1 ) . ˆ 2 , and% i n c r e a s e s r a p i d l y as x ( : ) d e p a r t s from the ’ v a l l e y ’ .
f h n d l = @( x ) 100∗( x (2 ) − x (1)∗ x (1 ) )ˆ2 + (1 − x ( 1 ) ) ˆ 2 ; % anonymous f u n c t i o n
x0 = [−1; 1 ] ; % s t a r t i n g po i n t
opt = opt imse t ( ’ fm i n s e a r ch ’ ) ;opt = opt imse t ( opt , ’ D i s p l a y ’ , ’ i t e r ’ ) ;
x s = fm in s e a r ch ( f hnd l , x0 , opt ) ; % invoke Ne lde r Meade
Iteration Func-count min f(x) Procedure
0 1 4
1 3 4 initial simplex
2 5 4 contract inside
3 7 4 contract inside
4 9 4 contract outside
5 11 4 contract inside
6 13 3.93698 expand
99 185 2.64787e-09 contract outside
100 187 4.05759e-10 contract inside
Optimization terminated:
the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e-04
and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e-04
18 / 28
Setting algorithmic parameters: optimset
Syntax options =optimset(’param1’,value1,’param2’,value2,...)options = optimset(optimfun)options = optimset(oldopts,’param1’,value1,...)options = optimset(oldopts,newopts)
Which parameters are meaningful depends on the function andthe algorithm.
20 / 28
Matlab-based Optimization
Introduction & function functions
fminbnd
fminsearch
lsqnonneg⇐
fzero
22 / 28
lsqnonneg
x = lsqnonneg( A, b)minimize ‖Ax − b‖2 subject to x ≥ 0
>> A = [ 1 1; 1 0; 0 1];>> b = [0; 1; -1];>> x1 = lsqnonneg(A, b)
x1 =
0.50000
23 / 28
Matlab-based Optimization
Introduction & function functions
fminbnd
fminsearch
lsqnonneg
fzero⇐
25 / 28
fzero
xs = fzero(fun, x0) tries to find a zero of thescalar-valued function fun near the scalar x0.
if x0 is a vector of length 2, then fzero assumes thatfun(x0(1)) and fun(x0(2)) have opposite signs. Assumingthat fun is continuous, fzero will return a scalar near wherefun changes sign.
If unsuccessful, fzero will return a NaN.
fzero(fun, x0, optim) allows one to set OPTIONS.
26 / 28
fzero: an example
>> fh = @(x) sin(x)
fh =
@(x)sin(x)
>> fzero(fh, 1)
ans =
1.5485e-24
>> fzero(fh, 3)
ans =
3.141627 / 28