Solving equations using linear optimization techniques

Uchenna Ezeobi 09/07/2014

Optimization Homework

1a) Range of Convergence

(i) f(x)=x4-12x3+47x2-60x ----------------------------------Eqn(1)

The graph of f(x)=0 is shown below for the five different method.

Figure 1: Graphs of the function (left) and convergence (Right) at root zero.

Bisection Method: Convergence will only occur between two intervals (a, b) when f (a)*f (b) < 0. For the bisection

method for root 0. Convergence will occur for the x-value range (-∞ 3), (3 5), (0 4) and (4 ∞)

Newton Method: For any continuous function between any intervals f’(x) ≠0 for any intervals for convergence to be

achieved. And also the other roots should not be included in the intervals. The interval for convergence are (-∞

0.943), (0.943 3.456), (3.456 4.601) and (4.601, ∞)

Quasi –Newton’s Method I and II: For these method, we should use the same range of convergence used for

Newton’s method because it approximates the derivative of a function using scant method. f’(x) ≠ 0. . The interval

for convergence are (-∞ 0.943), (0.943 3.456), (3.456 4.601) and (4.601, ∞)

Dennis and Schnabel: These method is a more modified version of Quasi-newton method. The range of converges

is the same as that of Newton Method. The interval for convergence are (-∞ 0.943), (0.943 3.456), (3.456 4.601)

and (4.601, ∞)

(ii) f’(x)=4x3-36x2+94x-60 --------------------------------------------Eqn(2)


Figure 1: Graphs of the function (Right) and convergence (Left) at root 0.943.

Bisection Method: Convergence will occur for the x-value range (-∞ 3.456), (0.943, 4.601) and (3.456, ∞).

Newton Method: Convergence will occur for the x-value range (-∞ 1.9199), (1.9199 4.0801) and (4.0801 ∞)

Quasi –Newton’s Method: Convergence will occur for the x-value range (-∞ 1.9199), (1.9199 4.0801) and (4.0801

∞)

Dennis and Schnabel: Convergence will occur for the x-value range (-∞ 1.9199), (1.9199 4.0801) and (4.0801 ∞)

(iii) f(x)=x2-3 ------------------------------------------------------------------Eqn(3)

Figure 3: Graphs of the function (Right) and convergence (Left) at root -1.732.

Bisection Method: Convergence will occur for the x-value range (-∞ 1.732) and (-1.732, ∞).

Newton Method: Convergence will occur for the x-value range (-∞ 0) and (0, ∞).


Quasi –Newton’s Method: Convergence will occur for the x-value range (-∞ 0) and (0, ∞).

Dennis and Schnabel: Convergence will occur for the x-value range (-∞ 0) and (0, ∞).

(iv) f’(x)=2x ------------------------------------------------------------------Eqn(4)

Figure 4: Graphs of the function (Right) and convergence (Left) at root 0.

Bisection Method: Convergence will occur for the x-value intervals (-∞ ∞).

Newton Method: Convergence will occur for the x-value intervals (-∞ ∞).

Quasi –Newton’s Method: Convergence will occur for the x-value intervals (-∞ ∞).

Dennis and Schnabel: Convergence will occur for the x-value intervals (-∞ ∞).

(v) f(x)=x2-1 ----------------------------------------------------------Eqn5


Figure 5: Graphs of the function (Right) and convergence (Left) at root -1.732.

Bisection Method: Convergence will occur for the x-value range (-∞ 1.732) and (-1.732, ∞).

Newton Method: Convergence will occur for the x-value range (-∞ 0) and (0, ∞).

Quasi –Newton’s Method: Convergence will occur for the x-value range (-∞ 0) and (0, ∞).

Dennis and Schnabel: Convergence will occur for the x-value range (-∞ 0) and (0, ∞).

(vi) f’(x)=2x ------------------------------------------------------------------Eqn(4)


Bisection Method: Convergence will occur for the x-value intervals (-∞ ∞).




(vii) f(x)=x2-2x+1--------------------------------------------------------------------Eqn(7)



Bisection Method: From the two graph, this method never converges. For any closed interval (a b) between the

root 1, f(a)*f(b)>0 which does not satisfy the condition for bisection method.

Newton Method: Convergence will occur for the x-value intervals (-∞ 0) and (0, ∞).

Quasi –Newton’s Method: Convergence will occur for the x-value intervals (-∞ 0) and (0, ∞).

Dennis and Schnabel: Convergence will occur for the x-value intervals (-∞ 0) and (0, ∞).

(viii) f(x)=2x-2 -----------------------------------------------------------------Eqn(8)

Figure 8: Graphs of the function (Right) and convergence (Left) at root 1

Bisection Method: Convergence will occur for the x-value(a b) intervals (-∞ 1) for ‘a’ and (1 ∞) for b.




(ix) f(x)=atan(x) -----------------------------------------------------------------Eqn(9)


Figure 9: Graphs of the function (Right) and convergence (Left) at root 0

Bisection Method: These will converge for interval (a,b) a should take any value between(-∞ 0) and b should be

any value (0 ∞)

Newton Method: : These method will converge at an interval because if │f(x+)│> │f(xc) │ then the half of the sum

of the x+ and xc is used. The interval of Convergence is

Quasi –Newton’s Method: The same explanation for Newton Method

Dennis and Schnabel: The same explanation for Newton Method

(x) f(x) = 1/(1+x2) -------------------------------------------------------------------Eqn10

Figure 10: Graphs of the function (Right) and convergence (Left) at root infinity


Bisection Method: The method diverges because the function has no roots. That means there is no x-intercepts.

Therefore f(x1)*f(x2)>0 which does not satisfy the Bisection criteria.

Newton Method: The function approaches 0 when x approaches. There is no root and therefore these method

never converges.

Quasi –Newton’s Method: The same explanation as Newton Method

Dennis and Schnabel: The same explanation as Newton and Quasi-Newton Method.

1b)

Code to solve problem

% Called by test_zero_funs.m to plot the results of all the methods

%Gets the number of rows and column for the data bis_evals plotted %x1=number of rows, bis_num= number of columns. The same applies to all %methods

[x1 bis_num]=size(bis_evals); [x2 newt_num]=size(newt_evals); [x3 q_newt_num]=size(q_newt_evals); [x4 q_newt2_num]=size(q_newt2_evals); [x5 ds_num]=size(ds_evals);

%Assigns an empty array that would be used to collect all the points for %P=1

bis_P_1_points = [ ]; newt_P_1_points = [ ]; q_newt_P_1_points = [ ]; q_newt2_P_1_points = [ ]; ds_P_1_points = [ ];

%Assigns an empty array that would be used to collect all the points for %P=2

bis_P_2_points = [ ]; newt_P_2_points = [ ]; q_newt_P_2_points = [ ]; q_newt2_P_2_points = [ ]; ds_P_2_points = [ ];

%Assigns an empty array that would be used to collect all the points for %the x-xlues for both P=1 and P=2.

bis_plot_x = [ ]; newt_plot_x = [ ]; q_newt_plot_x = [ ]; q_newt2_plot_x = [ ]; ds_plot_x = [ ];


% A for loop that collects all the values for both P=1 and P=2 for i = 1:1:bis_num-1 P_1= (bis_evals(i+1)-root1)/(bis_evals(i)-root1); %Computes when P=1 for

rate of convergence P_2= (bis_evals(i+1)-root1)/(bis_evals(i)-root1); %Computes when P=2 for

rate of convergence

bis_P_1_points=[bis_P_1_points P_1]; %puts every points for P=1 into an

array

bis_P_2_points = [bis_P_2_points P_2]; %puts every points for P=2 into

an array

bis_plot_x = [bis_plot_x i]; %stores the iterations into an array end

% The same applies for all the methods

for i = 1:1:newt_num-1 P_1= (newt_evals(i+1)-root1)/(newt_evals(i)-root1); %Computes when P=1

for rate of convergence P_2= (newt_evals(i+1)-root1)/(newt_evals(i)-root1); %Computes when P=2

for rate of convergence

newt_P_1_points=[newt_P_1_points P_1];

newt_P_2_points=[newt_P_2_points P_2];

newt_plot_x = [newt_plot_x i]; end

for i = 1:1:q_newt_num-1

P_1= (q_newt_evals(i+1)-root1)/(q_newt_evals(i)-root1); %Computes when

P=1 for rate of convergence

P_2= (q_newt_evals(i+1)-root1)/(q_newt_evals(i)-root1); %Computes when


q_newt_P_1_points=[q_newt_P_1_points P_1];

q_newt_P_2_points=[q_newt_P_2_points P_2];

q_newt_plot_x = [q_newt_plot_x i]; end

for i = 1:1:q_newt2_num-1 P_1= (q_newt2_evals(i+1)-root1)/(q_newt2_evals(i)-root1); %Computes when



P_2= (q_newt2_evals(i+1)-root1)/(q_newt2_evals(i)-root1); %Computes when


q_newt2_P_1_points=[q_newt2_P_1_points P_1];

q_newt2_P_2_points=[q_newt2_P_2_points P_2];

q_newt2_plot_x = [q_newt2_plot_x i]; end

for i = 1:1:ds_num-1 P_1= (ds_evals(i+1)-root1)/(ds_evals(i)-root1); %Computes when P=1 for

rate of convergence

P_2= (ds_evals(i+1)-root1)/(ds_evals(i)-root1); %Computes when P=2 for

rate of convergence

ds_P_1_points=[ds_P_1_points P_1];

ds_P_2_points=[ds_P_2_points P_2];

ds_plot_x = [ds_plot_x i]; end

figure; subplot(5,2,1),plot (bis_P_1_points, bis_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Bisection Method P=1')

subplot (5,2,2),plot (newt_P_1_points, newt_P_1_points); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Newton with exact derivatives P=1')

subplot (5,2,3),plot (q_newt_P_1_points, q_newt_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Quasi-Newton with \delta x finite diff. P=1')

subplot (5,2,4),plot (q_newt2_P_1_points, q_newt2_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Quasi-Newton using previous value P=1')

subplot (5,2,5),plot (ds_P_1_points, ds_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1));


title(sprintf('%s', func2str(fun))); legend('Dennis and Schnabel Method P=2');

subplot(5,2,6),plot (bis_P_2_points, bis_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Bisection Method P=2');

subplot (5,2,7),plot (newt_P_2_points, newt_P_1_points); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Newton with exact derivatives P=2')

subplot (5,2,8),plot (q_newt_P_2_points, q_newt_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Quasi-Newton with \delta x finite diff. P=2')

subplot (5,2,9),plot (q_newt2_P_2_points, q_newt2_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Quasi-Newton using previous value P=2')

subplot (5,2,10),plot (ds_P_2_points, ds_plot_x); xlabel('Number of function Evaluations'); ylabel(sprintf('Distance from the root (r=%5.2f)', root1)); title(sprintf('%s', func2str(fun))); legend('Dennis and Schnabel Method P=2')

(i) f(x)=x4-12x3+47x2-60x


Figure 11: Plot of rate of convergence

Bisection Method: Bisection method did not exhibit any characteristics.

Newton Method: : From the graph, when p=1 the graph approached 0 and when p=2 the graph approached 0.8.

This shows that it this method is Q-superliner and Q-quadratic.

Quasi –Newton’s Method I and II: It showed a characteristic of Q-super linear.

Dennis and Schnabel: This showed a characteristic of a Q-super linear.

(ii) f’(x)=4x3-36x2+94x-60


Bisection Method: Bisection Method does is Q-quadratic.

Newton Method: : From the graph it is both Q-super linear and Q- quadratic. A drastic change occurred at the tip.

Quasi –Newton’s Method I and II: It showed a characteristic of Q-super linear.

Dennis and Schnabel: This showed a characteristic of a Q-quadratic.


f(x)=x2-3

Bisection Method: Bisection Method does not approach any value

Newton Method: : From the graph it is both Q-super linear and Q- quadratic.

Quasi –Newton’s Method I and II: It showed a characteristic of Q-super linear Q- quadratic.

Dennis and Schnabel: This showed a characteristic of a Q-quadratic and Q- quadratic.


(iv) f’(x)=2x


Newton Method: This function did not collect much points that is why no graph is shown for all except bisection.

Rate of convergence cannot be determined.

Quasi –Newton’s Method I and II: The same explanation for Newton Method.

Dennis and Schnabel: The same explanation for newton method.


f(x)=x2-1


Newton Method: : From the graph it is both Q-super linear and Q- quadratic.

Quasi –Newton’s Method I and II: It showed a characteristic of Q-super linear Q- quadratic.

Dennis and Schnabel: This showed a characteristic of a Q-quadratic and Q- quadratic.


f’(x)=2x







(vii) f(x)=x2-2x+1

Bisection Method: Rate of convergence cannot be determined because it did not converge

Newton Method: This function is Q-quadratic for these method

Quasi –Newton’s Method I and II: This function is Q-quadratic

Dennis and Schnabel: This function is Q-quadratic for these method


f’(x)=2x-2







f(x)=atan(x)


Newton Method: : From the graph it is Q- quadratic.

Quasi –Newton’s Method I and II: It showed a characteristic of Q- quadratic.

Dennis and Schnabel: This showed a characteristic of a Q- superlinear.

2) Code for number 2

function [integral_x_value integral_fun ] = integral(ds_x, fun) %UNTITLED2 Summary of this function goes here % Detailed explanation goes here

[x1,number]=size(ds_x);

non_integ=ds_x(number);

integral_large=ceil(ds_x(non_integer)); integral_small=floor(ds_x(non_integer));

integral_fun_large = feval(fun, integral_large); integral_fun_small = feval(fun, integral_small);


if(abs(integral_fun_large)> abs(integral_fun_small)) integer_x_value = integral_large; integer_fun = integral_fun_large;

else integer_x_value = integral_small; integer_fun = integral_fun_small;

end

Final answer X-values is 1 and F(x)=0.09

3) f(x)= x^3-2x^2-6x

Figure : Graph of the function

Bisection Method: The range of convergence for the bisection method, for root x=-2 is that for any interval (a, b); a

has to fall in range (-∞-2) and b has to fall between (-2 0). If these condition stands, then f(a)*f(b) < 0.

Newton Method: For the Newton method convergence will always occur as long as x_0, the starting point is not

f’(x)=0. Therefore Convergence occurs at any point except x=-1.1196 and 1.7863.

Quasi –Newton’s Method: The same explanation as newton

Dennis and Schnabel: The same explanation as newton.


(ii) f’(x)= 3x^2 -4x-6

Bisection Method: The range of convergence for the bisection method, for root x=-1.1196 is that for any interval

(a, b); a has to fall in range (-∞-1.1196) and b has to fall between (-1.1196 1.7863). If these condition stands, then

f(a)*f(b) < 0.

Newton Method: For the Newton method convergence will always occur as long as x_0, the starting point is not

f’(x)=0. Therefore Convergence occurs at any point except x=0.333. All real numbers except x=0.333

Quasi –Newton’s Method: The same explanation as newton

Dennis and Schnabel: The same explanation as newton.


b) f(x)= x^3-2x^2-6x






(ii) f’(x)= 3x^2 -4x-6






4) When the initial point is far away from the root, the fast heuristic optimization program will be used to get very

close to the root. In other to determine when it gets close to the root, a tolerance has to be applied to my new

program. When it is close to the root then the slow converging optimization program is used to determine the

exact solution. Another tolerance has to be applied to tell the program when to terminate. The critical and

standard assumption is the application of tolerance. We could say when the difference between the root and only

value is 0.1 then apply the slow convergence optimization program. And then when it is 10^-8 close to the rrot

then the program should terminate.

Documents

Solving equations using linear optimization techniques