17
A NEW HYBRID WYL-AMRI CONJUGATE GRADIENT METHOD WITH SUFFICIENT DESCENT CONDITION FOR UNCONSTRAINED OPTIMIZATION Ibrahim S. Mohammed* Mustafa Mamat Abdelrhaman Abashar Kamil Uba Kamfa

Grds international conference on pure and applied science (6)

  • Upload
    gr-ds

  • View
    29

  • Download
    3

Embed Size (px)

Citation preview

Page 1: Grds international conference on pure and applied science (6)

A NEW HYBRID WYL-AMRI CONJUGATE GRADIENT METHOD WITH SUFFICIENT

DESCENT CONDITION FOR UNCONSTRAINED OPTIMIZATION

Ibrahim S. Mohammed*Mustafa Mamat

Abdelrhaman AbasharKamil Uba Kamfa

Page 2: Grds international conference on pure and applied science (6)

Contents :-* Introduction * Objectives of the research* Conjugate Gradient Version* New method and Algorithm* Numerical Results* Conclusion* References

Page 3: Grds international conference on pure and applied science (6)

* Introduction

Conjugate gradient method (CG) are designed to solve large scale unconstrained

optimization problem. In general, the method has the form

𝒎𝒊𝒏𝒙 ∈𝑹𝒏 𝒇(𝒙) (1.1)

where 𝒇 ∶ 𝑹𝒏 → 𝑹 is continuously differentiable . Conjugate gradient methods are

iterative methods of the form

𝒙𝒌+𝟏 = 𝒙𝒌 + 𝜶𝒌𝒅𝒌 (1.2)

where 𝒙𝒌+𝟏 is the current iterate point , 𝜶𝒌 > 𝟎 is step length which is computed

by carrying out a line search , 𝒅𝒌 is the search direction of the conjugate gradient

method define by

𝒅𝒌 −𝒈𝒌 , 𝒊𝒇 𝒌 = 𝟎

−𝒈𝒌 + 𝜷𝒌𝒅𝒌, 𝒊𝒇 𝒌 ≥ 𝟏(1.3)

Some classical formula’s for 𝜷𝒌 are given as follows

𝜷𝒌𝑯𝑺 =

𝒈𝒌𝑻 (𝒈𝒌−𝒈𝒌−𝟏)

(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏(1.4)

Page 4: Grds international conference on pure and applied science (6)

Cont : Introduction

𝜷𝒌𝑭𝑹 =

𝒈𝒌𝑻 𝒈𝒌

𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.5)

𝜷𝒌𝑷𝑹𝑷 =

𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)

𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.6)

𝜷𝒌𝑪𝑫 = −

𝒈𝒌𝑻 𝒈𝒌

𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.7)

𝜷𝒌𝑫𝒀 =

𝒈𝒌𝑻 𝒈𝒌

(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏

(1.8)

𝜷𝒌𝑳𝑺 = −

𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)

𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏

(1.9)

Page 5: Grds international conference on pure and applied science (6)

Cont : Introduction

• Zoutendijk proved that FR Method with exact line search is

globally convergent , Al – Baali extended this results to the

strong Wolfe - Powell line search.

• Recently, Wei et al. [7], propose a new CG formula

• Abashar et al ,modified RMIL method to suggest

Page 6: Grds international conference on pure and applied science (6)

• Research Objective

* To proposed a new formula for solving unconstrained optimization.

* To analyze the performance these new formulas based on standard optimizations test

problem functions.

* To proof the sufficient descent conditions of our new method

Page 7: Grds international conference on pure and applied science (6)

• Conjugate Gradient Version

(i) Hybrid CG methods (ii) Scaled CG methods. (iii) Three terms CG methods.

.

An important class of conjugate gradient algorithm is the hybrid CG method, for

example Hu and Storey [18] propose

Dai and Yuan [19] suggested two hybrid methods,

Page 8: Grds international conference on pure and applied science (6)

New method and Algorithm

we propose a new hybrid CG method is a combination between CG methods

(WYL, RMIL)

Algorithm

Step 1: Given6

0 10, nRx , set 00 gd if 0g then stop.

Step 2: Compute k by applying exact line search.

Step 3: kkkk dxx 1 if 1kg then stop.

Step 4: ComputehRW

k and generated kd by (1.3).

Step 5: Set 1 kk go to Step 2.

Page 9: Grds international conference on pure and applied science (6)

Numerical Results• Test problem functions considered by Andrei.

• Stopping criteria as Hillstrom , 𝑔𝑘 ≤ 𝜀.

• Subroutine programming using Matlab.

• Using exact line search.

• Performance profile introduced by Dolan and More.

Page 10: Grds international conference on pure and applied science (6)

Cont: Numerical Results

TABLE 1: A LIST OF PROBLEM FUNCTIONS

No Function Dim Initial Points

1 Six hump 2 (1, 1), (2, 2), (5,5), (10, -10)

2 Three hump 2 (24, 24), (29, 29), (33, 33), (50, 50)

3 Booth 2 (10, -10), (20, 20), (50, 50), (100, 100)

4 Treccani 2 (5, 5), (10,10), (-20, 20), (-50, -50)

5 Matyas 2 (1, 1), (5, 5), (10, 10), (50, 50)

6 Extended Maratos 2, 4 (0,0,0,0), (0.5,5, 0.5, 5), (10, 0.5, 10, 0.5), (70, 70, 70, 70)

7 Ext FREUD & ROTH 2, 4 (13, 13, 13, 13), (21, 21, 21, 21), (25, 25, 25,25), (23, 23, 23, 23)

8 Generalized Trig 2, 4, 10 (0.5, 5, …, 5),(5, 10, …, 10),(7,7, …, 7),(50, 50, …, 50)

9 Fletcher 2, 4, 10 (23, 23, …, 23), (45, 45, …, 45), (50, 5,…, 5), (70, 70, …,70)

10 Extended Penalty 2, 4, 10, 100 (0.5,5, …,5),(10,-0.5…,-0.5), (105,105, …,105), (130,130, …,130)

11 Raydan 1 2, 4, 10, 100 (1, 1, …,1), (3, 3, …, 3), (5, 5, …, 5), (-10, -10, …, -10)

12 Hager 2, 4, 10, 100 (3, -3, …, -3),(21, 21, …, 21), (-23, 23, …, 23), (23, 23, …, 23)

13 Rosenbrock 2, 4, 10, 100, 500, 1000, 10000 (7, 7, …, 7), (13, 13, …, 13), (23, 23, …, 23), (35, 35, …, 35)

14 Shallow 2, 4, 10, 100, 500, 1000, 10000 (21, -21, …, -21), (21, 21, …, 21), (50,50, …, 50),(130, 130, …, 130)

15 Tridiagonal 1 2, 4, 10, 100, 500, 1000, 10000 (0, 0,…, 0), (1, -1, …, -1), (17, -17, …, -17), (30, 30, …, 30)

16 Ext White & Holst 2, 4, 10, 100, 500, 1000, 10000 (-5, -5, …, -5), (2, -2, …, -2), (3, -3, …, -3), (7, -7, …, -7)

17 Ext Denschnb 2, 4, 10, 100, 500, 1000, 10000 (8, 8, …, 8), (11, 11, …,11), (12, 12, …, 12),(13, 13, …, 13)

18 Diagonal 4 2, 4, 10, 100, 500, 1000, 10000 (2, 2, …, 2), (5, 5, …,5), (10, 10, …, 10), (15, 15, …, 15)

Page 11: Grds international conference on pure and applied science (6)
Page 12: Grds international conference on pure and applied science (6)

Cont: Numerical Results

• Performance Profile based on Number iterations

Page 13: Grds international conference on pure and applied science (6)

Cont: Numerical Results

• Performance Profile based on CPU time

Page 14: Grds international conference on pure and applied science (6)

• Conclusion

* AMRI was able to solve 95% of test problems.

* WYL solve 97% of test problems.

* SW-A solve all test problems.

Page 15: Grds international conference on pure and applied science (6)

• References* M.R. Hestenes, E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. Sec. B 49 1952, pp. 409–432

*Z. Wei, S. Yao, L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 2006, pp. 1341–1350.

*G. Zoutendijk, Nonlinear programming computational methods, in:J.Abadie(Ed.), Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970, pp. 37–86.

*M.Rivaie,M.Mamat,L. June, M.Ismail, a new class of nonlinear conjugate gradient coefficient with global convergence properties, Appl.Math.comput.218 ,2012, pp.11323-11332.

. * Y.H. Dai, Y. Yuan. An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 2001, pp. 33–47.

* E. Dolan, J.J.More, Benchmarking optimization software with performance profile, Math. Prog. 91, 2002, pp. 201–213.

*Y. F. Hu, C. Storey . Global convergence result for conjugate gradient methods. J.Optim.Theory.Appl., 71,1991, pp. 399-405.

* N. Andrei, An unconstrained optimization test functions collection, Adv. Modell. Optim. 10, 2008, pp. 147–161.

Page 16: Grds international conference on pure and applied science (6)

Thank You

Page 17: Grds international conference on pure and applied science (6)

We encourage our participants to adopt a crisp 10 Minutes Presentation Format.

1 Minute: Self Introduction and Topic

2 Minutes: Problem Area/ Need of Research

2 Minutes: Inputs/ Efforts/ Methodology

2 Minutes: Outputs

1 Minute: Take Aways

2 Minutes: Discussion

This format is tried and tested, and makes the sessions verymeaningful, rather than monotonous and full of slumber.Knowledgeable participants, but with poor soft-skills also find iteasy to present. This format provides more time for networkingand discussions and idea sharing, which are the core purpose ofour conferences.