Upload
gr-ds
View
29
Download
3
Tags:
Embed Size (px)
Citation preview
A NEW HYBRID WYL-AMRI CONJUGATE GRADIENT METHOD WITH SUFFICIENT
DESCENT CONDITION FOR UNCONSTRAINED OPTIMIZATION
Ibrahim S. Mohammed*Mustafa Mamat
Abdelrhaman AbasharKamil Uba Kamfa
Contents :-* Introduction * Objectives of the research* Conjugate Gradient Version* New method and Algorithm* Numerical Results* Conclusion* References
* Introduction
Conjugate gradient method (CG) are designed to solve large scale unconstrained
optimization problem. In general, the method has the form
𝒎𝒊𝒏𝒙 ∈𝑹𝒏 𝒇(𝒙) (1.1)
where 𝒇 ∶ 𝑹𝒏 → 𝑹 is continuously differentiable . Conjugate gradient methods are
iterative methods of the form
𝒙𝒌+𝟏 = 𝒙𝒌 + 𝜶𝒌𝒅𝒌 (1.2)
where 𝒙𝒌+𝟏 is the current iterate point , 𝜶𝒌 > 𝟎 is step length which is computed
by carrying out a line search , 𝒅𝒌 is the search direction of the conjugate gradient
method define by
𝒅𝒌 −𝒈𝒌 , 𝒊𝒇 𝒌 = 𝟎
−𝒈𝒌 + 𝜷𝒌𝒅𝒌, 𝒊𝒇 𝒌 ≥ 𝟏(1.3)
Some classical formula’s for 𝜷𝒌 are given as follows
𝜷𝒌𝑯𝑺 =
𝒈𝒌𝑻 (𝒈𝒌−𝒈𝒌−𝟏)
(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏(1.4)
Cont : Introduction
𝜷𝒌𝑭𝑹 =
𝒈𝒌𝑻 𝒈𝒌
𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏
(1.5)
𝜷𝒌𝑷𝑹𝑷 =
𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)
𝒈𝒌−𝟏𝑻 𝒈𝒌−𝟏
(1.6)
𝜷𝒌𝑪𝑫 = −
𝒈𝒌𝑻 𝒈𝒌
𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏
(1.7)
𝜷𝒌𝑫𝒀 =
𝒈𝒌𝑻 𝒈𝒌
(𝒈𝒌−𝒈𝒌−𝟏)𝑻 𝒅𝒌−𝟏
(1.8)
𝜷𝒌𝑳𝑺 = −
𝒈𝒌𝑻(𝒈𝒌−𝒈𝒌−𝟏)
𝒅𝒌−𝟏𝑻 𝒈𝒌−𝟏
(1.9)
Cont : Introduction
• Zoutendijk proved that FR Method with exact line search is
globally convergent , Al – Baali extended this results to the
strong Wolfe - Powell line search.
• Recently, Wei et al. [7], propose a new CG formula
• Abashar et al ,modified RMIL method to suggest
• Research Objective
* To proposed a new formula for solving unconstrained optimization.
* To analyze the performance these new formulas based on standard optimizations test
problem functions.
* To proof the sufficient descent conditions of our new method
• Conjugate Gradient Version
(i) Hybrid CG methods (ii) Scaled CG methods. (iii) Three terms CG methods.
.
An important class of conjugate gradient algorithm is the hybrid CG method, for
example Hu and Storey [18] propose
Dai and Yuan [19] suggested two hybrid methods,
New method and Algorithm
we propose a new hybrid CG method is a combination between CG methods
(WYL, RMIL)
Algorithm
Step 1: Given6
0 10, nRx , set 00 gd if 0g then stop.
Step 2: Compute k by applying exact line search.
Step 3: kkkk dxx 1 if 1kg then stop.
Step 4: ComputehRW
k and generated kd by (1.3).
Step 5: Set 1 kk go to Step 2.
Numerical Results• Test problem functions considered by Andrei.
• Stopping criteria as Hillstrom , 𝑔𝑘 ≤ 𝜀.
• Subroutine programming using Matlab.
• Using exact line search.
• Performance profile introduced by Dolan and More.
Cont: Numerical Results
TABLE 1: A LIST OF PROBLEM FUNCTIONS
No Function Dim Initial Points
1 Six hump 2 (1, 1), (2, 2), (5,5), (10, -10)
2 Three hump 2 (24, 24), (29, 29), (33, 33), (50, 50)
3 Booth 2 (10, -10), (20, 20), (50, 50), (100, 100)
4 Treccani 2 (5, 5), (10,10), (-20, 20), (-50, -50)
5 Matyas 2 (1, 1), (5, 5), (10, 10), (50, 50)
6 Extended Maratos 2, 4 (0,0,0,0), (0.5,5, 0.5, 5), (10, 0.5, 10, 0.5), (70, 70, 70, 70)
7 Ext FREUD & ROTH 2, 4 (13, 13, 13, 13), (21, 21, 21, 21), (25, 25, 25,25), (23, 23, 23, 23)
8 Generalized Trig 2, 4, 10 (0.5, 5, …, 5),(5, 10, …, 10),(7,7, …, 7),(50, 50, …, 50)
9 Fletcher 2, 4, 10 (23, 23, …, 23), (45, 45, …, 45), (50, 5,…, 5), (70, 70, …,70)
10 Extended Penalty 2, 4, 10, 100 (0.5,5, …,5),(10,-0.5…,-0.5), (105,105, …,105), (130,130, …,130)
11 Raydan 1 2, 4, 10, 100 (1, 1, …,1), (3, 3, …, 3), (5, 5, …, 5), (-10, -10, …, -10)
12 Hager 2, 4, 10, 100 (3, -3, …, -3),(21, 21, …, 21), (-23, 23, …, 23), (23, 23, …, 23)
13 Rosenbrock 2, 4, 10, 100, 500, 1000, 10000 (7, 7, …, 7), (13, 13, …, 13), (23, 23, …, 23), (35, 35, …, 35)
14 Shallow 2, 4, 10, 100, 500, 1000, 10000 (21, -21, …, -21), (21, 21, …, 21), (50,50, …, 50),(130, 130, …, 130)
15 Tridiagonal 1 2, 4, 10, 100, 500, 1000, 10000 (0, 0,…, 0), (1, -1, …, -1), (17, -17, …, -17), (30, 30, …, 30)
16 Ext White & Holst 2, 4, 10, 100, 500, 1000, 10000 (-5, -5, …, -5), (2, -2, …, -2), (3, -3, …, -3), (7, -7, …, -7)
17 Ext Denschnb 2, 4, 10, 100, 500, 1000, 10000 (8, 8, …, 8), (11, 11, …,11), (12, 12, …, 12),(13, 13, …, 13)
18 Diagonal 4 2, 4, 10, 100, 500, 1000, 10000 (2, 2, …, 2), (5, 5, …,5), (10, 10, …, 10), (15, 15, …, 15)
Cont: Numerical Results
• Performance Profile based on Number iterations
Cont: Numerical Results
• Performance Profile based on CPU time
• Conclusion
* AMRI was able to solve 95% of test problems.
* WYL solve 97% of test problems.
* SW-A solve all test problems.
• References* M.R. Hestenes, E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. Sec. B 49 1952, pp. 409–432
*Z. Wei, S. Yao, L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 2006, pp. 1341–1350.
*G. Zoutendijk, Nonlinear programming computational methods, in:J.Abadie(Ed.), Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970, pp. 37–86.
*M.Rivaie,M.Mamat,L. June, M.Ismail, a new class of nonlinear conjugate gradient coefficient with global convergence properties, Appl.Math.comput.218 ,2012, pp.11323-11332.
. * Y.H. Dai, Y. Yuan. An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103, 2001, pp. 33–47.
* E. Dolan, J.J.More, Benchmarking optimization software with performance profile, Math. Prog. 91, 2002, pp. 201–213.
*Y. F. Hu, C. Storey . Global convergence result for conjugate gradient methods. J.Optim.Theory.Appl., 71,1991, pp. 399-405.
* N. Andrei, An unconstrained optimization test functions collection, Adv. Modell. Optim. 10, 2008, pp. 147–161.
•
Thank You
We encourage our participants to adopt a crisp 10 Minutes Presentation Format.
1 Minute: Self Introduction and Topic
2 Minutes: Problem Area/ Need of Research
2 Minutes: Inputs/ Efforts/ Methodology
2 Minutes: Outputs
1 Minute: Take Aways
2 Minutes: Discussion
This format is tried and tested, and makes the sessions verymeaningful, rather than monotonous and full of slumber.Knowledgeable participants, but with poor soft-skills also find iteasy to present. This format provides more time for networkingand discussions and idea sharing, which are the core purpose ofour conferences.