Home Another hybrid conjugate gradient method as a convex combination of WYL and CD methods
Article
Licensed
Unlicensed Requires Authentication

Another hybrid conjugate gradient method as a convex combination of WYL and CD methods

  • Imane Guefassa EMAIL logo , Yacine Chaib and Tahar Bechouat
Published/Copyright: June 21, 2024

Abstract

Conjugate gradient (CG) methods are a popular class of iterative methods for solving linear systems of equations and nonlinear optimization problems. In this paper, a new hybrid conjugate gradient (CG) method is presented and analyzed for solving unconstrained optimization problems, where the parameter β k is a convex combination of β k WYL and β k CD . Under the strong Wolfe line search, the new method possesses the sufficient descent condition and the global convergence properties. The preliminary numerical results show the efficiency of our method in comparison with other CG methods. Furthermore, the proposed algorithm HWYLCD was extended to solve the problem of a mode function.

MSC 2020: 90C30; 65K05; 62G05

References

[1] N. Andrei, An unconstrained optimization test functions collection, Adv. Model. Optim. 10 (2008), no. 1, 147–161. Search in Google Scholar

[2] N. Andrei, Another hybrid conjugate gradient algorithm for unconstrained optimization, Numer. Algorithms 47 (2008), no. 2, 143–156. 10.1007/s11075-007-9152-9Search in Google Scholar

[3] N. Andrei, Hybrid conjugate gradient algorithm for unconstrained optimization, J. Optim. Theory Appl. 141 (2009), no. 2, 249–264. 10.1007/s10957-008-9505-0Search in Google Scholar

[4] N. Andrei, New hybrid conjugate gradient algorithms for unconstrained optimization, Encyclopedia Optimization, Springer, Boston (2009), 2560–2571. 10.1007/978-0-387-74759-0_441Search in Google Scholar

[5] I. Bongartz, A. R. Conn, N. Gould and P. L. Toint, Constrained and unconstrained testing environment, ACM Trans. Math. Software (TOMS) 21 (1995), 123–160. 10.1145/200979.201043Search in Google Scholar

[6] Y. H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim. 10 (1999), no. 1, 177–182. 10.1137/S1052623497318992Search in Google Scholar

[7] Y. H. Dai and Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res. 103 (2001), 33–47. 10.1023/A:1012930416777Search in Google Scholar

[8] Y. H. Dai, Y. H. Yuan, G. H. Liu, D. F. Sun, X. Yin and Y. Yuan, Convergence properties of nonlinear conjugate gradient methods, SIAM J. Optim. 10 (1999), 348–358. 10.1137/S1052623494268443Search in Google Scholar

[9] S. S. Djordjević, New hybrid conjugate gradient method as a convex combination of FR and PRP methods, Filomat 30 (2016), no. 11, 3083–3100. 10.2298/FIL1611083DSearch in Google Scholar

[10] S. S. Djordjević, New hybrid conjugate gradient method as a convex combination of LS and CD methods, Filomat 31 (2017), no. 6, 1813–1825. 10.2298/FIL1706813DSearch in Google Scholar

[11] E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), no. 2, 201–213. 10.1007/s101070100263Search in Google Scholar

[12] W. F. Eddy, Optimum kernel estimators of the mode, Ann. Statist. 8 (1980), no. 4, 870–882. 10.1214/aos/1176345080Search in Google Scholar

[13] R. Fletcher, Practical Methods of Optimization, 2nd ed., John Wiley & Sons, Chichester, 1987. Search in Google Scholar

[14] R. Fletcher and C. M. Reeves, Function minimization by conjugate gradients, Comput. J. 7 (1964), 149–154. 10.1093/comjnl/7.2.149Search in Google Scholar

[15] J. C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim. 2 (1992), no. 1, 21–42. 10.1137/0802003Search in Google Scholar

[16] I. Guefassa, Y. Chaib and T. Bechouat, A hybrid conjugate gradient method between MLS and FR in nonparametric statistics, Commun. Comb. Optim. (2023), 10.22049/cco.2023.28935.1784. 10.22049/cco.2023.28935.1784Search in Google Scholar

[17] W. W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods, Pac. J. Optim. 2 (2006), no. 1, 35–58. Search in Google Scholar

[18] M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards 49 (1952), 409–436. 10.6028/jres.049.044Search in Google Scholar

[19] H. Huang, Z. Wei and S. Yao, The proof of the sufficient descent condition of the Wei–Yao–Liu conjugate gradient method under the strong Wolfe–Powell line search, Appl. Math. Comput. 189 (2007), no. 2, 1241–1245. 10.1016/j.amc.2006.12.006Search in Google Scholar

[20] V. D. Konakov, The asymptotic normality of the mode of multivariate distributions, Theory Probab. Appl. 19 (1974), 794–799. 10.1137/1118104Search in Google Scholar

[21] J. K. Liu and S. J. Li, New hybrid conjugate gradient method for unconstrained optimization, Appl. Math. Comput. 245 (2014), 36–43. 10.1016/j.amc.2014.07.096Search in Google Scholar

[22] Y. Liu and C. Storey, Efficient generalized conjugate gradient algorithms. I. Theory, J. Optim. Theory Appl. 69 (1991), no. 1, 129–137. 10.1007/BF00940464Search in Google Scholar

[23] E. Parzen, On estimation of a probability density function and mode, Ann. Math. Statist. 33 (1962), 1065–1076. 10.1214/aoms/1177704472Search in Google Scholar

[24] E. Polak and G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Rev. Française Informat. Recherche Opérationnelle 3 (1969), no. 16, 35–43. 10.1051/m2an/196903R100351Search in Google Scholar

[25] B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comp. Math. Phys. 9 (1969), 94–112. 10.1016/0041-5553(69)90035-4Search in Google Scholar

[26] T. W. Sager, An iterative method for estimating a multivariate mode and isopleth, J. Amer. Statist. Assoc. 74 (1979), no. 366, 329–339. 10.1080/01621459.1979.10482514Search in Google Scholar

[27] M. Samanta, Nonparametric estimation of the mode of a multivariate density, South African Statist. J. 7 (1973), 109–117. Search in Google Scholar

[28] D. Touati-Ahmed and C. Storey, Efficient hybrid conjugate gradient techniques, J. Optim. Theory Appl. 64 (1990), no. 2, 379–397. 10.1007/BF00939455Search in Google Scholar

[29] Z. Wei, S. Yao and L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 (2006), no. 2, 1341–1350. 10.1016/j.amc.2006.05.150Search in Google Scholar

[30] A. Zhou, Z. Zhu, H. Fan and Q. Qing, Three new hybrid conjugate gradient methods for optimization, Appl. Math. (Irvine) 2 (2011), no. 3, 303–308. 10.4236/am.2011.23035Search in Google Scholar

Received: 2023-03-09
Revised: 2024-06-04
Accepted: 2024-06-06
Published Online: 2024-06-21
Published in Print: 2024-09-01

© 2024 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 14.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/mcma-2024-2007/html
Scroll to top button