Incremental proximal gradient algorithm using penalization strategies and fixed point techniques for solving some convex bilevel optimization problems
Authors
A. Hanjing
- Department of Science and Mathematics, Rajamangala University of Technology, Isan Surin Campus, Surin, 32000, Thailand.
P. Thongpaen
- Department of Mathematics, Faculty of Science, Chiang Mai University, Chaing Mai, 50200, Thailand.
K. Kankam
- Elementary Education Program, Faculty of Education, Suan Dusit University Lampang Center, Lampang, 52100, Thailand.
S. Suantai
- Department of Mathematics, Faculty of Science, Chiang Mai University, Chaing Mai, 50200, Thailand.
Abstract
The objective of this paper is to propose a novel incremental proximal algorithm that incorporates gradient penalization strategies within a fixed-point framework for solving convex bilevel optimization problems. The outer-level objective is modeled as the sum of two composite convex functions, one of which is nonsmooth. We establish a convergence theorem for the proposed algorithm under suitable assumptions. Additionally, we provide numerical experiments to demonstrate the effectiveness of the method in solving image inpainting problems. Comparative results with existing algorithms in the literature indicate that the proposed approach exhibits superior convergence performance.
Share and Cite
ISRP Style
A. Hanjing, P. Thongpaen, K. Kankam, S. Suantai, Incremental proximal gradient algorithm using penalization strategies and fixed point techniques for solving some convex bilevel optimization problems, Journal of Mathematics and Computer Science, 41 (2026), no. 1, 34--57
AMA Style
Hanjing A., Thongpaen P., Kankam K., Suantai S., Incremental proximal gradient algorithm using penalization strategies and fixed point techniques for solving some convex bilevel optimization problems. J Math Comput SCI-JM. (2026); 41(1):34--57
Chicago/Turabian Style
Hanjing, A., Thongpaen, P., Kankam, K., Suantai, S.. "Incremental proximal gradient algorithm using penalization strategies and fixed point techniques for solving some convex bilevel optimization problems." Journal of Mathematics and Computer Science, 41, no. 1 (2026): 34--57
Keywords
- Incremental proximal and gradient method
- convex bilevel optimization problem
- fixed point technique
- image inpainting
MSC
References
-
[1]
J. S. Arora, Introduction to optimum design, Academic Press, Amsterdam (2004)
-
[2]
H. Attouch, M.-O. Czarnecki, Asymptotic behavior of coupled dynamical systems with multiscale aspects, J. Differ. Equ., 248 (2010), 1315–1344
-
[3]
H. Attouch, M.-O. Czarnecki, J. Peypouquet, Coupling forward-backward with penalty schemes and parallel splitting for constrained variational inequalities, SIAM J. Optim., 21 (2011), 1251–1274
-
[4]
H. Attouch, M.-O. Czarnecki, J. Peypouquet, Prox-penalization and splitting methods for constrained variational problems, SIAM J. Optim., 21 (2011), 149–173
-
[5]
S. Banert, R. I. Bo¸t, Backward penalty schemes for monotone inclusion problems, J. Optim. Theory Appl., 166 (2015), 930–948
-
[6]
H. H. Bauschke, P. L. Combettes, Convex analysis and monotone operator theory in Hilbert spaces, Springer, New York (2011)
-
[7]
M. S. Bazaraa, H. D. Sherali, C. M. Shetty, Nonlinear programming: theory and algorithms, John Wiley & Sons, Hoboken, New Jersey (2006)
-
[8]
A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), 183–202
-
[9]
D. P. Bertsekas, Incremental proximal methods for large scale convex optimization, Math. Program., 129 (2011), 163–195
-
[10]
D. Bertsekas, Convex optimization algorithms, Athena Scientific, Belmont, MA (2015)
-
[11]
R. I. Bo¸t, E. R. Csetnek, N. Nimana, Gradient-type penalty method with inertial effects for solving constrained convex optimization problems with smooth data, Optim. Lett., 12 (2018), 17–33
-
[12]
R. I. Bo¸t, E. R. Csetnek, N. Nimana, An inertial proximal-gradient penalization scheme for constrained convex optimization problems, Vietnam J. Math., 46 (2018), 53–71
-
[13]
R. I. Bo¸t, D.-K. Nguyen, A forward-backward penalty scheme with inertial effects for monotone inclusions. Applications to convex bilevel programming, Optimization, 68 (2019), 1855–1880
-
[14]
L. Bottou, Y. L. Cun, Large scale online learning, In:Advances in Neural Information Processing Systems, MIT Press, 16 (2003), 217–224
-
[15]
L. Bottou, F. E. Curtis, J. Nocedal, Optimization methods for large scale machine learning, SIAM Rev., 60 (2018), 223–311
-
[16]
S. Boyd, L. Vandenberghe, Convex optimization, Cambridge University Press, Cambridge (2004)
-
[17]
L. Bussaban, S. Suantai, A. Kaewkhao, A parallel inertial S-iteration forward-backward algorithm for regression and classification problems, Carpathian J. Math., 36 (2020), 35–44
-
[18]
J.-F. Cai, E. J. Candès, Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20 (2010), 1956–1982
-
[19]
P. L. Combettes, Quasi-Fejérian analysis of some optimization algorithms, Stud. Comput. Math., 8 (2001), 115–152
-
[20]
F. Cui, Y. Tang, Y. Yang, An inertial three-operator splitting algorithm with applications to image inpainting, arXiv preprint arXiv:1904.11684, (2019), 26 pages
-
[21]
D. Davis, W. Yin, A three-operator splitting scheme and its optimization applications, Set-Valued Var. Anal., 25 (2017), 829–858
-
[22]
I. Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT Press, Cambridge, MA (2016)
-
[23]
A. Hanjing, N. Pholasa, S. Suantai, An algorithm for solving common points of convex minimization problems with applications, Symmetry, 15 (2023), 14 pages
-
[24]
A. Hanjing, S. Suantai, Novel algorithms with inertial techniques for solving constrained convex minimization problems and applications to image inpainting, Mathematics, 11 (2023), 18 pages
-
[25]
P. Jailoka, S. Suantai, A. Hanjing, A fast viscosity forward-backward algorithm for convex minimization problems with an application in image recovery, Carpathian J. Math., 37 (2021), 449–461
-
[26]
K. C. Kiwiel, Convergence of approximate and incremental subgradient methods for convex optimization, SIAM J. Optim., 14 (2003), 807–840
-
[27]
T. Li, D. Acosta-Soba, A. Columbu, G. Viglialoro, Dissipative gradient nonlinearities prevent δ-formations in local and nonlocal attraction-repulsion chemotaxis models, Stud. Appl. Math., 154 (2025), 19 pages
-
[28]
W. R. Mann, Mean value methods in iteration, Proc. Amer. Math. Soc., 4 (1953), 506–510
-
[29]
N. Nimana, N. Petrot, Generalized forward-backward splitting with penalization for monotone inclusion problems, J. Global Optim., 73 (2019), 825–847
-
[30]
N. Nimana, N. Petrot, Splitting proximal with penalization schemes for additive convex hierarchical minimization problems, Optim. Methods Softw., 35 (2020), 1098–1118
-
[31]
J. Nocedal, S. J. Wright, Numerical optimization, Springer, New York (2006)
-
[32]
N. Noun, J. Peypouquet, Forward-backward penalty scheme for constrained convex minimization without inf-compactness, J. Optim. Theory Appl., 158 (2013), 787–795
-
[33]
N. Petrot, N. Nimana, Incremental proximal gradient scheme with penalization for constrained composite convex optimization problems, Optimization, 70 (2021), 1307–1336
-
[34]
J. Peypouquet, Coupling the gradient method with a general exterior penalization scheme for convex minimization, J. Optim. Theory Appl., 153 (2012), 123–138
-
[35]
B. T. Polyak, Introduction to optimization (Translated from the Russian), Optimization Software, Inc., Publications Division, New York (1987)
-
[36]
M. Schmidt, N. Le Roux, F. Bach, Minimizing finite sums with the stochastic average gradient, Math. Program., 162 (2017), 83–112
-
[37]
P. Thongpaen, W. Inthakon, A. Kaewkhao, S. Suantai, Convex minimization problems based on an accelerated fixed point algorithm with applications to image restoration problems, J. Nonlinear Var. Anal., 7 (2023), 87–101
-
[38]
P. Thongpaen, R. Wattanataweekul, A fast fixed-point algorithm for convex minimization problems and its application in image restoration problems, Mathematics, 9 (2021), 13 pages
-
[39]
Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004), 600–612
