全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

正则稀疏优化模型及算法研究综述
A Survey on Regularized Sparse Optimization Models and Algorithms

DOI: 10.12677/AIRR.2023.123018, PP. 155-166

Keywords: 正则稀疏优化模型,算法,正则项,工业工程
Regularized Sparse Optimization Model
, Algorithm, Regularization Term, Industrial Engineering

Full-Text   Cite this paper   Add to My Lib

Abstract:

稀疏优化在工业工程等实际问题中具有十分重要的作用。在过去的几十年里,很多实际问题可以归纳成正则稀疏优化模型并求解出欠定系统的稀疏解,因此其改进和算法设计得到广泛的研究。正则稀疏优化模型不仅可以将原问题降维,而且可以将不适定的问题转换为适定问题。关键问题是如何构造正则项,使得模型具有好的稀疏解的同时还有很好的泛化能力。本文,我们重点关注近30年来正则稀疏优化模型及算法的研究进展,总结归纳了最具代表性的几种正则稀疏优化模型及算法。最后,结合最新研究成果,针对损伤识别、故障诊断、超分辨率重建和电阻抗层析成像等实际问题,构造不同的新正则稀疏优化模型,并探讨其可以研究发展的方向以及更广阔的应用前景。
Sparse optimization plays a very important role in practical problems such as industrial engineering. In the past decades, many practical problems can be generalized into regularized sparse optimization models to solve the sparse solutions of underdetermined systems. Therefore, the improvement of such models and the design of algorithms have been widely studied. The regularized sparse optimization model can reduce the dimension of the original problem and transform the ill-posed problem into a well-posed problem. The key point is how to choose and construct the regularization terms so that the model can obtain a good sparse solution with good generalization ability. In this paper, we focus on the research progress of regularized sparse optimization models and algorithms in the past 30 years, and summarize the most representative models and algorithms. In addition, according to the latest research results, different new regularized sparse optimization models are constructed to solve practical problems such as damage identification, fault diagnosis, super-resolution reconstruction and electrical impedance tomography, and the development direction and broader application prospect of these models are discussed.

References

[1]  Bruckstein, A.M., Donoho, D.L. and Elad, M. (2009) From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images. SIAM Review, 51, 34-81.
https://doi.org/10.1137/060657704
[2]  Duarte, M.F., Davenport, M.A., Takhar, D., et al. (2008) Single-Pixel Imaging via Compressive Sampling. IEEE Signal Processing Magazine, 25, 83-91.
https://doi.org/10.1109/MSP.2007.914730
[3]  Wang, Y., Zhou, G.L., Zhang, X., Liu, W. and Caccetta, L. (2016) The Non-Convex Sparse Problem with Nonnegative Constraint for Signal Reconstruction. Journal of Optimization Theory and Applications, 170, 1009-1025.
https://doi.org/10.1007/s10957-016-0869-2
[4]  Xiu, X.C., Pan, L.L., Yang, Y. and Liu, W. (2022) Efficient and Fast Joint Sparse Constrained Canonical Correlation Analysis for Fault Detection. IEEE Transactions on Neural Networks and Learning Systems, 1-11.
https://doi.org/10.1109/TNNLS.2022.3201881
[5]  Donoho, D.L. (2006) Compressed Sensing. IEEE Transactions on Information Theory, 52, 1289-1306.
https://doi.org/10.1109/TIT.2006.871582
[6]  Luo, Z.Y., Qin, L.X., Kong, L.C. and Xiu, N.H. (2014) The Nonnegative Zero-Norm Minimization under Generalized Z-Matrix Measurement. Journal of Optimization Theory and Applications, 160, 854-864.
https://doi.org/10.1007/s10957-013-0325-5
[7]  Merhej, D., Diab, C., Khalil, M. and Prost, R. (2011) Embedding Prior Knowledge within Compressed Sensing by Neural Networks. IEEE Transactions on Neural Networks, 22, 1638-1649.
https://doi.org/10.1109/TNN.2011.2164810
[8]  Hastie, T., Tibshirani, R. and Wainwright, M. (2015) Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman & Hall/CRC, New York.
https://doi.org/10.1201/b18401
[9]  Torfifi, A., Shirvani, R.A., Soleymani, S. and Nasrabadi, N. M. (2018) Attention-Based Guided Structured Sparsity of Deep Neural Networks. ArXiv Preprint ArXiv: 1802.09902.
[10]  Zhang, D., Katz-Samuels, J., Figueiredo, M.A.T. and Balzano, L. (2018) Simultaneous Sparsity and Parameter Tying for Deep Learning using Ordered Weighted l1 Regularization. 2018 IEEE Statistical Signal Processing Workshop (SSP), Freiburg im Breisgau, 10-13 June 2018, 65-69.
https://doi.org/10.1109/SSP.2018.8450819
[11]  Mallat, S. and Zhang, Z. (1993) Matching Pursuit with Time-Frequency Dictionaries. IEEE Transactions on Signal Processing, 41, 3397-3415.
https://doi.org/10.1109/78.258082
[12]  Blumensath, T. and Davies, M.E. (2008) Iterative Thresholding for Sparse Approximations. Journal of Fourier Analysis and Applications, 14, 629-654.
https://doi.org/10.1007/s00041-008-9035-z
[13]  Herrity, K.K., Gilbert, A.C. and Tropp, J.A. (2006) Sparse Approximation via Iterative Thresholding. 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, 3, Toulouse, France.
[14]  Natraajan, B.K. (1995) Sparse Approximate Solutions to Linear Systems. SIAM Journal on Computing, 24, 227-234.
https://doi.org/10.1137/S0097539792240406
[15]  Chen, X.J., Ge, D.D., Wang, Z.Z. and Ye, Y.Y. (2014) Complexity of Unconstrained L2 -Lp Minimization. Mathematical Programming, 143, 371-383.
https://doi.org/10.1007/s10107-012-0613-0
[16]  Frank, I.E. and Freidman, J.H. (1993) A Statistical View of Some Chemometrics Regression Tools. Technometrics, 35, 109-135.
https://doi.org/10.1080/00401706.1993.10485033
[17]  Tropp, J.A. (2004) Greed Is Good: Algorithmic Results for Sparse Approximation. IEEE Transactions on Information Theory, 50, 2231-2242.
https://doi.org/10.1109/TIT.2004.834793
[18]  Bredies, K. and Lorenz, D.A. (2009) Iterated Hard Shrinkage for Minimization Problems with Sparsity Constraints. SIAM Journal on Scientific Computing, 30, 657-683.
https://doi.org/10.1137/060663556
[19]  Chen, S.S., Donoho, D.L. and Saunders, M.A. (1998) Atomic Decomposition by Basis Pursuit. SIAM Journal on Scientific Computing, 20, 33-61.
https://doi.org/10.1137/S1064827596304010
[20]  Candès, E.J. (2008) The Restricted Isometry Property and Its Implications for Compressed Sensing. Comptes Rendus Mathematique, 346, 589-592.
https://doi.org/10.1016/j.crma.2008.03.014
[21]  Donoho, D.L. (1995) De-Noising by Soft-Thresholding. IEEE Transactions on Information Theory, 41, 613-627.
https://doi.org/10.1109/18.382009
[22]  Chambolle, A., DeVore, R.A., Lee, N.-Y. and Lucier, B.J. (1998) Nonlinear Wavelet Image Processing: Variational Problems, Compression, and Noise Removal Through Wavelet Shrinkage. IEEE Transactions on Image Processing, 7, 319-335.
https://doi.org/10.1109/83.661182
[23]  Figueiredo, M.A.T. and Nowak, R.D. (2003) An EM Algorithm for Wavelet-Based Image Restoration. IEEE Transactions on Image Processing, 12, 906-916.
https://doi.org/10.1109/TIP.2003.814255
[24]  Daubechies, I., Defrise, M. and Mol, C.D. (2004) An Iterative Thresholding Algorithm for Linear Inverse Problems with a Sparsity Constraint. Communications on Pure and Applied Mathematics, 57, 1413-1457.
https://doi.org/10.1002/cpa.20042
[25]  Vonesch, C. and Unser, M. (2007) Fast Iterative Thresholding Algorithm for Wavelet-Regularized Deconvolution. In: Van De Ville, D., Goyal, V.K. and Papadakis, M., Eds., Proceedings of the SPIE Optics and Photonics 2007 Conference on Mathematical Methods: Wavelet XII, Vol. 6701, SPIE, Bellingham, 135-139.
https://doi.org/10.1117/12.733532
[26]  Kim, S.-J., Koh, K., Lustig, M., Boyd, S. and Gorinevsky, D. (2007) An Interior-Point Method for Large-Scale l1-Regularized Least Squares. IEEE Journal of Selected Topics in Signal Processing, 1, 606-617.
https://doi.org/10.1109/JSTSP.2007.910971
[27]  Osborne, M.R., Presnell, B. and Turlach, B.A. (2000) A New Approach to Variable Selection in Least Squares Problems. IMA Journal of Numerical Analysis, 20, 389-403.
https://doi.org/10.1093/imanum/20.3.389
[28]  Osborne, M.R., Presnell, B. and Turlach, B.A. (2000) On the Lasso and Its Dual. Journal of Computational and Graphical Statistics, 9, 319-337.
https://doi.org/10.1080/10618600.2000.10474883
[29]  Figueiredo, M.A.T., Nowak, R.D. and Wright, S.J. (2007) Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems. IEEE Journal of Selected Topics in Signal Processing, 1, 586-597.
https://doi.org/10.1109/JSTSP.2007.910281
[30]  Hale, E.T., Yin, W.T. and Zhang, Y. (2007) A Fifixed-Point Continuation Method for l1-Regularized Minimization with Applications to Compressed Sensing. CAAM Technical Report TR07-07.
https://hdl.handle.net/1911/102072
[31]  Becker, S., Bobin, J. and Candès, E.J. (2011) NESTA: A Fast and Accurate First-Order Method for Sparse Recovery. SIAM Journal on Imaging Sciences, 4, 1-39.
https://doi.org/10.1137/090756855
[32]  Ben-Tal, A. and Nemirovski, A. (2001) Lectures on Modern Convex Optimization-Analysis, Algorithms, and Engineering Applications. In: MPS-SIAM Series on Optimization, Society for Industrial and Applied Mathematics, Philadelphia.
https://doi.org/10.1137/1.9780898718829
[33]  Yang, J. and Zhang, Y. (2011) Alternating Direction Algorithms for l1-Problems in Compressive Sensing. SIAM Journal on Scientific Computing, 33, 250-278.
https://doi.org/10.1137/090777761
[34]  Meinshausen, N. and Yu, B. (2009) Lasso-Type Recovery of Sparse Representations for High-Dimensional Data. Annals of Statistics, 37, 246-270.
https://doi.org/10.1214/07-AOS582
[35]  Zou, H. (2006) The Adaptive Lasso and Its Oracle Properties. Journal of the American Statistical Association, 101, 1418-1429.
https://doi.org/10.1198/016214506000000735
[36]  Fan, J. and Li, R. (2001) Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties. Journal of the American Statistical Association, 96, 1348-1360.
https://doi.org/10.1198/016214501753382273
[37]  Candès, E.J., Wakin, M.B. and Boyd, S.P. (2008) Enhancing Sparsity by Reweighted l1 Minimization. Journal of Fourier Analysis and Applications, 14, 877-905.
https://doi.org/10.1007/s00041-008-9045-x
[38]  Chen, X.J., Xu, F.M. and Ye, Y.Y. (2010) Lower Bound Theory of Nonzero Entries in Solutions of l2-lp Minimization. SIAM Journal on Scientific Computing, 32: 2832-2852.
https://doi.org/10.1137/090761471
[39]  Xu, Z.B., Zhang, H., Wang, Y., Chang, X.Y. and Liang, Y. (2010) L1/2 Regularization. Science China Information Sciences, 53, 1159-1169.
https://doi.org/10.1007/s11432-010-0090-0
[40]  Xu, Z.B., Chang, X.Y., Xu, F.M. and Zhang, H. (2012) L1/2 Regularization: A Thresholding Representation Theory and a Fast Solver. IEEE Transactions on Neural Networks and Learning Systems, 23, 1013-1027.
https://doi.org/10.1109/TNNLS.2012.2197412
[41]  Figueiredo, M.A.T. and Nowak, R.D. (2005) A Bound Optimization Approach to Wavelet-Based Image Deconvolution. IEEE International Conference on Image Processing 2005, Genova, 14-14 September 2005.
https://doi.org/10.1109/ICIP.2005.1530172
[42]  Figueiredo, M.A.T., Bioucas-Dias, J.M. and Nowak, R.D. (2007) Majorization Minimization Algorithms for Wavelet-Based Image Restoration. IEEE Transactions on Image Processing, 16, 2980-2991.
https://doi.org/10.1109/TIP.2007.909318
[43]  Chen, X.J. and Zhou, W.J. (2010) Convergence of Reweighted l1 Minimization Algorithms and Unique Solution of Truncated lp Minimization. Technical Report, Hong Kong Polytechnic University.
https://www.researchgate.net/publication/228532259_Convergence_of_reweighted_l1_minimization_algorithms_and_unique_solution_of_truncated_lp_minimization
[44]  Chen, X.J. and Zhou, W.J. (2014) Convergence of the Reweighted l1 Minimization Algorithm for l2-lp Minimization. Computational Optimization and Applications, 59, 47-61.
https://doi.org/10.1007/s10589-013-9553-8
[45]  Lai, M.J. and Wang, J.Y. (2011) An Unconstrained lq Minimization with for Sparse Solution of Underdetermined Linear Systems. SIAM Journal on Optimization, 21, 82-101.
https://doi.org/10.1137/090775397
[46]  Chen, X.J. (2012) Smoothing Methods for Nonsmooth, Nonconvex Minimization. Mathematical Programming, 134, 71-99.
https://doi.org/10.1007/s10107-012-0569-0
[47]  Lai, M.-J., Xu, Y.Y. and Yin, W.T. (2013) Improved Iteratively Reweighted Least Squares for Unconstrained Smoothed lq Minimization. SIAM Journal on Numerical Analysis, 51, 927-957.
https://doi.org/10.1137/110840364
[48]  Lou, Y.F., Yin, P.H., He, Q. and Xin, J. (2015) Computing Sparse Representation in a Highly Coherent Dictionary Based on Difffference of L1 and L2. Journal of Scientific Computing, 64, 178-196.
https://doi.org/10.1007/s10915-014-9930-1
[49]  Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2011) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends? in Machine Learning, 3, 1-122.
https://doi.org/10.1561/2200000016
[50]  Bai, Y.Q. and Shen, K.J. (2016) Alternating Direction Method of Multipliers for l1-l2-Regularized Logistic Regression Model. Journal of the Operations Research Society of China, 4, 243-253.
https://doi.org/10.1007/s40305-015-0090-2
[51]  Selesnick, I. (2017) Sparse Regularization via Convex Analysis. IEEE Transactions on Signal Processing, 65, 4481-4494.
https://doi.org/10.1109/TSP.2017.2711501
[52]  Li, Q., Bai, Y.Q., Yu, C.J. and Yuan, Y.-X. (2019) A New Piecewise Quadratic Approximation Approach for L0 Norm Minimization Problem. Science China Mathematics, 62, 185-204.
https://doi.org/10.1007/s11425-017-9315-9
[53]  Gao, X.R., Bai, Y.Q. and Li, Q. (2021) A Sparse Optimization Problem with Hybrid L2-Lp Regularization for Application of Magnetic Resonance Brain Images. Journal of Combinatorial Optimization, 42, 760-784.
https://doi.org/10.1007/s10878-019-00479-x
[54]  Gao, X.R., Bai, Y.Q., Fang, S.-C., et al. (2023) A New Hybrid lp-l2 Model for Sparse Solutions with Applications to Image Processing. Journal of Industrial and Management Optimization, 19, 890-915.
https://doi.org/10.3934/jimo.2021211
[55]  吕昊, 冯仲仁, 王雄江, 等. 基于混合鲸鱼退火算法和稀疏正则化的结构损伤识别[J]. 振动与冲击, 2021, 40(17): 85-91.
[56]  宋泽树, 宋泽树, 石娟娟, 等. 广义平滑对数正则化稀疏分解方法研究及其在齿轮箱复合故障诊断中的应用[J]. 机械工程学报, 2022, 58(23): 123-137.
[57]  杨雪, 李峰, 鹿明, 等. 混合稀疏表示模型的超分辨率重建[J]. 遥感学报, 2022, 26(8): 1685-1697.
[58]  黄淑英, 吴昕, 杨勇, 等. 自适应正则化稀疏表示的遥感图像SR重建[J]. 小型微型计算机系统, 2023, 44(3): 573-581.
[59]  马敏, 于洁, 范文茹. 基于改进低秩稀疏正则化的CFRP电阻抗层析成像算法研究[J]. 振动与冲击, 2022, 41(14): 151-157.
[60]  Ma, T.-H., Xu, Z.B., Meng, D.Y. and Zhao, X.-L. (2021) Hyperspectral Image Restoration Combining Intrinsic Image Characterization with Robust Noise Modeling. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14: 1628-1644.
https://doi.org/10.1109/JSTARS.2020.3046488
[61]  Wang, Y.Y., He, Z.M., Zhan, X., Fu, Y.H. and Zhou, L.M. (2022) Three-Dimensional Sparse SAR Imaging with Generalized Lq Regularization. Remote Sensing, 14, Article No. 288.
https://doi.org/10.3390/rs14020288
[62]  Xu, Y.S. and Zeng, T.S. (2023) Sparse Deep Neural Network for Nonlinear Partial Differential Equations. Numerical Mathematics: Theory, Methods and Applications, 16, 58-78.
https://doi.org/10.4208/nmtma.OA-2022-0104

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133