%0 Journal Article
%T 一种新型的分数阶梯度下降法在深度神经网络中的应用<br>A Novel Fractional-Order Gradient Descent Method with Its Application in Deep Neural Network
%A 吴昊
%J Advances in Applied Mathematics
%P 3182-3192
%@ 2324-8009
%D 2024
%I Hans Publishing
%R 10.12677/aam.2024.137304
%X 文章基于Caputo分数阶微积分，提出了一种新型的适用于神经网络模型训练的分数阶梯度下降法。该算法通过改变积分区间下界，成功将分数阶阶次拓展到了(0, 2)区间，增加了阶次的选择范围，同时，本文基于梯度裁剪机制，从遗憾函数的角度证明了该算法的收敛性，保证了算法的理论可行性。最后，基于CIFAR-10公开数据集的数值实验表明，在选择了合适的阶次的情况下，本文所提出的算法相比于传统的整数阶梯度法，能够获得更快的收敛速度和更高的收敛精度。<br />
This study introduces a novel fractional gradient descent algorithm based on Caputo fractional calculus which is tailored for training neural network models. By adjusting the lower limit of the integral interval, the proposed algorithm extends the fractional order to the (0, 2) range, thereby enhancing the choices of fractional order. Concurrently, this work proves the convergence of the proposed algorithm in detail from the perspective of the regret function based on the gradient clipping mechanism, affirming its theoretical validity. Finally, the numerical experiment based on the publicly available CIFAR-10 dataset, reveals that the proposed algorithm outperforms conventional integer-order gradient method in terms of both convergence speed and convergence accuracy when operated at an optimal order.
%K Caputo分数阶微积分，梯度下降法，遗憾，深度神经网络<br>Caputo Fractional Calculus
%K Gradient Descent Method
%K Regret
%K Deep Neural Network
%U http://www.hanspub.org/journal/PaperInformation.aspx?PaperID=91615