全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...

DBN网络的深度确定方法

DOI: 10.13195/j.kzyjc.2013.1390, PP. 256-260

Keywords: 深度信念网络,网络深度,无监督学习,数字识别

Full-Text   Cite this paper   Add to My Lib

Abstract:

针对DBN网络隐含层层数难以选择的问题,首先从数学生物学角度分析了随机初始化的梯度下降法导致网络训练失败的原因,并进行验证,证明了RBM重构误差与网络能量的正相关定理;然后根据隐含层和误差的关系,提出一种基于重构误差的网络深度判断方法,在训练过程中自组织地训练网络,使其能够以一种接近人类处理问题的方式解决AI问题.手写数字识别的实验表明,该方法能够有效提高运算效率,降低运算成本.

References

[1]  Lee T S, Mumford D. Hierarchical Bayesian inference in the visual cortex[J]. Optical Society of America, 2003, 20(7): 1434-1448.
[2]  Rossi A F, Desimone R, Ungerleider L G. Contextual modulation in primary visual cortex of macaques[J]. J of Neuroscience, 2001, 21(5): 1689-1709.
[3]  Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
[4]  Dahl G E, Yu D, Deng L, et al. Large vocabulary continuous speech recognition with context-dependent DBN-HMMS[C]. Proc of IEEE Int Conf on Acoustics, Speech and Signal Processing. Prague, 2011: 4688-4691.
[5]  Deselaers T, Hasan S, Bender O, et al. A deep learning approach to machine transliteration[C]. Proc of the 4th Workshop on Statistical Machine Translation. Athens, 2009: 233-241.
[6]  Fasel I, Berry J. Deep belief networks for real-time extraction of tongue contours from ultrasound during speech[C]. Proc of the 20th Int Conf on Pattern Recognition. Stroudsburg: Association for Computational Linguistics, 2010: 1493-1496.
[7]  Deng L, Seltzer M L, Yu D, et al. Binary coding of speech spectrograms using a deep auto-encoder[C]. Proc of the 11th Annual Conf on Int Speech Communication Association. Makuhair, 2010: 1692-1695.
[8]  陈宇, 郑德权, 赵铁军. 基于Deep Belief Nets 的中文名实体关系抽取[J]. 软件学报, 2012, 23(10): 2572-2585.
[9]  (Chen Y, Zheng D Q, Zhao T J. Chinese relation extraction based on deep belief nets[J]. J of Software, 2012, 23(10): 2572-2585.)
[10]  Bengio Y. Learning deep architectures for AI[J]. Foundations & Trends in Machine Learning, 2009, 2(1): 1-127.
[11]  Yoshua Bengio, Pascal Lamblin, Dan Popovici, et al. Greedy layer-wise training of deep networks[C]. Advances in Neural Information Processing Systems 19 (NIPS 2006). Vancouver, 2007: 153-160.
[12]  Hinton G E, Osindero S, Teh Y. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.
[13]  Yang Hu, Yong Yu. Learning Restricted Boltzmann Machines using Mode-Hopping MCMC[C]. The 4th Int Conf on Machine Learning and Computing. Xi’an, 2012, 20: 105-110.
[14]  Le Roux, Nicolas, Yoshua Bengio. Representational power of restricted boltzmann machines and deep belief networks[J]. Neural Computation, 2008, 20(6): 1631-1649.
[15]  Thomas P, Karnowski. Deep spatiotemporal feature learning with application to image classification[C]. The 9th Int Conf on Machine Learning and Applications. Washington, 2010: 883-889.
[16]  Li Deng, Dong Yu, John Platt. Scalable stacking and learning for building deep architectures[C]. ICASSP. Kyoto, 2012: 2133-2137.
[17]  Bengio Yoshua, Olivier Delalleau. On the expressive power of deep architectures[C]. Algorithmic Learning Theory. Berlin: Springer /Heidelberg, 2011: 18-36.
[18]  MarcAurelio Ranzato, Christopher Poultney, Sumit Chopra, et al. Efficient learning of sparse representations with an energy-based model[C]. Advances in Neural Information Processing Systems(NIPS 2006). Vancouver, 2007: 1137-1144.

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133