全部 标题 作者
关键词 摘要

OALib Journal期刊
ISSN: 2333-9721
费用:99美元

查看量下载量

相关文章

更多...
-  2016 

基于领域知识的神经网络泛化性能研究进展 Research advances of neural network generalization performance based on domain knowledge

Keywords: 神经网络,泛化性能,领域知识,先验知识

Full-Text   Cite this paper   Add to My Lib

Abstract:

从样本复杂性、结构复杂性、学习策略和建模技术4个方面对基于领域知识的神经网络泛化性能研究进展进行了评述,指出了目前基于领域知识神经网络泛化性能研究存在的主要问题是只是利用研究对象的单调性、凸性、对称性和增益等一些简单非线性特征来虚拟训练样本、形成非监督学习算法约束条件、构造节点作用函数等方面.利用关于研究复杂对象部分已知的物理机制或动力学特性来建立有一定物理基础的神经网络模型,从而有效控制网络训练存在的过学习问题是今后神经网络泛化理论与方法研究的主要发展趋势

References

[1]  Abu-Mostafa Y S.Learning from hints in neural networks[J].Journal of Complexity,1990,6:192-198.
[2]  Moody J S.The efficient number of parameters:An analysis of generalization and regularization in nonlinear learning system[J].San Mateo,NIPS4,1992:847-854.
[3]  Barron A R.Approximation and estimation bounds for artificial neural networks[J].Machine Learning,1994,14:115-133.
[4]  Geman,Bienenstock,Doursat.Neural networks and the bias variance dilemma[J].Neural Computation,1992,4(1):1-58.
[5]  Castellano G,Fanilli A M,Pelillo M.A pruning algorithm for feed-forward neural networks[J].Neural Networks,1997,8(3):519-531.
[6]  Ponnapalli P V S.A formal selection and pruning algorithm for feed-forward artificial neural network optimization[J].Neural Networks,1999,10(4):964-968.
[7]  Kwok T Y,Yeung D Y.Constructive algorithm for structure learning in feed-forward neural networks for regression problems[J].Neural Networks,1997,8(3):630-645.
[8]  Lehtokangas M.Modified cascade-correlation learning for classification[J].Neural Networks,2000,11(3):795-798.
[9]  Hu T S,Lam K C,Thomas N G.River flow time series prediction with range-dependent neural network[J].Hydrological Sciences Journal,2001,46(5):729-745.
[10]  Perrone M P.General averaging results for convex optimization[C]//Proceedings of 1993 Connectionist Models Summer School,Hillsdale,1994:364-367.
[11]  Sietsma J,Dow R J F.Creating artificial neural networks that generation[J].Neural Networks,1991,4:67-79.
[12]  Maniezzo V.Genetic evolution of the topology and weight distribution of neural networks[J].Neural Networks,1994,5:39-53.
[13]  Fahlman S E,Lebiere C.The cascade connection learning architecture[C]//Advance in Neural Information Processing Systems,1990:524-532.
[14]  Amari S,Murata N,Muller K R.Asymptotic statistical theory of networks,over-training and cross-validation[J].Neural Networks,1997,8(5):985-996.
[15]  Partridge D.Network generation differences quantified[J].Neural Networks,1996,9(2):263-271.
[16]  Kanaya F,Miyake S.Bayesian statistical behavior and valid generalization of pattern classifying neural networks[J].Neural Networks,1991,2(4):471-475.
[17]  Hartman.Training feedforward neural networks with gain constraints[J].Neural Computation,2000,12(4):811-829.
[18]  Deco.Unsupervised mutual information criterion for elimination of overtraining in supervised multilayer networks[J].Neural Computation,1994,7:1398-1419.
[19]  Ehrenfeucht A.A general lower bound on the number of examples needed for learning[C]//Proceedings of the Workshop on Computational Learning Theory,Morgan Kaufmann,1988.
[20]  Bishop C M.Neural Networks for Pattern Recognition[M].Oxford University Press,1995.
[21]  Breiman L.Bagging predictors[J].Machine Learning,1996,24(2):123-140.
[22]  申金媛,刘玥,张文伟,等.利用汉明距离优选神经网络学习样本[J].光学学报,2000,20(9):1229-1233.Shen Jinyuan,Liu Yue,Zhang Wenwei,et al.Selecting of learning samples based on Hamming distance[J].Acta Optica Sinica,2000,20(9):1229-1233.
[23]  任俊玲.基于广义置信度的样本选择方法[J].中文信息学报,2007,21(3):106-109.Ren Junling.A pattern selection algorithm based on the generalized confidence[J].Journal of Chinese Information Processing,2007,21(3):106-109.
[24]  杨华波,张士峰,蔡洪.BP神经网络中定性信息的使用[J].计算技术与自动化,2005,24(3):15-17.Yang Huabo,Zhang Shifeng,Cai Hong.The using of qualitative information in BP neural networks[J].Computing Technology and Automation,2005,24(3):15-17.
[25]  陈祥光,薛锦诚,傅若农,等.引入模拟辅助样本提高BP网络的泛化能力[J].分析科学学报,2002,18(2):137-141.Chen Xiangguang,Xue Jincheng,Fu Ruonong,et al.Improvement of the generalization of BP networks with the aid of simulating assistant samples[J].Journal of Analytical Science,2002,18(2):137-141.
[26]  高学星,孙华刚,侯保林.使用不同置信级训练样本的神经网络学习方法[J].电子与信息学报,2014,36(6):1308-1311.Gao Xuexing,Sun Huagang,Hou Baolin.A neural network learning method using samples with different confidence levels[J].Journal of Electronics&Information Technology,2014,36(6):1308-1311.
[27]  Wang,Venkatesh.Optimal stopping and effective machine complexity in learning[J].NIPS6,CA,San Mat,1994:303-310.
[28]  Kearns M.A bound on the error of cross validation using the approximation and estimation rates with consequence for the training-test split[J].Neural Computation,1997,9:1143-1161.
[29]  Polycarpou M,Loannou P.Learning and convergence analysis of neural type structure networks[J].Neural Networks,1992,3:39-50.
[30]  吕柏权.使用三层神经元网络的先验信息新学习方法[J].中国科学,2004,34(4):374-390.
[31]  胡铁松,沈佩君.前馈网络泛化性能改进的目的规划方法研究[J].系统工程学报,1997,12(2):34-39.Hu Tiesong,Shen Peijun.The goal programming method for improving generalization performance of the feedforward neural networks[J].Journal of System Engineering,1997,12(2):34-39.
[32]  Srecko Milanic,Stanko Strmcnik.Incorporating prior knowledge into artificial neural networks-an industrial case study[J].Neurocomputing,2004,62:131-151.
[33]  Hu T S,Lam K C,Thomas N G.A modified neural network for improving river flow prediction[J].Hydrological Sciences Journal,2005,50(2):299-318.
[34]  Jain,Srinivasulu.Integrated approach to modelling decomposed flow hydrograph using artificial neural network and conceptual techniques[J].Journal of Hydrology,2006,317(3-4):291-306.
[35]  Lloyd H C Chua,Tommy S W Wong.Runoff forecasting for an asphalt plane by artificial neural networks and comparisons with kinematic wave and autoregressive moving average models[J].Journal of Hydrology,2010,397:191-201.
[36]  Al-Mashouq K A.Including hints in training neural networks[J].Neural Computation,1991,3:418-427.
[37]  Abu-Mostafa Y S.Financial application of learning from hints[J].Ambridge,1995,MA:411-418.
[38]  Sill,Abu-MostafaY S.Advances in Neural Information Processing Systems[M].MIT Press,1997:643-640.
[39]  魏海坤,徐翤鑫,宋文忠.神经网络的泛化理论和泛化方法[J].自动化学报,2001,27(6):806-815.Wei Haikun,Xu Sixin,Song Wenzhong.Generalization theory and generalization methods for neural networks[J].Acta Automatic Sinica,2001,27(6):806-815.
[40]  陈翀伟,陈伟,陈德钊,等.基于先验知识的前馈网络对原油实沸点蒸馏曲线的仿真[J].高校化工学报,2001,15(4):351-356.Chen Chongwei,Chen Wei,Chen Dezhao,et al.Feedforward networks based on prior knowledge and its application in modeling the true boiling point curve of the crude oil[J].Journal of Chemical Engineering of Chinese Universities,2001,15(4):351-356.
[41]  Sanguesa,Cortes.Prior knowledge for learning networks in non-probabilistic settings[J].International Journal of Approximate Reasoning,2000,24:103-120.
[42]  Niyogo,Girosi.On the relationship between generalization error,hypothesis complexity,and sample complexity for radial basis function[J].Neural Computation,1996,8:819-842.
[43]  李祚泳,彭荔红.BP网络学习能力与泛化能力满足的不确定关系式[J].中国科学(E辑),2003,33(10):887-895.
[44]  Ferrari,Jensenius.A constrained optimization approach to preserving prior knowledge during incremental training[J].Neural Networks,2008,19(6):996-1009.
[45]  Mezard M,Nadal J P.Learning in feed-forward layered networks:the tiling algorithm[J].Journal of Physics,1989,22:2191-2203.
[46]  Baum E M.Neural networks algorithm that learn in polynomial time from example and queries[J].Neural Networks,1991,2(3):5-19.
[47]  George N K.On over-fitting,generalization and randomly expanded training sets[J].Neural Networks,2000,11(5):1050-1057.
[48]  Frean M.The upstart algorithm:a method for constructing and training feed-forward neural network[J].Neural Computation,1990,2:198-209.
[49]  吴佑寿.利用输入信号先验知识构造某些分类神经网络的研究[J].中国科学(E辑),1996,26(2):140-144.
[50]  Zhao S Dillon.Incorporating prior knowledge in the form of production rules into neural networks using boolean-like neurons[J].Applied Intelligence,1997,7:275-285.
[51]  薛福珍,柏洁.基于先验知识和神经网络的非线性建模与预测控制[J].系统仿真学报,2004,(5):1057-1059.Xue Fuzhen,Bai Jie.Nonlinear modeling and predictive control based on prior knowledge and neural networks[J].Journal of System Simulation,2004,(5):1057-1059.
[52]  Fogel.An information criterion for optimal neural network selection[J].Neural Networks,1991,2(5):490-497.
[53]  Bartlett P L.For valid generalization:the size of the weight is more important than the size of the network[J].NIPS9,MA,Cambridge,1995:134-140.
[54]  Freund Y,Schapire R E.A decision-theoretic generalization of on-line learning and an application to boosting[J].Journal of Computer and System Sciences,1997,55(1):119-139.
[55]  Guan Donghai,Yuan Weiwei,Lee Youngkoo.Training data selection based on fuzzy c-means[C]//IEEE World Congress on Computational Intelligence,Hong Kong,2008:761-765.
[56]  Wang,Venkatesh.Temporal dynamics of generalization in neural networks[J].NIPS7,MA,Cambridge,1995:263-270.
[57]  Stork,Allen.How to solve the n-bit parity problem with two hidden units[J].Neural Networks,1992,5(6):923-926.
[58]  Federico Girosi,Nicholas Tung Chan.Prior knowledge and the creation of“Virtual”examples for RBF networks[C]//Neural Networks for Signal Processing,1995:201-210.
[59]  Barbar,Saad.Does extra knowledge necessarily improve generalization[J].Neural Computation,1996,8(1):202-214.
[60]  Psichogios D C,Ungar L H.SVD-Net:an algorithm that automatically selections network structure[J].Neural Networks,1994,5(3):513-515.
[61]  Baum E M,Haussler D.What size net gives valid generalization[J].JAMC,1989,36(4):929-965.
[62]  Kamimurar.Unification of information maximization and minimization[C]//Neural Information Processing Systems,1996:508-514.
[63]  Vapnik V N.The Nature of Statistical Learning Theory[M].New York:Springer Verlag,1995:1-20.
[64]  Mackay D J C.Information based objective function for active data selection[J].Neural Computation,1992,4:590-604.
[65]  何超,徐立新,张宇河.CMAC算法收敛性分析及泛化能力研究[J].控制与决策,2001,16(5):523-530.He Chao,Xu Lixin,Zhang Yuhe.Convergence and generalization ability of CMAC[J].Control and Decision,2001,16(5):523-530.
[66]  Holmstrom L,Koistinen P.Using additive noise in back propagation training[J].Neural Networks,1992,3(1):24-38.
[67]  An G.The effect of adding noise during back-propagation training on a generalization performance[J].Neural Computation,1996,8:643-671.
[68]  Hara K,Nakayama K Karaf.A training data selection in on-line training for multilayer neural networks[C]//IEEE World Congress on Computational Intelligence,1998,3:2247-2252.
[69]  顾正彬,李贺军,李克智,等.C/C复合材料等温CVI工艺模糊神经网络建模[J].稀有金属材料与工程,2004,33(10):1037-1040.Gu Zhengbin,Li Hejun,Li Kezhi,et al.Modeling of isothermal CVI process of C/C composites by fuzzy neural network[J].Rare Mental Materials and Engineering,2004,33(10):1037-1040.
[70]  Lincoln W L,Skrzypek J.Synergy of clustering multiple back-propagation networks[J].NIPS2,San Mateo,CA,1990:650-657.
[71]  Atiya A,Ji C.How initial condition affect generalization performance in large networks[J].Neural Networks,1997,8(2):448-451.
[72]  张海涛,陈宗海,向微,等.机理混合自适应时延神经网络建模和控制算法[J].系统仿真学报,2004,16(12):2709-2712.Zhang Haitao,Chen Zonghai,Xiang Wei,et al.An algorithm of modeling and control based on mechanism hybrid adaptive time delay neural network[J].Journal of System Simulation,2004,16(12):2709-2712.
[73]  Abu-Mostafa Y S.A method of learning from hints[J].NIPS,CA,1993:73-80.
[74]  Perantonis,Ampazis.Constrained learning in neural networks:Application to stable factorization of 2Dpolynomials[J].Neural Processing Letters,1998,(7):5-14.
[75]  YaochuJin,Bernhard Sendhoff.Knowledge incorporation into neural networks from fuzzy rules[J].Neural Processing Letters,1999,10(3):231-242.
[76]  李祚泳,易勇鸷.BP网络学习能力与泛化能力之间满足的定量关系式[J].电子学报,2003,31(9):1341-1344.Li Zuoyong,Yi Yongzhi.Quantitative relation between learning ability and generalization ability of BP neural network[J].Acta Electronic Sinica,2003,31(9):1341-1344.
[77]  Reed R.Pruning algorithms-A survey[J].Neural Networks,1993,4:740-747.
[78]  Bo S.Optimal weight decay in perception[C]//Proceedings of the International Conference on Neural Networks,1996:551-556.
[79]  许少华,路阳,席海青,等.样本先验知识在神经网络训练中的应用[J].大庆石油学院学报,2004,28(6):66-69.Xu Shaohua,Lu Yang,Xi Haiqing,et al.Application of sample prior knowledge to nerve networks training[J].Journal of Daqing Petroleum Institute,2004,28(6):66-69.
[80]  Daniel T.Solving inverse problem by Bayesian neural network iterative inversion with ground truth incorporation[J].Signal Processing,1997,45(11):553-567.
[81]  Nowlan S J.Simplifying neural networks by soft weight sharing[J].Neural Computation,1992,4:473-493.
[82]  Jean,Wang.Weight smoothing to improve network generalization[J].Neural Networks,1994,5(5):752-763.
[83]  胡铁松.岩体爆破效应预测的前馈网络目的规划方法[J].水利学报,1997,(9):55-59.Hu Tiesong.The goal programming algorithm for feedforward neural network and its application to the prediction of rock mass blasting[J].Journal of Hydraulic Engineering,1997,(9):55-59.
[84]  陈立甲,伞冶,王子才,等.锅炉过热器系统机理与神经网络组合建模方法[J].中国电机工程学报,2001,21(1):73-76.Chen Lijia,San Ye,Wang Zicai,et al.A physical law and neural network integrated modeling method for bolier superheater systems[J].Proceedings of the CSEE,2001,21(1):73-76.

Full-Text

Contact Us

service@oalib.com

QQ:3279437679

WhatsApp +8615387084133