|
|
- 2016
基于筛选机制的L1核学习机分布式训练方法
|
Abstract:
摘要： 为降低无线传感器网络中核学习机训练时的数据通信代价和节点计算代价,研究了基于筛选机制的L1正则化核学习机分布式训练方法。提出了一种节点局部训练样本筛选机制,各节点利用筛选出的训练样本,在节点模型对本地训练样本的预测值与邻居节点间局部最优模型对本地训练样本预测值相一致的约束下,利用增广拉格朗日乘子法求解L1正则化核学习机分布式优化问题,利用交替方向乘子法求解节点本地的L1正则化核学习机的稀疏模型;仅依靠相邻节点间传输稀疏模型的协作方式,进一步优化节点局部模型,直至各节点模型收敛。基于此方法,提出了基于筛选机制的L1正则化核最小平方误差学习机的分布式训练算法。仿真实验验证了该算法在模型预测正确率、模型稀疏率、数据传输量和参与模型训练样本量上的有效性和优势。
Abstract: To decrease the amount of data transferred and the computing cost during training a kernel machine in wireless sensor network, a distributed training method for L1-regularized Kernel Minimum Square Error machine based on filtering mechanism was proposed. First, filtering mechanism of samples was presented and used on each node. Second, with consistency constraint on the local model of each node and its local optimal one obtained by exchanging the local model with its all neighbours, the distributed optimization problem of L1-regularized Kernel Minimum Square Error machine was solved by Augmented Lagrange Method of Multipliers, and the local optimization problem of L1-regularized Kernel Minimum Square Error machine on each node was solved by Alternating Direction Method of Multipliers. Then, the spares model obtained on each node was transferred to its all neighbor nodes. This process iterates until the local model on each node converges. For carrying out this method,a novel distributed training algorithm for L1-regularized Kernel Minimum Square Error based on filtering of samples was proposed. Simulation results prove the validity of the proposed algorithm in terms of convergence, sparse rate of model, the amount of data transferred and the number of samples used in model training
| [1] | 吕方旭,张金成,郭相科,等. 基于WSN的战场声目标多特征联合智能分类识别[J]. 科学技术与工程, 2013, 13(35):10713-10721. Lü Fangxu, ZHANG Jincheng, GUO Xiangke, et al. Multi feature joint intelligent classification and recognition of battlefield acoustic targets based on WSN[J]. Science Technology and Engineering, 2013, 13(35):10713-10721. |
| [2] | 韩屏,李方敏,罗婷. 无线传感器网络的分布式目标跟踪算法[J]. 北京邮电大学学报, 2009, 32(1):90-94. HAN Ping, LI Fangmin, LUO Ting. Distributed target tracking algorithm for wireless sensor networks[J]. Journal of Beijing University of Posts and Telecommunications, 2009, 32(1):90-94. |
| [3] | SCH？LKOPF B, SMOLA A. Learning with kernels: support vector machines, regularization, optimization and beyond[J]. MIT Press, 2002: 61-118. |
| [4] | BERTSEKAS D P, TSITSIKLIS J N. Parallel and distributed computation: numerical methods[M]. London: Prentice-Hall, 1997: 243-247. |
| [5] | 刘倩,崔晨,周杭霞. 改进型SVM多类分类算法在无线传感器网络中的应用[J].中国计量学院学报, 2013, 24(3):298-303. LIU Qian, CUI Chen, ZHOU Hangxia. Application of improved SVM multi class classification algorithm in wireless sensor networks[J]. Journal of China Jiliang University, 2013, 24(3):298-303. |
| [6] | RAJ A B, RAMESH M V, KULKARNI R V, et al. Security enhancement in wireless sensor networks using machine learning[C] // Proceedings of IEEE 14th International Conference on High Performance Computing and Communications. New York: ACM, 2012: 1264-1269. |
| [7] | 及歆荣,侯翠琴,侯义斌. 无线传感器网络下线性支持向量机分布式协同训练方法研究[J].电子与信息学报,2015, 37(3):708-714. JI Xinrong, HOU Cuiqin, HOU Yibin. Research on distributed cooperative training method for linear support vector machine in wireless sensor networks[J]. Journal of Electronics and Information, 2015, 37(3):708-714. |
| [8] | TAGHVAEEYAN S, RAJAMANI R. Portable roadside sensors for vehicle counting, classification, and speed measurement[J]. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(1):73-83. |
| [9] | SHAHID N, NAQVI I H, QAISAR BIN S. Quarter-sphere svm: attribute and spatio-temporal correlations based outlier and event detection in wireless sensor networks[J]. IEEE Wireless Communication and Networking Conference: Mobile and Wireless Networks, 2012: 2048-2053. |
| [10] | GUESTRIN C, BODIK P, THIBAUX R, et al. Distributed regression: an efficient framework for modeling sensor network data[C] // Proceedings of the International Conference on Information Processing in Sensor Networks. New York: ACM, 2004: 1-10. |
| [11] | PREDD J B, KULKARNI S R, POOR H V. Distributed kernel regression: an algorithm for training collaboratively[C] // Proceedings of IEEE Information Theory Workshop. New York: IEEE, 2006: 332-336. |
| [12] | PREDD J B, KULKARNI S R, POOR H V. A Collaborative training algorithm for distributed learning[J]. IEEE Transactions on Information Theory, 2009, 55(4):1856-1870. |
| [13] | FLOURI K, BEFERULL-LOZANO B, TSAKALIDES P. Optimal gossip algorithm for distributed consensus SVM training in wireless sensor networks[C] // Proceedings of the 16th International Conference on Digital Signal Proces<i>sin</i>(DSP '09). Piscataway: IEEE, 2009: 886-891. |
| [14] | FLOURI K, BEFERULL-LOZANO B, TSAKALIDES P.Training a support-vector machine-based classifier in distributed sensor networks[C] // Proceedings of the 14nd European Signal Processing Conference. Florence:[S.l.] , 2006: 4-8. |
| [15] | YUMAO L, ROYCHOWDHURY V, VANDENBERGHE L. Distributed parallel support vector machines in strongly connected networks[J]. IEEE Transactions on Neural Networks, 2008, 19(7):1167-1178. |
| [16] | XU Jianhua, ZHANG Xuegong, LI Yanda. Kernel MSE algorithm: a unified framework for KFD, LS-SVM and KRR[C] // Proceedings of International Joint Conference on Neural Networks. Piscataway: IEEE, 2001: 1486-1491. |
| [17] | BOYD S, PARIKH N, CHU E. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends in Machine Learning, 2011, 3(1):1-122. |
