Deep learning (DL) techniques, more specifically Convolutional Neural Networks (CNNs), have become increasingly popular in advancing the field of data science and have had great successes in a wide array of applications including computer vision, speech, natural language processing, etc. However, the training process of CNNs is computationally intensive and has high computational cost, especially when the dataset is huge. To overcome these obstacles, this paper takes advantage of distributed frameworks and cloud computing to develop a parallel CNN algorithm. MapReduce is a scalable and fault-tolerant data processing tool that was developed to provide significant improvements in large-scale data-intensive applications in clusters. A MapReduce-based CNN (MCNN) is developed in this work to tackle the task of image classification. In addition, the proposed MCNN adopted the idea of adding dropout layers in the networks to tackle the overfitting problem. Close examination of the implementation of MCNN as well as how the proposed algorithm accelerates learning are discussed and demonstrated through experiments. Results reveal high classification accuracy and significant improvements in speedup, scaleup and sizeup compared to the standard algorithms.
References
[1]
Lecun, Y., Bottou, L., Bengio, Y. and Haffner, P. (1998) Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86, 2278-2324. https://doi.org/10.1109/5.726791
[2]
Egmont-Petersen, M., de Ridder, D. and Handels, H. (2002) Image Processing with Neural Networks—A Review. Pattern Recognition, 35, 2279-2301. https://doi.org/10.1016/s0031-3203(01)00178-9
[3]
Simonyan, K. and Zisserman, A. (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv: 1409.1556. https://doi.org/10.48550/arXiv.1409.1556
[4]
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015) ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115, 211-252. https://doi.org/10.1007/s11263-015-0816-y
[5]
Nowlan, S.J. and Platt, J.C. (1994) A Convolutional Neural Network Hand Tracker. Proceedings of the 7th International Conference on Neural Information Processing Systems, Denver Colorado, 1 January 1994, 901-903.
[6]
Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J. and Zisserman, A. (2014) The Pascal Visual Object Classes Challenge: A Retrospective. International Journal of Computer Vision, 111, 98-136. https://doi.org/10.1007/s11263-014-0733-5
[7]
Lin, T., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., et al. (2014) Microsoft COCO: Common Objects in Context. Computer Vision—ECCV 2014, Zurich, 6-12 September 2014, 740-755. https://doi.org/10.1007/978-3-319-10602-1_48
[8]
Ciresan, D., Giusti, A., Gambardella, L. and Schmidhuber, J. (2012) Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images. Advances in Neural Information Processing Systems, 25, 2843-2851.
[9]
Girshick, R., Donahue, J., Darrell, T. and Malik, J. (2014) Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 23-28 June 2014, 580-587. https://doi.org/10.1109/cvpr.2014.81
[10]
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/cvpr.2016.90
[11]
Leung, J. and Chen, M. (2019) Image Recognition with MapReduce Based Convolutional Neural Networks. 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), New York, 10-12 October 2019, 119-125. https://doi.org/10.1109/uemcon47517.2019.8992932
[12]
Cun, Y.L., Boser, B., Denker, J.S., Henderson, D., Howar, R.E., Hubbard, W. and Jackel, L.D. (1990) Handwritten Digit Recognition with a Back-Propagation Network. Advances in Neural Information Processing Systems, 160, 396-404.
[13]
Nebauer, C. (1998) Evaluation of Convolutional Neural Networks for Visual Recognition. IEEE Transactions on Neural Networks, 9, 685-696. https://doi.org/10.1109/72.701181
[14]
Norlander, R., Grahn, J. and Maki, A. (2015) Wooden Knot Detection Using Convnet Transfer Learning. Image Analysis, Copenhagen, 15-17 June 2015, 263-274. https://doi.org/10.1007/978-3-319-19665-7_22
[15]
Park, J., Kwon, B., Park, J. and Kang, D. (2016) Machine Learning-Based Imaging System for Surface Defect Inspection. International Journal of Precision Engineering and Manufacturing-Green Technology, 3, 303-310. https://doi.org/10.1007/s40684-016-0039-x
[16]
Kwak, J.T., Xu, S. and Wood, B.J. (2015) Efficient Data Mining for Local Binary Pattern in Texture Image Analysis. Expert Systems with Applications, 42, 4529-4539. https://doi.org/10.1016/j.eswa.2015.01.055
[17]
Oliva, J.T., Lee, H.D., Spolaôr, N., Coy, C.S.R. and Wu, F.C. (2016) Prototype System for Feature Extraction, Classification and Study of Medical Images. Expert Systems with Applications, 63, 267-283. https://doi.org/10.1016/j.eswa.2016.07.008
[18]
Affonso, C., Rossi, A.L.D., Vieira, F.H.A. and de Leon Ferreira de Carvalho, A.C.P. (2017) Deep Learning for Biological Image Classification. Expert Systems with Applications, 85, 114-122. https://doi.org/10.1016/j.eswa.2017.05.039
[19]
Chebbi, I., Boulila, W. and Farah, I.R. (2016) Improvement of Satellite Image Classification: Approach Based on Hadoop/MapReduce. 2016 2nd International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), Monastir, 21-23 March 2016, 31-34. https://doi.org/10.1109/atsip.2016.7523046
[20]
Alham, N.K., Li, M., Liu, Y. and Hammoud, S. (2011) A Mapreduce-Based Distributed SVM Algorithm for Automatic Image Annotation. Computers & Mathematics with Applications, 62, 2801-2811. https://doi.org/10.1016/j.camwa.2011.07.046
[21]
Cao, J., Cui, H., Shi, H. and Jiao, L. (2016) Big Data: A Parallel Particle Swarm Optimization-Back-Propagation Neural Network Algorithm Based on Mapreduce. PLOS ONE, 11, e0157551. https://doi.org/10.1371/journal.pone.0157551
[22]
Nair, V. and Hinton, G.E. (2010) Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, 21-24 June 2010, 807-814.
[23]
He, K., Zhang, X., Ren, S. and Sun, J. (2015) Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 7-13 December 2015, 1026-1034. https://doi.org/10.1109/iccv.2015.123
[24]
Alex, K. (2009) Learning Multiple Layers of Features from Tiny Images. https://www.cs.toronto.edu/kriz/learning-features-2009-TR.pdf
[25]
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. (2014) Dropout: A Simple Way to Prevent Neural Networks from Overfitting. The Journal of Machine Learning Research, 15, 1929-1958.
[26]
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., et al. (2015) Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 7-12 June 2015, 1-9. https://doi.org/10.1109/cvpr.2015.7298594
[27]
Liu, Y., Yang, J., Huang, Y., Xu, L., Li, S. and Qi, M. (2015) MapReduce Based Parallel Neural Networks in Enabling Large Scale Machine Learning. Computational Intelligence and Neuroscience, 2015, Article 297672. https://doi.org/10.1155/2015/297672
