Traffic sign recognition (TSR, or Road Sign Recognition, RSR) is one of the Advanced Driver Assistance System (ADAS) devices in modern cars. To concern the most important issues, which are real-time and resource efficiency, we propose a high efficiency hardware implementation for TSR. We divide the TSR procedure into two stages, detection and recognition. In the detection stage, under the assumption that most German traffic signs have red or blue colors with circle, triangle or rectangle shapes, we use Normalized RGB color transform and Single-Pass Connected Component Labeling (CCL) to find the potential traffic signs efficiently. For Single-Pass CCL, our contribution is to eliminate the “merge-stack” operations by recording connected relations of region in the scan phase and updating the labels in the iterating phase. In the recognition stage, the Histogram of Oriented Gradient (HOG) is used to generate the descriptor of the signs, and we classify the signs with Support Vector Machine (SVM). In the HOG module, we analyze the required minimum bits under different recognition rate. The proposed method achieves 96.61% detection rate and 90.85% recognition rate while testing with the GTSDB dataset. Our hardware implementation reduces the storage of CCL and simplifies the HOG computation. Main CCL storage size is reduced by 20% comparing to the most advanced design under typical condition. By using TSMC 90 nm technology, the proposed design operates at 105 MHz clock rate and processes in 135 fps with the image size of 1360 × 800. The chip size is about 1 mm2 and the power consumption is close to 8 mW. Therefore, this work is resource efficient and achieves real-time requirement.
References
[1]
Mammeri, A., Boukerche, A. and Almulla, M. (2013) Design of Traffic Sign Detection, Recognition, and Transmission Systems for Smart Vehicles. IEEE Wireless Communications, 20, 36-43. https://doi.org/10.1109/MWC.2013.6704472
[2]
Gomez-Moreno, H., Maldonado-Bascon, S., Gil-Jimenez, P. and Lafuente-Arroyo, S. (2010) Goal Evaluation of Segmentation Algorithms for Traffic Sign Recognition. IEEE Transactions on Intelligent Transportation Systems, 11, 917-930.
https://doi.org/10.1109/TITS.2010.2054084
[3]
Khan, J.F., Bhuiyan, S.M.A. and Adhami, R.R. (2011) Image Segmentation and Shape Analysis for Road-Sign Detection. IEEE Transactions on Intelligent Transportation Systems, 12, 83-96. https://doi.org/10.1109/TITS.2010.2073466
[4]
Liang, M., Yuan, M., Hu, X., Li, J. and Liu, H. (2013) Traffic Sign Detection by ROI Extraction and Histogram Features-Based Recognition. The 2013 International Joint Conference on Neural Networks, Dallas, 4-9 August 2013, 1-8.
[5]
Gonzalez, A., Garrido, M., Llorca, D.F., Gavilan, M., Fernandez, J.P., Alcantarilla, P.F., Parra, I., Herranz, F., Bergasa, L.M., Sotelo, M. and Revenga de Toro, P. (2011) Automatic Traffic Signs and Panels Inspection System Using Computer Vision. IEEE Transactions on Intelligent Transportation Systems, 12, 485-499.
https://doi.org/10.1109/TITS.2010.2098029
[6]
Overett, G., Tychsen-Smith, L., Petersson, L., Pettersson, N. and Andersson, L. (2014) Creating Robust High-Throughput Traffic Sign Detectors Using Centre-Surround Hog Statistics. Machine Vision and Applications, 25, 713-726.
https://doi.org/10.1007/s00138-011-0393-1
[7]
Dalal, N. and Triggs, B. (2005) Histograms of Oriented Gradients for Human Detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, 20-25 June 2005, Vol. 1, 886-893.
https://doi.org/10.1109/CVPR.2005.177
[8]
Seo, Y.W., Lee, J., Zhang, W. and Wettergreen, D. (2015) Recognition of Highway Workzones for Reliable Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems, 16, 708-718.
[9]
Zhang, K., Sheng, Y. and Li, J. (2012) Automatic Detection of Road Traffic Signs from Natural Scene Images Based on Pixel Vector and Central Projected Shape Feature. IET Intelligent Transport Systems, 6, 282-291.
https://doi.org/10.1049/iet-its.2011.0105
[10]
Jin, J., Fu, K. and Zhang, C. (2014) Traffic Sign Recognition with Hinge Loss Trained Convolutional Neural Networks. IEEE Transactions on Intelligent Transportation Systems, 15, 1991-2000. https://doi.org/10.1109/TITS.2014.2308281
[11]
Ruta, A., Li, Y. and Liu, X. (2010) Real-Time Traffic Sign Recognition from Video by Class-Specific Discriminative Features. Pattern Recognition, 43, 416-430.
https://doi.org/10.1016/j.patcog.2009.05.018
[12]
Chen, Z., Huang, X., Ni, Z. and He, H. (2014) A GPU-Based Real-Time Traffic Sign Detection and Recognition System. IEEE Symposium on Computational Intelligence in Vehicles and Transportation Systems, Orlando, 9-12 December 2014, 1-5.
https://doi.org/10.1109/CIVTS.2014.7009470
[13]
Zaklouta, F. and Stanciulescu, B. (2014) Real-Time Traffic Sign Recognition in Three Stages. Robotics and Autonomous Systems, 62, 16-24.
https://doi.org/10.1016/j.robot.2012.07.019
[14]
Greenhalgh, J. and Mirmehdi, M. (2012) Real-Time Detection and Recognition of Road Traffic Signs. IEEE Transactions on Intelligent Transportation Systems, 13, 1498-1506. https://doi.org/10.1109/TITS.2012.2208909
[15]
Yuan, X., Hao, X., Chen, H. and Wei, X. (2014) Robust Traffic Sign Recognition Based on Color Global and Local Oriented Edge Magnitude Patterns. IEEE Transactions on Intelligent Transportation Systems, 15, 1466-1477.
https://doi.org/10.1109/TITS.2014.2298912
[16]
Yang, Y., Luo, H., Xu, H. and Wu, F. (2016) Towards Real-Time Traffic Sign Detection and Classification. IEEE Transactions on Intelligent Transportation Systems, 17, 2022-2031. https://doi.org/10.1109/TITS.2015.2482461
[17]
Liu, C., Chang, F., Chen, Z. and Liu, D. (2016) Fast Traffic Sign Recognition via High-Contrast Region Extraction and Extended Sparse Representation. IEEE Transactions on Intelligent Transportation Systems, 17, 79-92.
https://doi.org/10.1109/TITS.2015.2459594
[18]
Rosenfeld, A. and Pfaltz, J.L. (1966) Sequential Operations in Digital Picture Processing. Journal of the ACM, 13, 471-494.
https://doi.org/10.1145/321356.321357
[19]
AbuBaker, A., Qahwaji, R., Ipson, S. and Saleh, M. (2007) One Scan Connected Component Labeling Technique. IEEE International Conference on Signal Processing and Communications, Dubai, 24-27 November 2007, 1283-1286.
https://doi.org/10.1109/ICSPC.2007.4728561
[20]
Cortes, C. and Vapnik, V. (1995) Support-Vector Networks. Machine Learning, 20, 273-297. https://doi.org/10.1007/BF00994018
[21]
Houben, S., Stallkamp, J., Salmen, J., Schlipsing, M. and Igel, C. (2013) Detection of Traffic Signs in Real-World Images: The German Traffic Sign Detection Benchmark. The 2013 International Joint Conference on Neural Networks, Dallas, 4-9 August 2013, 1-8. https://doi.org/10.1109/IJCNN.2013.6706807
[22]
The OpenCV Library. http://opencv.org
[23]
Chang, C.-C. and Lin, C.-J. (2011) Libsvm: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology, 2, 27.
[24]
Yang, X.D. (1988) Design of Fast Connected Components Hardware. Computer Society Conference on Computer Vision and Pattern Recognition, Ann Arbor, 5-9 Jun 1988, 937-944. https://doi.org/10.1109/CVPR.1988.196345
[25]
Johnston, C.T. and Bailey, D.G. (2008) FPGA Implementation of a Single Pass Connected Components Algorithm. 4th IEEE International Symposium on Electronic Design, Test and Applications, Hong Kong, 23-25 January 2008, 228-231.
https://doi.org/10.1109/DELTA.2008.21
[26]
Ma, N., Bailey, D.G. and Johnston, C.T. (2008) Optimised Single Pass Connected Components Analysis. International Conference on Field-Programmable Technology, Taipei, 7-10 December 2008, 185-192.
https://doi.org/10.1109/FPT.2008.4762382
[27]
Klaiber, M.J., Bailey, D.G., Baroud, Y.O. and Simon, S. (2016) A Resource Efficient Hardware Architecture for Connected Component Analysis. IEEE Transactions on Circuits and Systems for Video Technology, 26, 1334-1349.
https://doi.org/10.1109/TCSVT.2015.2450371
[28]
Souani, C., Faiedh, H. and Besbes, K. (2014) Efficient Algorithm for Automatic Road Sign Recognition and Its Hardware Implementation. Journal of Real-Time Image Processing, 9, 79-93. https://doi.org/10.1007/s11554-013-0348-z
[29]
Waite, S. (2013) FPGA-Based Traffic Sign Recognition for Advanced Driver Assistance Systems. Journal of Transportation Technologies, 3, 1-16.
[30]
Aguirre-Dobernack, N., Guzmn-Miranda, H. and Aguirre, M.A. (2013) Implementation of a Machine Vision System for Real-Time Traffic Sign Recognition on FPGA. 39th Annual Conference of the IEEE Industrial Electronics Society, Vienna, 10-13 November 2013, 2285-2290. https://doi.org/10.1109/IECON.2013.6699487
[31]
Park, J., Kwon, J., Oh, J., Lee, S., Kim, J.Y. and Yoo, H.J. (2012) A 92mw Real-Time Traffic Sign Recognition System with Robust Illumination Adaptation and Support Vector Machine. IEEE Journal of Solid-State Circuits, 47, 2711-2723.
https://doi.org/10.1109/JSSC.2012.2211691
