In this
paper, a study and evaluation of the combination of GPS/GNSS techniques and
advanced image processing algorithms for distressed human detection,
positioning and tracking, from a fully autonomous Unmanned Aerial Vehicle
(UAV)-based rescue support system, are presented. In particular, the issue of human detection both on
terrestrial and marine environment under several illumination and background
conditions, as the human silhouette in water differs significantly from a
terrestrial one, is
addressed. A robust approach, including an adaptive distressed human detection
algorithm running every N input image frames combined with a much faster
tracking algorithm, is proposed. Real time or near-real-time distressed human
detection rates achieved, using a single, low cost day/night NIR camera mounted
onboard a fully autonomous UAV for Search and Rescue (SAR) operations. Moreover,
the generation of our own dataset, for the image processing algorithms training
is also presented. Details about both hardware and software configuration as
well as the assessment of the proposed approach performance are fully
discussed. Last, a comparison of the proposed approach to other human detection
methods used in the literature is presented.
References
[1]
Seemanthini, K. and Manjunath, S. (2018) Human Detection and Tracking Using HOG for Action Recognition. Procedia Computer Science, 132, 1317-1326.
https://doi.org/10.1016/j.procs.2018.05.048
[2]
Rakibe, R. and Patil, B. (2013) Background Subtraction Algorithm Based Human Motion Detection. International Journal of Scientific and Research Publications, 3, No. 5. https://doi.org/10.1109/ETCS.2010.440
[3]
Darwish, A. Hakami, H. and Chaczko, Z. (2015) Review of Human Motion Detection Based on Background Subtraction Techniques. International Journal of Computer Applications, 122, 1-5. https://doi.org/10.5120/21757-4988
[4]
Subudhi, B.N., Ghosh, S. and Ghosh, A. (2013) Change Detection for Moving Object Segmentation with Robust Background Construction under Wronskian Framework. Machine Vision and Applications, 24, 795-809.
https://doi.org/10.1007/s00138-012-0475-8
[5]
Benezeth, Y., Emile, B., Laurent, H. and Rosenberger, C. (2008) A Real Time Human Detection System Based on Far Infrared Vision. In: Elmoataz, A., Lezoray, O., Nouboud, F. and Mammass, D., Εds., Image and Signal Processing, ICISP 2008, Lecture Notes in Computer Science, Volume 5099, Springer, Heidelberg.
https://doi.org/10.1007/978-3-540-69905-7_9
[6]
Maier, J. and Humenberger, Μ. (2018) Movement Detection Based on Dense Optical Flow for Unmanned Aerial Vehicles. International Journal of Advanced Robotic Systems, 10. https://doi.org/10.5772/52764
[7]
Chowdhury, S., Kowsar, M. and Deb, K. (2018) Human Detection Utilizing Adaptive Background Mixture Models and Improved Histogram of Oriented Gradients. ICT Express, 4, 216-220. https://doi.org/10.1016/j.icte.2017.11.016
[8]
Lee, C., Flynn, M., Vidal, R., Reiter, A. and Hager, G. (2016) Temporal Convolutional Networks for Action Segmentation and Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 26 June-1 July 2016.
[9]
Lestari, P. and Schade, H. (2017) RGB-depth Image Based Human Detection Using Viola-Jones and Chan-Vese Active Contour Segmentation. In: Advances in Signal Processing and Intelligent Recognition Systems, Springer, Heidelberg, 285-296.
https://doi.org/10.1007/978-3-319-67934-1_25
[10]
Gonzalez, D. and Hayet, J. (2013) Fast Human Detection in RGB-D Images with Progressive SVM-Classification. In: Klette, R., Rivera, M. and Satoh, S., Eds., Image and Video Technology, PSIVT 2013, Lecture Notes in Computer Science, Volume 8333. Springer, Heidelberg.
[11]
Gade, R. and Moeslund, T.B. (2014) Thermal Cameras Applications: A Survey. Machine Vision and Applications, 25, 245-262.
https://doi.org/10.1007/s00138-013-0570-5
[12]
De Oliveria, D. and Wehrmeister, M. (2018) Using Deep Learning and Low-Cost RGB and Thermal Cameras to Detect Pedestrians in Aerial Images Captured by Multirotor UAV. Sensors, 18, 2244. https://doi.org/10.3390/s18072244
[13]
Lee, J.N. and Kwak, K.C. (2014) A Trends Analysis of Image Processing in Unmanned Aerial Vehicle. International Scholarly and Scientific Research & Innovation, 8, 261-264. https://doi.org/10.5281/zenodo.1090562
[14]
Radovic, M., Adarkwa, O. and Wang, Q. (2017) Object Recognition in Aerial Images Using Convolutional Neural Networks. Journal of Imaging, 3, 21-30.
https://doi.org/10.3390/jimaging3020021
[15]
Lygouras, E., Gasteratos, A., Tarchanidis, K. and Mitropoulos, A. (2018) ROLFER: A Fully Autonomous Aerial Rescue Support System. Microprocessors and Microsystems, 61, 32-42. https://doi.org/10.1016/j.micpro.2018.05.014
[16]
Kostavelis, I. and Gasteratos, A. (2017) Robots in Crisis Management: A Survey. 4th International Conference on Information Systems for Crisis Response and Management in Mediterranean Countries (ISCRAM-Med), Xanthi, Greece, 18-20 October 2017, 43-56. https://doi.org/10.1007/978-3-319-67633-3_4
[17]
Quan, A., Herrmann, C. and Soliman, H. (2019) Project Vulture, a Prototype for Using Drones in Search and Rescue Operations. 15th International Conference on Distributed Computing in Sensor Systems (DCOSS), Santorini, Greece, 29-31 May 2019. https://doi.org/10.1109/DCOSS.2019.00113
[18]
Andriluka, M., et al. (2010) Vision Based Victim Detection from Unmanned Aerial Vehicles. Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, 18-22 October 2010, 1740-1747.
https://doi.org/10.1109/IROS.2010.5649223
[19]
Blondel, P., Potelle, A., Pégard, C. and Lozano, R. (2014) Fast and Viewpoint Robust Human Detection for SAR Operations. 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics, Hokkaido, Japan, 27-30 October 2014.
https://doi.org/10.1109/SSRR.2014.7017675
[20]
Jingxuan, S., Boyang, L., Yifan, J. and Chih-Yung, W. (2016) A Camera-Based Target Detection and Positioning UAV System for SAR Purposes. Sensors, 16, 1778.
https://doi.org/10.3390/s16111778
[21]
Bu, F., and Gharajeh, M. (2019) Intelligent and Vision-Based Fire Detection Systems: A Survey. Image and Vision Computing, 91, Article ID: 103803.
https://doi.org/10.1016/j.imavis.2019.08.007
[22]
Bejiga, M., Zeggada, A., Nouffidj, A. and Melgani, F. (2017) A Convolutional Neural Network Approach for Assisting Avalanche Search and Rescue Operations with UAVs Imagery. Remote Sensors, 9, 100. https://doi.org/10.3390/rs9020100
[23]
Pham, H., La, H., Seifer, D. and Nguyen, L. (2018) Reinforcement Learning for UAVs Autonomous Navigation Using Function Approximation. Proceedings of the 16th IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), Philadelphia, PA, 6-8 August 2018.
https://doi.org/10.1109/SSRR.2018.8468611
[24]
Niroui, F., Zhang, K., Kashino, Z. and Nejat, G. (2019) Deep Reinforcement Learning Robot for Search and Rescue Operations: Exploration in Unknown Cluttered Environments. IEEE Robotics and Automation Letters, 4, 610-617.
https://doi.org/10.1109/LRA.2019.2891991
[25]
Sampredo, C., Rodriguez-Ramos, A., Bavle, H., Carrio, A., de la Puente, P. and Campoy, P. (2018) A Fully Autonomous Aerial Robot for Search and Rescue Applications in Indoor Environments Using Learning-based Techniques. Journal of Intelligent Robotic Systems, 95, 601-627. https://doi.org/10.1007/s10846-018-0898-1
[26]
Adawadkar, K. (2017) Python Programming-Applications and Future. International Journal of Advanced Engineering and Research Development.
http://ijaerd.com/papers/special_papers/IT032.pdf
[27]
Culjak, I., Abram, D., Pribanic, T., Dzapo, H. and Cifrek, M. (2012) A Brief Introduction to OpenCV. 2012 Proceedings of the 35th International Convention MIPRO, Opatija, Croatia, 21-25 May 2012.
[28]
Abadi, M., et al. (2016) TensorFlow: A System for Large-Scale Machine Learning. OSDI 2016, Savannah, GA, 2-4 November 2016, 265-283.
[29]
King, D. (2009) Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research, 10, 1755-1758.
[30]
Lin, T., et al. (2014) Microsoft COCO: Common Objects in Context. arXiv 2014, arXiv:1405.0312. https://doi.org/10.1007/978-3-319-10602-1_48
[31]
Huang, J., et al. (2012) Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors. arXiv 2017, arXiv:1611.10012.
https://doi.org/10.1109/CVPR.2017.351
[32]
Xia, L., Chen, C. and Aggarwal, J.K. (2011) Human Detection Using Depth Information by Kinect. CVPR 2011 Workshops, Springs, CO, 20-25 June 2011, 15-22.
https://doi.org/10.1109/CVPRW.2011.5981811
[33]
Choi, B., Mericli, C., Biswas, J. and Veloso, M. (2013) Fast Human Detection for Indoor Mobile Robots Using Depth Images. 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany, 6-10 May 2013.
https://doi.org/10.1109/ICRA.2013.6630711
[34]
Ikemura, S. and Fujiyoshi, H. (2010) Real-Time Human Detection Using Relational Depth Similarity Features. In: Kimmel, R., Klette, R. and Sugimoto, A., Eds., Computer Vision-ACCV 2010. Lecture Notes in Computer Science, Volume 6495, Springer, Heidelberg. https://doi.org/10.1007/978-3-642-19282-1_3
