In this paper, we will be looking at our efforts to find a novel solution for motion deblurring in videos. In addition, our solution has the requirement of being camera-independent. This means that the solution is fully implemented in software and is not aware of any of the characteristics of the camera. We found a solution by implementing a Convolutional Neural Network-Long Short Term Memory (CNN-LSTM) hybrid model. Our CNN-LSTM is able to deblur video without any knowledge of the camera hardware. This allows it to be implemented on any system that allows the camera to be swapped out with any camera model with any physical characteristics.
References
[1]
Zhu, Y.-F. (2010) Blur Detection for Surveillance Video Based on Heavy-Tailed Distribution. 2010 Asia Pacific Conference on Postgraduate Research in Microelectronics and Electronics, Shanghai, 22-24 September 2010, 101-105. https://doi.org/10.1109/PRIMEASIA.2010.5604950
[2]
Guo, C., Wang, Q., Dai, H.-N. and Li, P. (2022) VDN: Variant-Depth Network for Motion Deblurring. Computer Animation and Virtual Worlds, 33, e2066. https://doi.org/10.1002/cav.2066
[3]
Fang, N. and Zhan, Z.Q. (2022) High-Resolution Optical Flow and Frame-Recurrent Network for Video Super-Resolution and Deblurring. Neurocomputing, 489, 128-138. https://doi.org/10.1016/j.neucom.2022.02.067
[4]
Qi S., Jia, J.Y. and Aseem, A. (2008) High-Quality Motion Deblurring from a Single Image. ACM Transactions on Graphics, Vol. 27. https://doi.org/10.1145/1360612.1360672
[5]
Krishnan, D., Tay, T. and Fergus, R. (2011) Blind Deconvolution Using a Normalized Sparsity Measure. CVPR 2011, Colorado Springs, 20-25 June 2011, 233-240.
[6]
Li, X., Zheng, S.C. and Jia, J.Y. (2013) Unnatural L0 Sparse Representation for Natural Image Deblurring. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, 23-28 June 2013, 1107-1114. https://doi.org/10.1109/CVPR.2013.147
[7]
Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T. and Freeman, W.T. (2006) Removing Camera Shake from a Single Photograph. ACM Transactions on Graphics, 25, 787-794. https://doi.org/10.1145/1141911.1141956
[8]
Levin, A., Weiss, Y., Durand, F. and Freeman, W.T. (2009) Understanding and Evaluating Blind Deconvolution Algorithms. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, 20-25 June 2009, 1964-1971. https://doi.org/10.1109/CVPR.2009.5206815
[9]
Levin, A., Weiss, Y., Durand, F. and Freeman, W.T. (2011) Efficient Marginal Likelihood Optimization in Blind Deconvolution. CVPR 2011, Colorado Springs, 20-25 June 2011, 2657-2664. https://doi.org/10.1109/CVPR.2011.5995308
[10]
Liu, S.G., Wang, H.B., Wang, J. and Pan, C.H. (2016) Blur-Kernel Bound Estimation From Pyramid Statistics. IEEE Transactions on Circuits and Systems for Video Technology, 26, 1012-1016. https://doi.org/10.1109/TCSVT.2015.2418585
[11]
Agrawal, A., Xu, Y. and Raskar, R. (2009) Invertible Motion Blur in Video. ACM Transactions on Graphics, 28, 1-8. https://doi.org/10.1145/1576246.1531401
[12]
Qiao, C.B., Lau, R.W.H., Sheng, B., Zhang, B.X. and Wu, E.H. (2017) Temporal Coherence-Based Deblurring Using Non-Uniform Motion Optimization. IEEE Transactions on Image Processing, 26, 4991-5004.
[13]
Russell, S. and Norvig, P. (2021) Artificial Intelligence: A Modern Approach. Pearson, London.
[14]
Chen, H.Y., Teng, M.G., Shi, B.X., Wang, Y.Z. and Huang, T.J. (2022) A Residual Learning Approach to Deblur and Generate High Frame Rate Video With an Event Camera. IEEE Transactions on Multimedia, 1-14. https://doi.org/10.1109/TMM.2022.3199556
[15]
Yang, F., Xiao, L. and Yang, J.X. (2020) Video Deblurring Via 3D CNN and Fourier Accumulation Learning. 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, 04-08 May 2020, 2443-2447. https://doi.org/10.1109/ICASSP40776.2020.9054514
[16]
Matsushita, Y., Kawasaki, H., Ono, S. and Ikeuchi, K. (2014) Simultaneous Deblur and Super-Resolution Technique for Video Sequence Captured by Hand-Held Video Camera. 2014 IEEE International Conference on Image Processing, Paris, 27-30 October 2014, 4562-4566. https://doi.org/10.1109/ICIP.2014.7025925
[17]
Nah, S., Kim, T.H. and Lee, K.M. (2018) Deep Multi-Scale Convolutional Neural Network for Dynamic Scene Deblurring. Computer Vision and Pattern Recognition, 1, 1-21.
[18]
Setup Blocks Environment for AirSim. https://microsoft.github.io/AirSim/unreal_blocks/
[19]
Shouhuzhedelang (2017) Zhang Jia Jie Mountain. Unreal Engine Marketplace. https://www.unrealengine.com/marketplace/en-US/product/zhangjiajie-mountain
[20]
Sun, J., Cao, W., Xu, Z. and Ponce, J. (2015) Learning a Convolutional Neural Network for Non-Uniform Motion Blur Removal. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 07-12 June 2015, 769-777. https://doi.org/10.1109/CVPR.2015.7298677
[21]
Kim T.H. and Lee, K.M. (2014) Segmentation-Free Dynamic Scene Deblurring, Computer Vision and Pattern Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 23-28 June 2014, 2766-2773.
