The objective of this investigation was to develop and investigate methods for point cloud generation by image matching using aerial image data collected by quadrocopter type micro unmanned aerial vehicle (UAV) imaging systems. Automatic generation of high-quality, dense point clouds from digital images by image matching is a recent, cutting-edge step forward in digital photogrammetric technology. The major components of the system for point cloud generation are a UAV imaging system, an image data collection process using high image overlaps, and post-processing with image orientation and point cloud generation. Two post-processing approaches were developed: one of the methods is based on Bae Systems’ SOCET SET classical commercial photogrammetric software and another is built using Microsoft？’s Photosynth？ service available in the Internet. Empirical testing was carried out in two test areas. Photosynth processing showed that it is possible to orient the images and generate point clouds fully automatically without any a priori orientation information or interactive work. The photogrammetric processing line provided dense and accurate point clouds that followed the theoretical principles of photogrammetry, but also some artifacts were detected. The point clouds from the Photosynth processing were sparser and noisier, which is to a large extent due to the fact that the method is not optimized for dense point cloud generation. Careful photogrammetric processing with self-calibration is required to achieve the highest accuracy. Our results demonstrate the high performance potential of the approach and that with rigorous processing it is possible to reach results that are consistent with theory. We also point out several further research topics. Based on theoretical and empirical results, we give recommendations for properties of imaging sensor, data collection and processing of UAV image data to ensure accurate point cloud generation.
References
[1]
Everaerts, J. NEWPLATFORMS—Unconventional Platforms (Unmanned Aircraft Systems) for Remote Sensing. Official Publication No. 56;; Gopher: Amsterdam, The Netherlands, 2009.
[2]
Eisenbeiss, H. UAV Photogrammetry. DISS. ETH No. 18515;; Institute of Geodesy and Photogrammetry (IGP), ETH Zurich: Zurich, Switzerland, 2009.
[3]
Lelong, C.C.D.; Burger, P.; Jubelin, G.; Roux, B.; Labbé, S.; Baret, F. Assessment of unmanned aerial vehicles imagery for quantitative monitoring of wheat crop in small plots. Sensors 2008, 8, 3557–3585, doi:10.3390/s8053557.
[4]
Berni, J.A.; Zarco-Tejada, P.J.; Suárez, L.; Fereres, E. Thermal and narrowband multispectral remote sensing for vegetation monitoring from an unmanned aerial vehicle. IEEE Trans. Geosci. Remote Sens 2009, 47, 722–738, doi:10.1109/TGRS.2008.2010457.
[5]
Hunt, E.R., Jr.; Hively, W.D.; Fujikawa, S.J.; Linden, D.S.; Daughtry, C.S.T.; McCarty, G.W. Acquisition of NIR-green-blue digital photographs from unmanned aircraft for crop monitoring. Remote Sens 2010, 2, 290–305, doi:10.3390/rs2010290.
[6]
Bandea, H.; Chiabrando, F.; Giulio Tonolo, F.; Marenchino, D. Mapping of archeological areas using a low-cost UAV the augusta bagiennorum test site. Proceedings of the XXI International CIPA Symposium, Athens, Greece, 1–6 September 2007.
[7]
Casbeer, D.W.; Kingston, D.B.; Beard, R.W.; Mclain, T.W.; Li, S.-M.; Mehra, R. Cooperative forest fire surveillance using a team of small unmanned air vehicles. Int. J. Syst. Sci 2006, 37, 351–360, doi:10.1080/00207720500438480.
[8]
Gutjahr, K.; Hafner, P.; Ofner, M.; L？ngaue, K.; Wieser, M.; Kühtreiber, N. Performance of INS/GNSS integration methods in context of a near real-time airborne mapping platform. Proceedings of the International Calibration and Orientation Workshop EuroCOW 2010, Castelldefels, Spain, 10–12 February 2010. Official EuroSDR publication No. 57.
[9]
Zhou, G. Near real-time orthorectification and mosaic of small UAV video flow for time-critical event response. IEEE Trans. Geosci. Remote Sens 2009, 47, 739–747, doi:10.1109/TGRS.2008.2006505.
[10]
Zhou, G. Geo-referencing of video flow from small low-cost civilian UAV. IEEE Trans. Autom. Sci. Eng 2010, 7, 156–166, doi:10.1109/TASE.2008.2010948.
[11]
Heintz, F.; Rudol, P.; Doherty, P. From images to traffic behaviours—A UAV tracking and monitoring application. Proceedings of the 10th International Conference on Information Fusion, Quebec, QC, Canada, 9–12 July 2007.
[12]
Haala, N. Comeback of digital image matching. In Photogrammetric Week ’09; Fritch, D., Ed.; Wichmann Verlag: Heidelberg, Germany, 2009; pp. 289–301.
[13]
Gülch, E. Advaced matching techniques for high precision surface and terrain models. In Photogrammetric Week ’09; Fritch, D., Ed.; Wichmann Verlag: Heidelberg, Germany, 2009; pp. 303–315.
[14]
Haala, N.; Hastedt, H.; Wolf, K.; Ressl, C.; Baltrusch, S. Digital photogrammetric camera evaluation—Generation of digital elevation models. Photogramm. Fernerkund. Geoinf 2010, 2, 99–115.
[15]
Leberl, F.; Irschara, A.; Pock, T.; Meixner, P.; Gruber, M.; Scholz, S.; Wiechert, A. Point clouds: Lidar versus 3D vision. Photogramm. Eng. Remote Sens 2010, 76, 1123–1134, doi:10.14358/PERS.76.10.1123.
[16]
Gehrke, S.; Morin, K.; Downey, M.; Bowhrer, N.; Fuchs, T. Semi-global matching: An alternative to LIDAR for DSM generation? Proceedings of the 2010 Canadian Geomatics Conference and Symposium of Commission I, Calgary, AB, Canada, 15–18 June 2010.
[17]
Nagai, M.; Chen, T.; Shibasaki, R.; Kumgai, H.; Ahmed, A. UAV-borne 3-D mapping system by multisensory integration. IEEE Trans. Geosci. Remote Sens 2009, 47, 701–708, doi:10.1109/TGRS.2008.2010314.
[18]
Jaakkola, A.; Hyypp？, J.; Kukko, A.; Yu, X.; Kaartinen, H.; Lehtom？ki, M.; Lin, Y. A low-cost multi-sensoral mobile mapping system and its feasibility for tree measurements. ISPRS J. Photogramm. Remote Sens 2010, 65, 514–522, doi:10.1016/j.isprsjprs.2010.08.002.
[19]
Hakala, T.; Suomalainen, J.; Peltoniemi, J.I. Acquisition of bidirectional reflectance factor dataset using a micro unmanned aerial vehicle and a consumer camera. Remote Sens 2010, 2, 819–832, doi:10.3390/rs2030819.
[20]
Microdrones Gmbh, Available online: http://www.microdrones.com (accessed on 21 November 2011).
[21]
Ricoh GR 3, Available online: http://www.ricoh.com/r_dc/gr/gr_digital3/ (accessed on 21 November 2011).
[22]
Panasonic Lumix GF1, Available online: http://panasonic.net/avc/lumix/systemcamera/gms/gf1/specifications.html (accessed on 21 November 2011).
[23]
DCRAW, Available online: http://www.cybercom.net/~dcoffin/dcraw/ (accessed on 21 November 2011).
[24]
Fraser, C.S. Digital camera self-calibration. ISPRS J. Photogramm. Remote Sens 1997, 52, 149–159, doi:10.1016/S0924-2716(97)00005-1.
[25]
Kraus, K. Photogrammetry, Volume 1: Fundamentals and Standard Processes; Fred. Dümmlers Verlag: Bonn, Germany, 1993.
[26]
Kraus, K. Photogrammetry, Volume 2: Advanced Methods and Applications; Fred. Dümmlers Verlag: Bonn, Germany, 1997.
[27]
Honkavaara, E. Calibrating Digital Photogrammetric Airborne Imaging Systems Using a Test FieldPh.D. Dissertation. Helsinki University of Technology, Espoo, Finland, 2008.
[28]
Cramer, M. Performance of GPS/inertial solutions in photogrammetry. In Photogrammetric Week ’01; Fritch, D., Spiller, R., Eds.; Wichmann Verlag: Heidelberg, Germany, 2001; pp. 29–62.
[29]
Heipke, C.; Jacobsen, K.; Wegmann, H. Analysis of the results of the OEEPE test “Integrated Sensor Orientation”. In OEEPE Official Publication, No 43; Heipke, C., Jacobsen, K., Wegmann, H., Eds.; Werbedruck Schreckhase: Spangenberg, Germany, 2002.
[30]
Büyüksalih, G.; Li, Z. Practical experiences with automatic aerial triangulation using different software packages. Photogramm. Rec 2005, 18, 131–155.
[31]
Jacobsen, K.; Cramer, M.; Ladst？dter, R.; Ressl, C.; Spreckels, V. DGPF-Project: Evaluation of digital photogrammetric camera systems geometric performance. Photogramm. Fernerkund. Geoinf 2010, 2, 83–97.
[32]
Chiang, K.-W.; Chang, H.-W.; Li, C.-Y.; Huang, Y.-W. An artificial neural network embedded position and orientation determination algorithm for low cost MEMS INS/GPS integrated sensors. Sensors 2009, 9, 2586–2610, doi:10.3390/s90402586. 22574034
[33]
El-Sheimy, N.; Niu, X. The Promise of MEMS to the Navigation Community, Available online: http://www.insidegnss.com/auto/IG0307el-sheimyFinal.pdf (accessed on 21 November 2011).
[34]
Se, S.; Firoozfam, P.; Goldstein, N.; Dutkiewicz, M.; Pace, P. Automated UAV-based video exploitation for mapping and surveillance. Proceedings of the 2010 Canadian Geomatics Conference and Symposium of Commission I, Calgary, AB, Canada, 15–18 June 2010.
[35]
Tsai, M.L.; Chiang, K.W.; Huang, Y.W.; Lin, Y.S.; Tsai, J.S.; Lo, C.F.; Lin, Y.S.; Wu, C.H. The development of a direct georeferencing ready UAV based photogrammetry platform. Proceedings of the 2010 Canadian Geomatics Conference and Symposium of Commission I, Calgary, AB, Canada, 15–18 June 2010.
[36]
Agarwal, S.; Snavely, N.; Simon, I.; Seitz, S.M.; Szeliski, R. Building Rome in a day. Proceedings of International Conference on Computer Vision, Kyoto, Japan, 27 September–4 October 2009.
[37]
Frahm, J.-M.; Pollefeys, M.; Lazebnik, S.; Gallup, D.; Clipp, B.; Raguram, R.; Wu, C.; Zach, C.; Johnson, T. Fast robust large-scale mapping from video and internet photo collections. ISPRS J. Photogramm. Remote Sens 2010, 65, 538–549, doi:10.1016/j.isprsjprs.2010.08.009.
[38]
Available online: http://photosynth.net/ (accessed on 21 November 2011).
[39]
Available online: http://www.vexcel.com/geospatial/geosynth/ (accessed on 21 November 2011).
[40]
Lowe, D. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis 2004, 60, 91–110, doi:10.1023/B:VISI.0000029664.99615.94.
[41]
Lingua, A.; Marenchino, D.; Nex, F. Performance analysis of the SIFT operator for automatic feature extraction and matching in photogrammetric applications. Sensors 2009, 9, 3745–3766, doi:10.3390/s90503745. 22412336
[42]
Bay, H.; Ess, A.; Tuytelaars, T.; van Gool, L. SURF: Speeded up robust features. CVIU 2008, 110, 346–359.
[43]
Barazzetti, L.; Remondino, F.; Scaioni, M.; Brumana, R. Fully automatic UAV image-based sensor orientation. Proceedings of the 2010 Canadian Geomatics Conference and Symposium of Commission I, Calgary, AB, Canada, 15–18 June 2010.
[44]
Irschara, A.; Kaufmann, V.; Klopschitz, M.; Bischof, H.; Leberl, F. Towards fully automatic photogrammetric reconstruction using digital images taken from UAVs. Proceedings of the ISPRS TC VII Symposium—100 Years ISPRS, Vienna, Austria, 5–7 July 2010.
[45]
F？rstner, W. Matching strategies for point transfer. In Photogrammetric Week ’95; Fritch, D., Hobbie, D., Eds.; Wichmann Verlag: Heidelberg, Germany, 1995; pp. 173–183.
[46]
Heipke, C. Overview on image matching techniques. Proceedings of the OEEPE-Workshop on Application of Digital Photogrammetric Workstations, Lausanne, Switzerland, 4–6 March 1996; pp. 173–189.
DeVenecia, K.; Walker, S.; Zhang, B. New approaches to generating and processing high resolution elevation data with imagery. In Photogrammetric Week ’07; Fritch, D., Ed.; Wichmann Verlag: Heidelberg, Germany, 2009; pp. 297–308.
[49]
Zhang, B.; Miller, S.; DeVenecia, K. Automatic terrain extraction using multiple image pair and back matching. Proceedings of the ASPRS 2006 Annual Conference, Reno, Nevada, 1–5 May 2006.
[50]
Baltsavias, E.; Gruen, A.; Zhang, L.; Waser, L.T. High-quality image matching and automated generation of 3D tree models. Int. J. Remote Sens 2008, 29, 1243–1259, doi:10.1080/01431160701736513.
[51]
iWitness, Available online: http://www.iwitnessphoto.com/ (accessed on 21 November 2011).
[52]
Synth exporter, CodePlex Open Source Community, Available online: http://synthexport.codeplex.com/ (accessed on 21 November 2011).
[53]
Walker, S. New features in SOCET SET？. In Photogrammetric Week ’07; Fritch, D., Ed.; Wichmann Verlag: Heidelberg, Germany, 2007; pp. 35–40.
[54]
Honkavaara, E.; Markelin, L.; Rosnell, T.; Nurminen, K. Influence of solar elevation in radiometric and geometric performance of multispectral photogrammetry. ISPRS J. Photogramm. Remote Sens 2012, 67, 13–26, doi:10.1016/j.isprsjprs.2011.10.001.
[55]
Miller, S. Photogrammetric products. In ASPRS Manual of Photogrammetry, 5th ed; McGlone, J.C., Mikhail, E., Bethel, J., Eds.; American Society for Photogrammetry and Remote Sensing: Bethesda, MD, USA, 2004; pp. 983–1013.
[56]
Honkavaara, E.; Peltoniemi, J.; Ahokas, E.; Kuittinen, R.; Hyypp？, J.; Jaakkola, J.; Kaartinen, H.; Markelin, L.; Nurminen, K.; Suomalainen, J. A permanent test field for digital photogrammetric systems. Photogramm. Eng. Remote Sens 2008, 74, 95–106.
[57]
Available online: http://www.microsoft.com/ultracam/en-us/default.aspx (accessed on 21 November 2011).
[58]
Ebner, H. Theoretical accuracy models for block triangulation. BUL 1972, 40, 214–221.
[59]
F？rstner, W. The reliability of block triangulation. Photogramm. Eng. Remote Sens 1985, 51, 1137–1149.
[60]
Burman, H. Calibration and Orientation of Airborne Image and Laser Scanner Data Using GPS and INSPh.D. Dissertation. Royal Institute of Technology, Stockholm, Sweden, 2000.
