A prerequisite for any system that enhances drivers’ awareness of road conditions and threatening situations is the correct sensing of the road geometry and the vehicle’s relative pose with respect to the lane despite shadows and occlusions. In this paper we propose an approach for lane segmentation and tracking that is robust to varying shadows and occlusions. The approach involves color-based clustering, the use of MSAC for outlier removal and curvature estimation, and also the tracking of lane boundaries. Lane boundaries are modeled as planar curves residing in 3D-space using an inverse perspective mapping, instead of the traditional tracking of lanes in the image space, i.e., the segmented lane boundary points are 3D points in a coordinate frame fixed to the vehicle that have a depth component and belong to a plane tangent to the vehicle’s wheels, rather than 2D points in the image space without depth information. The measurement noise and disturbances due to vehicle vibrations are reduced using an extended Kalman filter that involves a 6-DOF motion model for the vehicle, as well as measurements about the road’s banking and slope angles. Additional contributions of the paper include: (i) the comparison of textural features obtained from a bank of Gabor filters and from a GMRF model; and (ii) the experimental validation of the quadratic and cubic approximations to the clothoid model for the lane boundaries. The results show that the proposed approach performs better than the traditional gradient-based approach under different levels of difficulty caused by shadows and occlusions.
References
[1]
Peden, M.; Scurfield, R.; Sleet, D.; Mohan, D.; Hyder, A.A.; Jarawan, E.; Mathers, C. World Report on Road Traffic Injury Prevention; World Health Organization: Geneva, Switzerland, 2004.
[2]
Baskar, L.; De Schutter, B.; Hellendoorn, J.; Papp, Z. Traffic control and intelligent vehicle highway systems: A survey. IET Intell. Transp. Syst. 2011, 5, 38–52.
[3]
Alvarez, S.; Sotelo, M.A.; Oca？a, M.; Llorca, D.; Parra, I.; Bergasa, L. Perception advances in outdoor vehicle detection for automatic cruise control. Robotica 2010, 28, 765–779.
[4]
Gustafsson, F. Automotive safety systems. IEEE Signal Process. Mag. 2009, 26, 32–47.
[5]
Jimenez-Pinto, J.; Torres-Torriti, M. Face salient points and eyes tracking for robust drowsiness detection. Robotica 2012, 30, 731–741.
[6]
Lin, C.T.; Wu, R.C.; Liang, S.F.; Chao, W.H.; Chen, Y.J.; Jung, T.P. EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE Trans. Circ. Syst. I 2005, 52, 2726–2738.
[7]
Estrin, D.; Govindan, R.; Heidemann, J.; Kumar, S. Next Century Challenges: Scalable Coordination in Sensor Networks. Proceedings of the International Conference on Mobile Computing and Networking, Seattle, WA, USA, 15–19 August 1999; pp. 263–270.
[8]
De la Escalera, A.; Armingol, J.M. Vehicle detection and tracking for visual understanding of road environments. Robotica 2010, 28, 847–860.
[9]
Sun, Z.; Bebis, G.; Miller, R. On-road vehicle detection: A review. IEEE Trans. Patt. Anal. Mach. Intell. 2006, 28, 694–711.
[10]
Bar Hillel, A.; Lerner, R.; Levi, D.; Raz, G. Recent progress in road and lane detection: A survey. Mach. Vision Appl. 2012, doi:10.1007/s00138-011-0404-2.
[11]
Guo, C.; Mita, S.; McAllester, D. Robust road detection and tracking in challenging scenarios based on Markov random fields with unsupervised learning. IEEE Trans. Intell. Transp. Syst. 2012, 13, 1338–1354.
[12]
McCall, J.; Trivedi, M. Video-based lane estimation and tracking for driver assistance: Survey, system, and evaluation. IEEE Trans. Intell. Transp. Syst. 2006, 7, 20–37.
[13]
Sarholz, F.; Klappstein, J.; Diewald, F.; Dickmann, J.; Radig, B. Evaluation of Different Quality Functions for Road Course Estimation Using Imaging Radar. Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany, 5–9 June 2011; pp. 882–887.
[14]
Miyake, Y.; Natsume, K.; Hoshino, K. Road-Shape Recognition Using On-vehicle Millimeter-Wave Radar. Proceedings of the IEEE Intelligent Vehicles Symposium, Istanbul, Turkey, 13–15 June 2007; pp. 75–80.
[15]
Ma, B.; Lakshmanan, S.; Hero, A.O.I. Simultaneous detection of lane and pavement boundaries using model-based multisensor fusion. IEEE Trans. Intell. Transp. Syst. 2000, 1, 135–147.
[16]
Han, J.; Kim, D.; Lee, M.; Sunwoo, M. Enhanced road boundary and obstacle detection using a downward-looking LIDAR sensor. IEEE Trans. Veh. Technol. 2012, 61, 971–985.
[17]
Wijesoma, W.; Kodagoda, K.; Balasuriya, A. Road-boundary detection and tracking using ladar sensing. IEEE Trans. Robot. Autom. 2004, 20, 456–464.
[18]
Citroen's Lane Departure Warning System. Available online: http://www.piecescitroensport.citroen.com/CWW/en-US/TECHNOLOGIES/SECURITY/AFIL/ (accessed on 6 March 2013).
[19]
Peugeot's Lane Departure Warning System. Available online: http://www.peugeot.com/en/innovation/safety/prevent/ldws-lane-departure-warning-system.aspx (accessed on 6 March 2013).
[20]
Hsiao, P.Y.; Yeh, C.W.; Huang, S.S.; Fu, L.C. A portable vision-based real-time lane departure warning system: Day and night. IEEE Trans. Veh. Technol. 2009, 58, 2089–2094.
[21]
Lee, J.W. A machine vision system for lane-departure detection. Comput. Vision Image Underst. 2002, 86, 52–78.
[22]
Otsuka, Y.; Muramatsu, S.; Takenaga, H.; Kobayashi, Y.; Monj, T. Multitype Lane Markers Recognition Using Local Edge Direction. Proceedings of the IEEE Intelligent Vehicle Symposium, Versailles, France, 17–21 June 2002; pp. 604–609.
[23]
Kluge, K.; Lakshmanan, S. A Deformable-Template Approach to Lane Detection. Proceedings of the Intelligent Vehicles Symposium, Detroit, MI, USA, 25–26 September 1995; pp. 54–59.
[24]
Kosecka, J.; Blasi, R.; Taylor, C.; Malik, J. A Comparative Study of Vision-Based Lateral Control Strategies for Autonomous Highway Driving. Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, 16–20 May 1998; pp. 1903–1908.
[25]
Zhang, J.; Nagel, H.H. Texture-Based Segmentation of Road Images. Proceedings of the Intelligent Vehicles Symposium, Paris, France, 24–26 October 1994; pp. 260–265.
[26]
Thorpe, C.; Hebert, M.; Kanade, T.; Shafer, S. Vision and navigation for the Carnegie-Mellon Navlab. IEEE Trans. Patt. Anal. Mach. Intell. 1988, 10, 362–373.
[27]
Fernandez-Maloigne, C.; Bonnet, W. Texture and Neural Network for Road Segmentation. Proceedings of the Intelligent Vehicles Symposium, Detroit, MI, USA, 25–26 September 1995; pp. 344–349.
[28]
Cheng, H.Y.; Yu, C.C.; Tseng, C.C.; Fan, K.C.; Hwang, J.N.; Jeng, B.S. Environment classification and hierarchical lane detection for structured and unstructured roads. IET Comput. Vision 2010, 4, 37–49.
[29]
Jeong, P.; Nedevschi, S. Efficient and robust classification method using combined feature vector for lane detection. IEEE Trans. Circ. Syst. Video Technol. 2005, 15, 528–537.
[30]
Wang, J.; Schroedl, S.; Mezger, K.; Ortloff, R.; Joos, A.; Passegger, T. Lane keeping based on location technology. IEEE Trans. Intell. Transp. Syst. 2005, 6, 351–356.
[31]
Ruyi, J.; Klette, R.; Tobi, V.; Shigang, W. Lane detection and tracking using a new lane model and distance transform. Mach. Vision Appl. 2011, 22, 721–737.
[32]
Fischler, M.A.; Bolles, R.C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. ACM Commun. 1981, 24, 381–395.
[33]
Li, Q.; Zheng, N.; Cheng, H. Springrobot: A prototype autonomous vehicle and its algorithms for lane detection. IEEE Trans. Intell. Transp. Syst. 2004, 5, 300–308.
[34]
Wang, J.; Schroedl, S.; Mezger, K.; Ortloff, R.; Joos, A.; Passegger, T. Lane keeping based on location technology. IEEE Trans. Intell. Transp. Syst. 2005, 6, 351–356.
[35]
Kim, Z. Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 2008, 9, 16–26.
[36]
Danescu, R.; Nedevschi, S. Probabilistic lane tracking in difficult road scenarios using stereovision. IEEE Trans. Intell. Transp. Syst. 2009, 10, 272–282.
[37]
Amditis, A.; Bimpas, M.; Thomaidis, G.; Tsogas, M.; Netto, M.; Mammar, S.; Beutner, A.; M？ Andhler, N.; Wirthgen, T.; Zipser, S.; et al. A Situation-adaptive lane-keeping support system: Overview of the SAFELANE approach. IEEE Trans. Intell. Transp. Syst. 2010, 11, 617–629.
[38]
Cheng, H.Y.; Jeng, B.S.; Tseng, P.T.; Fan, K.C. Lane detection with moving vehicles in the traffic scenes. IEEE Trans. Intell. Transp. Syst. 2006, 7, 571–582.
[39]
Cheng, Y. Mean shift, mode seeking, and clustering. IEEE Trans. Patt. Anal. Mach. Intell. 1995, 17, 790–799.
[40]
Comaniciu, D.; Meer, P. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Patt. Anal. Mach. Intell. 2002, 24, 603–619.
[41]
Duda, R.O.; Hart, P.E.; Stork, D.G. Pattern Classification, 2nd ed. ed.; Wiley-Interscience: New York, NY, USA, 2000.
[42]
Torres-Torriti, M.; Jouan, A. Gabor vs. GMRF Features for SAR Imagery Classification. Proceedings of the 2001 International Conference on Image Processing, Thessaloniki, Greece, 7–10 October 2001; pp. 1043–1046.
[43]
Freeman, W.; Adelson, E. The design and use of steerable filters. IEEE Trans. Patt. Anal. Mach. Intell. 1991, 13, 891–906.
[44]
Russ, J.C. The Image Processing Handbook, 5th ed. ed.; Taylor and Francis, Inc.: London, UK, 2006.
[45]
Aly, M. Caltech Lanes Dataset. Available online: http://vision.caltech.edu/malaa/datasets/caltech-lanes/ (accessed on 6 March 2013).
[46]
Shapiro, L.G.; Stockman, G.C.; Shapiro, L.G.; Stockman, G. Computer Vision; Prentice Hall: Upper Saddle River, NJ, USA, 2001.
[47]
Nedevschi, S.; Schmidt, R.; Graf, T.; Danescu, R.; Frentiu, D.; Marita, T.; Oniga, F.; Pocol, C. 3D Lane Detection System Based on Stereovision. Proceedings of the 7th International IEEE Conference on Intelligent Transportation Systems, 4–6 October 2004; Washington, DC, USA; pp. 161–166.
[48]
Jha, M.K.; Schonfeld, P.; Jong, J.C.; Kim, E. Intelligent Road Design. In Advances in Transport; WIT Press: Billerica, MA, USA, 2006.
[49]
Walton, D.J.; Meek, D.S. Computer-aided design for horizontal alignment. J. Transp. Eng. 1989, 115, 411–424.
[50]
Zhang, Z. Parameter estimation techniques: A tutorial with application to conic fitting. Image Vision Comput. 1997, 15, 59–76.
[51]
Torr, P.; Zisserman, A. MLESAC: A new robust estimator with application to estimating image geometry. Comput. Vision Image Underst. 2000, 78, 138–156.
[52]
Tapia-Espinoza, R.; Torres-Torriti, M. A Comparison of Gradient Versus Color and Texture Analysis for Lane Detection and Tracking. Proceedings of the 6th Latin American Robotics Symposium, Valparaiso, Chile, 29–30 October 2009; pp. 1–6.
[53]
Ristic, B.; Arulampalam, S.; Gordon, N. Beyond the Kalman Filter: Particle Filters for Tracking Applications; Artech House: Boston, MA, USA, 2004.
[54]
Peralta-Cabezas, J.L.; Torres-Torriti, M.; Guarini-Hermann, M. A comparison of Bayesian prediction techniques for mobile robot trajectory tracking. Robotica 2008, 26, 571–585.
[55]
Rajamani, R. Vehicle Dynamics and Control, 2nd ed. ed.; Springer: Berlin, Germany, 2012.
[56]
Jazar, R.N. Vehicle Dynamics: Theory and Application; Springer: Berlin, Germany, 2008.
[57]
Mammar, S.; Glaser, S.; Netto, M. Time to line crossing for lane departure avoidance: A theoretical study and an experimental setting. IEEE Trans. Intell. Transp. Syst. 2006, 7, 226–241.
[58]
Li, M.; Staunton, R.C. Optimum Gabor filter design and local binary patterns for texture segmentation. Patt. Recogn. Lett. 2008, 29, 664–672.
[59]
Young, I.T.; Gerbrands, J.J.; Van Vliet, L.J. Fundamentals of Image Processing; Delft University of Technology: Delft, The Netherlands, 1998.
[60]
Raz, R. On the Complexity of Matrix Product. SIAM J. Comput. 2012, 32, 1356–1369.
[61]
Press, W.H.; Teukolsky, S.A.; Vetterling, W.T.; Flannery, B.P. Numerical Recipes 3rd Edition: The Art of Scientific Computing; Cambridge University Press: New York, NY, USA, 2007.
[62]
Georgescu, B.; Shimshoni, I.; Meer, P. Mean Shift Based Clustering in High Dimensions: A Texture Classification Example. Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France, 13–16 October 2003; pp. 456–463.
[63]
Choi, S.; Kim, T.; Yu, W. Performance Evaluation of RANSAC Family. Proceedings of 2009 British Machine Vision Conference, London, UK, 7–10 September 2009; pp. 1–12.
[64]
Raguram, R.; Frahm, J.M.; Pollefeys, M. A comparative analysis of RANSAC techniques leading to adaptive real-time random sample consensus. Lect. Note. Comput. Sci. 2008, 5303, 500–513.
[65]
Bonato, V.; Marques, E.; Constantinides, G.A. A Floating-point extended Kalman filter implementation for autonomous mobile robots. J. Signal Process. Syst. 2009, 56, 41–50.
[66]
Pnevmatikakis, E.A.; Rad, K.R.; Huggins, J.; Paninski, L. Fast Kalman filtering and forward-backward smoothing via a low-rank perturbative approach. J. Comput. Graph. Stat. 2013. in press.
