In recent years, new architectures and technologies have been proposed for Vehicular Ad Hoc networks (VANETs). Due to the cost and complexity of deploying such networks, most of these proposals rely on simulation. However, we find that most of the experiments made to validate these proposals tend to overlook the most important and representative factors. Moreover, the scenarios simulated tend to be very simplistic (highways or Manhattan-based layouts), which could seriously affect the validity of the obtained results. In this paper, we present a statistical analysis based on the 2k factorial methodology to determine the most representative factors affecting traffic safety applications under real roadmaps. Our purpose is to determine which are the key factors affecting Warning Message Dissemination in order to concentrate research tests on such parameters, thus avoiding unnecessary simulations and reducing the amount of simulation time required. Simulation results show that the key factors affecting warning messages delivery are the density of vehicles and the roadmap used. Based on this statistical analysis, we consider that VANET researchers must evaluate the benefits of their proposals using different vehicle densities and city scenarios, to obtain a broad perspective on the effectiveness of their solution. Finally, since city maps can be quite heterogeneous, we propose a roadmap profile classification to further reduce the number of cities evaluated.
Gramaglia, M.; Bernardos, C.J.; Calderon, M. Virtual induction loops based on cooperative vehicular communications. Sensors 2013, 13, 1467–1476.
[3]
Rahim, A.; Khan, Z.; Muhaya, F.T.B.; Sher, M.; Kim, T.H. Sensor based framework for secure multimedia communication in VANET. Sensors 2010, 10, 10146–10154.
[4]
Martinez, F.J.; Fogue, M.; Toh, C.K.; Cano, J.C.; Calafate, C.T.; Manzoni, P. Computer simulations of VANETs using realistic city topologies. Wirel. Pers. Commun. 2012, 69, 639–663.
[5]
IEEE 802.11 Working Group. IEEE Standard for Information Technology–Telecommunications and Information Exchange between Systems–Local and Metropolitan Area Networks–Specific Requirements–Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment 6: Wireless Access in Vehicular Environments; The Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2010.
[6]
Jain, R. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modelling; John Wiley & Sons: Hoboken, NJ, USA, 1991.
[7]
Zuo, J.; Wang, Y.; Liu, Y.; Zhang, Y. Performance Evaluation of Routing Protocol in VANET with Vehicle-Node Density. Proceedings of the 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM 2010), Chengdu, China, 23–25 September 2010; pp. 1–4.
[8]
Giordano, E.; Frank, R.; Ghosh, A.; Pau, G.; Gerla, M. Two Ray or not Two Ray this is the Price to Pay. Proceedings of the IEEE 6th International Conference on Mobile Adhoc and Sensor Systems (MASS'09), Wuhan/Beijing, China, 23–25 September 2009; pp. 603–608.
[9]
Khorashadi, B.; Chen, A.; Ghosal, D.; Chuah, C.N.; Zhang, M. Impact of Transmission Power on the Performance of UDP in Vehicular Ad Hoc Networks. Proceedings of the IEEE International Conference on Communications (ICC'07), Glasgow, Scotland, UK, 24–28 June 2007; pp. 3698–3703.
[10]
Cenerario, N.; Delot, T.; Ilarri, S. A content-based dissemination protocol for VANETs: Exploiting the encounter probability. IEEE Trans. Intell. Transp. Syst. 2011, 12, 771–782.
[11]
Sahoo, J.; Wu, E.K.; Sahu, P.; Gerla, M. Binary-partition-assisted MAC-layer broadcast for emergency message dissemination in VANETs. IEEE Trans. Intell. Transp. Syst. 2011, 12, 757–770.
[12]
Costa, P.; Frey, D.; Migliavacca, M.; Mottola, L. Towards Lightweight Information Dissemination in Inter-Vehicular Networks. Proceedings of the 3rd International Workshop on Vehicular Ad Hoc Networks (VANET '06), Los Angeles, CA, USA, 29 September 2006; pp. 20–29.
[13]
Viriyasitavat, W.; Bai, F.; Tonguz, O. UV-CAST: An Urban Vehicular Broadcast Protocol. Proceedings of the IEEE Vehicular Networking Conference (VNC), Jersey City, NJ, USA, 13–15 December 2010; pp. 25–32.
[14]
Liu, C.; Chigan, C. RPB-MD: Providing robust message dissemination for vehicular Ad Hoc networks. Ad Hoc Netw. 2012, 10, 497–511.
[15]
Gupta, R.A.; Agarwal, A.K.; Chow, M.Y.; Wang, W. Performance Assessment of Data and Time-Sensitive Wireless Distributed Networked-Control-Systems in Presence of Information Security. Proceedings of the IEEE Military Communications Conference (MILCOM 2007), Orlando, FL, USA, 29–31 October 2007; pp. 1–7.
[16]
Liu, C.; MacGregor, M.H.; Harms, J. Improving Multipath Routing Performance in WSNs by Tuning IEEE 802.11 Parameters. Proceedings of the 6th ACM International Symposium on Mobility Management and Wireless Access (MobiWac '08), Vancouver, BC, Canada, 30–31 October 2008; ACM: New York, NY, USA, 2008; pp. 142–146.
[17]
Vaz de Melo, P.O.; da Cunha, F.D.; Almeida, J.M.; Loureiro, A.A.; Mini, R.A. The Problem of Cooperation among Different Wireless Sensor Networks. Proceedings of the 11th International Symposium on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM '08), Vancouver, BC, Canada, 27–31 October 2008; ACM: New York, NY, USA, 2008; pp. 86–91.
[18]
Perkins, D.; Hughes, H.; Owen, C. Factors Affecting the Performance of Ad Hoc Networks. Proceedings of the IEEE International Conference on Communications (ICC 2002), New York, NY, USA, 28 April–2 May 2002; pp. 2048–2052.
[19]
Perkins, D.; Hughes, H. Investigating the performance of TCP in mobile Ad Hoc networks. Comput. Commun. 2002, 25, 1132–1139.
[20]
Fogue, M.; Garrido, P.; Martinez, F.J.; Cano, J.C.; Calafate, C.T.; Manzoni, P. Analysis of the Most Representative Factors Affecting Warning Message Dissemination in VANETs under Real Roadmaps. Proceedings of the 19th annual meeting of the IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), Singapore, 25–27 July 2011; pp. 197–204.
[21]
Fogue, M.; Garrido, P.; Martinez, F.J.; Cano, J.C.; Calafate, C.T.; Manzoni, P. Evaluating the impact of a novel message dissemination scheme for Vehicular Networks using real maps. Transp. Res. Part C Emerg. Technol. 2012, 25, 61–80.
[22]
Sanguesa, J.A.; Fogue, M.; Garrido, P.; Martinez, F.J.; Cano, J.C.; Calafate, C.T.; Manzoni, P. An infrastructureless approach to estimate vehicular density in urban environments. Sensors 2013, 13, 2399–2418.
[23]
Jiang, D.; Chen, Q.; Delgrossi, L. Communication Density: A Channel Load Metric for Vehicular Communications Research. Proceedings of the IEEE International Conference on Mobile Ad Hoc and Sensor Systems (MASS 2007), Pisa, Italy, 8–11 October 2007; pp. 1–8.
[24]
Eichler, S. Performance Evaluation of the IEEE 802.11p WAVE Communication Standard. Proceedings of the Vehicular Technology Conference (VTC-2007 Fall), Baltimore MD, USA, 30 September–3 October 2007.
[25]
Jiang, D.; Chen, Q.; Delgrossi, L. Optimal Data Rate Selection for Vehicle Safety Communications. Proceedings of the Fifth ACM International Workshop on VehiculAr Inter-NETworking (VANET'08), San Francisco, CA, USA, 15 September 2008; ACM: New York, NY, USA, 2008; pp. 30–38.
[26]
Liu, C.; Chigan, C. RPB-MD: Providing robust message dissemination for vehicular Ad Hoc networks. Ad Hoc Netw. 2012, 10, 497–511.
[27]
Tseng, Y.C.; Ni, S.Y.; Chen, Y.S.; Sheu, J.P. The broadcast storm problem in a mobile Ad Hoc network. Wirel. Netw. 2002, 8, 153–167.
[28]
Wisitpongphan, N.; Tonguz, O.; Parikh, J.; Mudalige, P.; Bai, F.; Sadekar, V. Broadcast storm mitigation techniques in vehicular Ad Hoc networks. IEEE Wirel. Commun. 2007, 14, 84–94.
[29]
Suriyapaibonwattana, K.; Pornavalai, C. An Effective Safety Alert Broadcast Algorithm for VANET. Proceedings of the International Symposium on Communications and Information Technologies (ISCIT '08), Vientiane, Lao, 21–23 October 2008; pp. 247–250.
[30]
Alasmary, W.; Zhuang, W. Mobility impact in IEEE 802.11p infrastructureless vehicular networks. Ad Hoc Netw. 2012, 10, 222–230.
[31]
Harri, J.; Filali, F.; Bonnet, C. Mobility models for vehicular Ad Hoc networks: A survey and taxonomy. IEEE Commun. Surv. Tutor. 2009, 11, 19–41.
[32]
Krajzewicz, D.; Rossel, C. Simulation of Urban MObility (SUMO). Available online: http://sumo.sourceforge.net (accessed on 19 February 2013).
[33]
OpenStreetMap, Collaborative Project to Create a Free Editable Map of the World, 2011, Available online: http://www.openstreetmap.org (accessed on 19 February 2013).
[34]
TIGER, Topologically Integrated Geographic Encoding and Referencing, 2011, Available online: http://www.census.gov/geo/www/tiger (accessed on 19 February 2013).
[35]
Martinez, F.J.; Cano, J.C.; Calafate, C.T.; Manzoni, P. CityMob: A Mobility Model Pattern Generator for VANETs. Proceedings of the IEEE Vehicular Networks and Applications Workshop (Vehi-Mobi, Held with ICC 2008), Beijing, China, 19 May 2008; pp. 370–374.
[36]
Krauss, S.; Wagner, P.; Gawron, C. Metastable states in a microscopic model of traffic flow. Phys. Rev. E. 1997, 55, 5597–5602.
[37]
Krajzewicz, D.; Hertkorn, G.; Rossel, C.; Wagner, P. SUMO (Simulation of Urban MObility)—An Open-Source Traffic Simulation. Proceedings of the 4th International Middle Eastern Simulation Multiconference (MESM 2002), Sharjah, United Arab Emirates, 28–30 October 2002; pp. 183–187.
[38]
Wagner, P. How human drivers control their vehicle. Eur. Phys. J. B. 2006, 52, 427–431.
[39]
Martinez, F.J.; Toh, C.K.; Cano, J.C.; Calafate, C.T.; Manzoni, P. Realistic Radio Propagation Models (RPMs) for VANET Simulations. Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC 2009), Budapest, Hungary, 5–8 April 2009; pp. 1–6.
[40]
Fall, K.; Varadhan, K. ns Notes and Documents. The VINT Project. UC Berkeley, LBL, USC/ISI, and Xerox PARC, 2000, Available online: http://www.isi.edu/nsnam/ns/ns-documentation.html (accessed on 19 February 2013).
[41]
Fogue, M.; Garrido, P.; Martinez, F.J.; Cano, J.C.; Calafate, C.T.; Manzoni, P. A Realistic Simulation Framework for Vehicular Networks. Proceedings of the 5th International ICST Conference on Simulation Tools and Techniques (SIMUTools 2012), Desenzano, Italy, 19–23 March 2012; pp. 39–46.
[42]
MacQueen, J.B. Some Methods for Classification and Analysis of MultiVariate Observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA, 21 June–18 July 1965; Cam, L.M.L., Neyman, J., Eds.; University of California Press: Berkeley, CA, USA, 1967; Volume 1, pp. 281–297.
