Associating attributes to pedestrians in a crowd is relevant for various areas like surveillance, customer profiling and service providing. The attributes of interest greatly depend on the application domain and might involve such social relations as friends or family as well as the hierarchy of the group including the leader or subordinates. Nevertheless, the complex social setting inherently complicates this task. We attack this problem by exploiting the small group structures in the crowd. The relations among individuals and their peers within a social group are reliable indicators of social attributes. To that end, this paper identifies social groups based on explicit motion models integrated through a hypothesis testing scheme. We develop two models relating positional and directional relations. A pair of pedestrians is identified as belonging to the same group or not by utilizing the two models in parallel, which defines a compound hypothesis testing scheme. By testing the proposed approach on three datasets with different environmental properties and group characteristics, it is demonstrated that we achieve an identification accuracy of 87% to 99%. The contribution of this study lies in its definition of positional and directional relation models, its description of compound evaluations, and the resolution of ambiguities with our proposed uncertainty measure based on the local and global indicators of group relation.
References
[1]
Haritaoglu, I.; Flickner, M. Detection and Tracking of Shopping Groups in Stores. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA, 8– 14 December 2001; pp. I-431–I-438.
[2]
Chang, M.C.; Krahnstoever, N.; Ge, W. Probabilistic Group-Level Motion Analysis and Scenario Recognition. Proceedings of the 2011 IEEE International Conference on Computer Vision, Barcelona, Spain, 6– 13 November 2011; pp. 747–754.
[3]
Yu, T.; Lim, S.N.; Patwardhan, K.A.; Krahnstoever, N. Monitoring, Recognizing and Discovering Social Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20– 25 June 2009; pp. 1462–1469.
[4]
McPhail, C.; Wohlstein, R. Using film to analyze pedestrian behavior. Sociol. Method. Res. 1982, 10, 347.
Aggarwal, J.K.; Ryoo, M.S. Human activity analysis: A review. ACM Comput. Surv. 2011, doi:10.1145/1922649.1922653.
[7]
Cristani, M.; Raghavendra, R.; Del Bue, A.; Murino, V. Human behavior analysis in video surveillance: A social signal processing perspective. Neurocomputing 2013, 100, 86–97.
[8]
Gatica-Perez, D. Automatic nonverbal analysis of social interaction in small groups: A review. Image Vision Comput. 2009, 27, 1775–1787.
[9]
Costa, M. Interpersonal distances in group walking. J. Nonverbal Behav. 2010, 34, 15–26.
[10]
Chen, Y.Y.; Hsu, W.H.; Liao, H.Y.M. Discovering Informative Social Subgraphs and Predicting Pairwise Relationships from Group Photos. Proceedings of the 20th ACM International Conference on Multimedia, Nara, Japan, 29 October–2 November 2012.
[11]
Singla, P.; Kautz, H.; Luo, J.; Gallagher, A. Discovery of Social Relationships in Consumer Photo Collections Using Markov Logic. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Anchorage, AK, USA, 23– 28 June 2008; pp. 1–7.
[12]
Xia, S.; Shao, M.J.L.; Fu, Y. Understanding kin relationships in a photo. IEEE Trans. Multimed. 2012, 14, 1046–1056.
[13]
Wang, G.; Gallagher, A.C.; Luo, J.; Forsyth, D.A. Seeing people in social context: Recognizing people and social relationships. Lect. Note. Comput. Sci. 2010, 6315, 169–182.
[14]
Gallagher, A.C.; Chen, T. Understanding Images of Groups of People. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20– 25 June 2009; pp. 256–263.
[15]
Murillo, A.; Kwak, I.; Bourdev, L.; Kriegman, D.; Belongie, S. Urban Tribes: Analyzing Group Photos from a Social Perspective. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA, 16– 21 June 2012; pp. 28–35.
[16]
Ding, L.; Yilmaz, A. Learning relations among movie characters: A social network perspective. Lect. Notes Comput. Sci. 2010, 6314, 410–423.
[17]
Wang, X.; Tieu, K.; Grimson, E. Correspondence-free activity analysis and scene modeling in multiple camera views. IEEE Trans. Patt. Anal. Mach. Intell. 2010, 32, 56–71.
[18]
Wu, B.; Nevatia, R. Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses. Int. J. Comput. Vis. 2009, 82, 185–204.
[19]
Jorge, P.; Abrantes, A.; Marques, J. On-Line Tracking Groups of Pedestrians with Bayesian Networks. Proceedings of the 6th International Workshop on Performance Evaluation for Tracking and Surveillance, Prague, Czech Republic, 10 May 2004.
[20]
Pellegrini, S.; Ess, A.; Schindler, K.; Gool, L.J.V. You'll Never Walk Alone: Modeling Social Behavior for Multi-Target Tracking. Proceedings of the IEEE 12th International Conference on Computer Vision, 29 September– 2 October 2009; pp. 261–268.
[21]
Pellegrini, S.; Ess, A.; Gool, L.J.V. Improving data association by joint modeling of pedestrian trajectories and groupings. Lect. Notes Comput. Sci. 2010, 6311, 452–465.
[22]
Bose, B.; Wang, X.; Grimson, E. Multi-Class Object Tracking Algorithm that Handles Fragmentation and Grouping. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17– 22 June 2007; pp. 1–8.
[23]
Wertheimer, M. Laws of Organization in Perceptual Forms. In A Source Book of Gestalt Psychology; Routledge and Kegan Paul: London, UK, 1938.
[24]
Wang, X.; Ma, K.T.; Ng, G.W.; Grimson, W.E.L. Trajectory analysis and semantic region modeling using nonparametric hierarchical bayesian models. Int. J. Comput. Vis. 2011, 95, 287–312.
[25]
Yan, W.; Forsyth, D.A. Learning the Behavior of Users in a Public Space through Video Tracking. Proceedings of the 7th IEEE Workshops on Application of Computer Vision, Breckenridge, CO USA, 5– 7 January 2005; pp. 370–377.
[26]
Habe, H.; Honda, K.; Kidode, M. Human Interaction Analysis Based on Walking Pattern Transitions. Proceedings of the 3rd ACM/IEEE International Conference on Distributed Smart Cameras, Como, Italy, 30 August– 2 September 2009; pp. 1–8.
[27]
Zen, G.; Lepri, B.; Ricci, E.; Lanz, O. Space Speaks: Towards Socially and Personality Aware Visual Surveillance. Proceedings of the 1st ACM International Workshop on Multimodal Pervasive Video Analysis, Florence, Italy, 25–29 October 2010; pp. 37–42.
[28]
Choi, W.; Shahid, K.; Savarese, S. Learning Context for Collective Activity Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 20– 25 June 2011; pp. 3273–3280.
[29]
French, A.P.; Naeem, A.; Dryden, I.L.; Pridmore, T.P. Using Social Effects to Guide Rracking in Complex Scenes. Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, London, UK, 5– 7 September 2007; pp. 212–217.
[30]
Calderara, S.; Prati, A.; Cucchiara, R. Mixtures of von mises distributions for people trajectory shape analysis. IEEE Trans. Circuit. Syst. Video Technol. 2011, 21, 457–471.
[31]
Yücel, Z.; Ikeda, T.; Miyashita, T.; Hagita, N. Identification of Mobile Entities Based on Trajectory and Shape Information. Proceedings of the IEEE /RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25– 30 September 2011; pp. 3589–3594.
[32]
Yücel, Z.; Zanlungo, F.; Ikeda, T.; Miyashita, T.; Hagita, N. Modeling Indicators of Coherent Motion. Proceedings of the IEEE /RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal, 7– 12 October 2012; pp. 1–8.
[33]
Yücel, Z.; Miyashita, T.; Hagita, N. Modeling and Identification of Group Motion via Compound Evaluation of Positional and Directional Cues. Proceedings of the 21st International Conference on Pattern Recognition, Tsukuba, Japan, 11–15 November 2012; pp. 1–8.
[34]
Ge, W.; Collins, R.T.; Ruback, B. Vision-based analysis of small groups in pedestrian crowds. IEEE Trans. Patt. Anal. Mach. Intell. 2012, 34, 1003–1016.
[35]
Sandikci, S.; Zinger, S.; de With, P.H.N. Detection of human groups in videos. Lect. Notes Comput. Sci. 2011, 6915, 507–518.
[36]
Bahlmann, C. Directional features in online handwriting recognition. Patt. Recog. 2006, 39, 115–125.
[37]
Helbing, D.; Molnar, P. Social force model for pedestrian dynamics. Phys. Rev. E 1995, 51, 4282–4286.
[38]
Mehran, R.; Oyama, A.; Shah, M. Abnormal Crowd Behavior Detection Using Social Force Model. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20– 25 June 2009; pp. 935–942.
[39]
Lerner, A.; Chrysanthou, Y.; Lischinski, D. Crowds by example. Comput. Graph. Forum 2007, 26, 655–664.
[40]
Fisher, R. The PETS04 Surveillance Ground-Truth Data Sets. Proceedings of the 6th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Prague, Czech Republic, 10 May 2004.
[41]
Zeynep Yücel. Available online: http://www.irc.atr.jp/zeynep/research (accessed on 9 January 2013).
[42]
Moussa？d, M.; Perozo, N.; Garnier, S.; Helbing, D.; Theraulaz, G. The walking behaviour of pedestrian social groups and its impact on crowd dynamics. PLoS One 2010, doi:10.1371/journal.pone.0010047.
[43]
Glas, D.; Miyashita, T.; Ishiguro, H.; Hagita, N. Laser Tracking of Human Body Motion Using Adaptive Shape Modeling. Proceedings of the International Conference on Intelligent Robots and Systems, San Diego, CA, USA, 29 October–2 November 2007; pp. 602–608.
[44]
ETH Zurich—Department of Information Technology and Electrical Engineering Computer Vision Laboratory. Available online: http://www.vision.ee.ethz.ch/datasets/index.en.html (accessed on 9 January 2013).
[45]
Benchmark Data for PETS-ECCV 2004. Available online: http://www-prima.inrialpes.fr/PETS04/caviar_data.html (accessed on 9 January 2013).
[46]
Moussa？d, M.; Helbing, D.; Garnier, S.; Johansson, A.; Combe, M.; Theraulaz, G. Experimental study of the behavioural mechanisms underlying self-organization in human crowds. Proc. R. Soc. B. 2009, doi:10.1098/rspb.2009.0405.
[47]
Helbing, D.; Farkas, I.; Molnar, P.; Vicsek, T. Simulation of Pedestrian Crowds in Normal and Evacuation Situations. In Pedestrian and Evacuation Dynamics; Springer: Berlin, Germany, 2002.
[48]
Daamen, W.; Hoogendoorn, S. Free Speed Distributions for Pedestrian Traffic. Proceedings of the 85th Annual Meeting of Transportation Research Board, Washington DC, USA, 22– 26 January 2006.
[49]
Abramowitz, M.; Stegun, I. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables; Courier Dover: New York, NY, USA, 1965.
[50]
Francesco Zanlungo. Available online: https://sites.google.com/site/francescozanlungo/squarelinepicking (accessed on 9 January 2013).
[51]
Mardia, K.V.; Jupp, P.E. Directional Statistics; John Wiley and Sons Ltd.: New York, NY, USA, 2000.
[52]
Kose, C.; Wesel, R. Robustness of Likelihood Ratio Tests: Hypothesis Testing under Incorrect Models. Proceedings of the Thirty-Fifth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA, 4–7 November 2001; pp. 1738–1742.
[53]
Hall, E.T. The Hidden Dimension; Doubleday: Garden City, NY, USA, 1966.
