All Title Author
Keywords Abstract

PLOS ONE  2011 

Measures and Limits of Models of Fixation Selection

DOI: 10.1371/journal.pone.0024038

Full-Text   Cite this paper   Add to My Lib


Models of fixation selection are a central tool in the quest to understand how the human mind selects relevant information. Using this tool in the evaluation of competing claims often requires comparing different models' relative performance in predicting eye movements. However, studies use a wide variety of performance measures with markedly different properties, which makes a comparison difficult. We make three main contributions to this line of research: First we argue for a set of desirable properties, review commonly used measures, and conclude that no single measure unites all desirable properties. However the area under the ROC curve (a classification measure) and the KL-divergence (a distance measure of probability distributions) combine many desirable properties and allow a meaningful comparison of critical model performance. We give an analytical proof of the linearity of the ROC measure with respect to averaging over subjects and demonstrate an appropriate correction of entropy-based measures like KL-divergence for small sample sizes in the context of eye-tracking data. Second, we provide a lower bound and an upper bound of these measures, based on image-independent properties of fixation data and between subject consistency respectively. Based on these bounds it is possible to give a reference frame to judge the predictive power of a model of fixation selection . We provide open-source python code to compute the reference frame. Third, we show that the upper, between subject consistency bound holds only for models that predict averages of subject populations. Departing from this we show that incorporating subject-specific viewing behavior can generate predictions which surpass that upper bound. Taken together, these findings lay out the required information that allow a well-founded judgment of the quality of any model of fixation selection and should therefore be reported when a new model is introduced.


[1]  Itti L, Koch C (2001) Computational modelling of visual attention. Nature Reviews Neuroscience 2: 194–203.
[2]  Itti L, Baldi P (2005) Bayesian surprise attracts human attention. Vision Research 49: 1295–306.
[3]  Kanan C, Tong M, Zhang L, Cottrell G (2009) SUN: Top-down saliency using natural statistics. Visual Cognition 17: 979–1003.
[4]  Kienzle W, Franz M, Sch?lkopf B, Wichmann F (2009) Center-surround patterns emerge as optimal predictors for human saccade targets. Journal of Vision 9: 1–15.
[5]  Parkhurst D, Law K, Niebur E (2002) Modeling the role of salience in the allocation of overt visual attention. Vision Research 42: 107–123.
[6]  Peters R, Iyer A, Itti L, Koch C (2005) Components of bottom-up gaze allocation in natural images. Vision Research 45: 2397–2416.
[7]  Zhang L, Tong M, Marks T, Shan H, Cottrell G (2008) SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision 8: 1–32.
[8]  Tatler B, Vincent B (2009) The prominence of behavioural biases in eye guidance. Visual Cognition 17: 1029–1054.
[9]  Tatler B (2007) The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. Journal of Vision 7: 1–17.
[10]  Tatler BW, Baddeley RJ, Gilchrist ID (2005) Visual correlates of fixation selection: effects of scale and time. Vision Research 45: 643–59.
[11]  Cerf M, Harel J, Ein?user W, Koch C (2008) Predicting human gaze using low-level saliency combined with face detection. Advances in Neural Information Processing Systems 20: 241–248.
[12]  Cerf M, Frady EP, Koch C (2009) Faces and text attract gaze independent of the task: Experi mental data and computer model. Journal of Vision 9: 1–15.
[13]  Hwang A, Higgins E, Pomplun M (2009) A model of top-down attentional control during visual search in complex scenes. Journal of Vision 9: 1–18.
[14]  Torralba A, Oliva A, Castelhano MS, Henderson JM (2006) Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychological Review 113: 766–86.
[15]  Ehinger KA, Hidalgo-Sotelo B, Torralba A, Oliva A (2009) Modeling Search for People in 900 Scenes: A combined source model of eye guidance. Visual Cognition 17: 945–978.
[16]  Einh?user W, Spain M, Perona P (2008) Objects predict fixations better than early saliency. Journal of Vision 8: 1–18.
[17]  Harel J, Koch C, Perona P (2007) Graph-based visual saliency. Advances in Neural Information Processing Systems 19: 545–552.
[18]  Acik A, Onat S, Schumann F, Ein?user W, K?nig P (2009) Effects of luminance contrast and its modifications on fixation behavior during free viewing of images from different categories. Vision research 49: 1541–1553.
[19]  Baddeley RJ, Tatler BW (2006) High frequency edges (but not contrast) predict where we fixate: A Bayesian system identification analysis. Vision research 46: 2824–33.
[20]  Elazary L, Itti L (2008) Interesting objects are visually salient. Journal of Vision 8: 1–15.
[21]  Betz T, Kietzmann T, Wilming N, K?nig P (2010) Investigating task-dependent top-down effects on overt visual attention. Journal of Vision 10: 1–14.
[22]  Butko NJ, Movellan JR (2008) I-POMDP: An infomax model of eye movement. 2008 7th IEEE International Conference on Development and Learning 139–144.
[23]  Renninger LW, Verghese P, Coughlan J (2007) Where to look next? eye movements reduce local uncertainty. J Vis 7: 6.
[24]  Bruce NDB, Tsotsos JK (2009) Saliency, attention, and visual search: an information theoretic approach. J Vis 9: 5.1–24.
[25]  Kootstra G, de Boer B, Schomaker LRB (2011) Predicting eye fixations on complex visual stimuli using local symmetry. Cognit Comput 3: 223–240.
[26]  Yanulevskaya V, Marsman JB, Cornelissen F, Geusebroek JM (2011) An image statistics-based model for fixation prediction. Cognit Comput 3: 94–104.
[27]  Parikh N, Itti L, Weiland J (2010) Saliency-based image processing for retinal prostheses. Journal of Neural Engineering 7: 16006.
[28]  Fawcett T (2006) An introduction to ROC analysis. Pattern recognition letters 27: 861–874.
[29]  Itti L, Baldi P (2005) A principled approach to detecting surprising events in video. Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE, volume 1. pp. 631–637.
[30]  Frey H, Honey C, K?nig P (2008) What's color got to do with it? The inuence of color on visual attention in different categories. Journal of Vision 8: 1–17.
[31]  A?ik A, Sarwary A, Schultze-Kraft R, Onat S, K?nig P (2010) Developmental changes in natural viewing behavior: bottom-up and top- down differences between children, young adults and older adults. Frontiers in Psychology 2:
[32]  Hausser J, Strimmer K (2009) Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks. The Journal of Machine Learning Research 10: 1469–1484.
[33]  Miller G (1955) Note on the bias of information estimates. Information Theory in Psychology: Problems and Methods II-B II: 95–100.
[34]  Nemenman I, Shafee F, Bialek W (2002) Entropy and inference, revisited. Advances in Neural Information Processing Systems 1: 471–478.
[35]  Chao A, Shen T (2003) Nonparametric estimation of Shannon's index of diversity when there are unseen species in sample. Environmental and Ecological Statistics 10: 429–443.
[36]  Holste D, Grosse I, Herzel H (1998) Bayes' estimators of generalized entropies. Journal of Physics A: Mathematical and General 31: 2551.
[37]  Krichevsky R, Trofimov V (2002) The performance of universal encoding. IEEE Transactions on Information Theory 27: 199–207.
[38]  Schürmann T, Grassberger P (1996) Entropy estimation of symbol sequences. Chaos 6: 414–427.
[39]  Trybula S (1958) Some problems of simultaneous minimax estimation. The Annals of Mathematical Statistics 29: 245–253.
[40]  Gilland J (2008) Driving, eye-tracking and visual entropy: Exploration of age and task effects. Ph.D. thesis, The University of South Dakota.
[41]  Recarte M, Nunes L (2000) Effects of verbal and spatial-imagery tasks on eye fixations while driving. Journal of Experimental Psychology Applied 6: 31–43.


comments powered by Disqus

Contact Us


微信:OALib Journal