All Title Author
Keywords Abstract


Spatial Stereoresolution for Depth Corrugations May Be Set in Primary Visual Cortex

DOI: 10.1371/journal.pcbi.1002142

Full-Text   Cite this paper   Add to My Lib

Abstract:

Stereo “3D” depth perception requires the visual system to extract binocular disparities between the two eyes' images. Several current models of this process, based on the known physiology of primary visual cortex (V1), do this by computing a piecewise-frontoparallel local cross-correlation between the left and right eye's images. The size of the “window” within which detectors examine the local cross-correlation corresponds to the receptive field size of V1 neurons. This basic model has successfully captured many aspects of human depth perception. In particular, it accounts for the low human stereoresolution for sinusoidal depth corrugations, suggesting that the limit on stereoresolution may be set in primary visual cortex. An important feature of the model, reflecting a key property of V1 neurons, is that the initial disparity encoding is performed by detectors tuned to locally uniform patches of disparity. Such detectors respond better to square-wave depth corrugations, since these are locally flat, than to sinusoidal corrugations which are slanted almost everywhere. Consequently, for any given window size, current models predict better performance for square-wave disparity corrugations than for sine-wave corrugations at high amplitudes. We have recently shown that this prediction is not borne out: humans perform no better with square-wave than with sine-wave corrugations, even at high amplitudes. The failure of this prediction raised the question of whether stereoresolution may actually be set at later stages of cortical processing, perhaps involving neurons tuned to disparity slant or curvature. Here we extend the local cross-correlation model to include existing physiological and psychophysical evidence indicating that larger disparities are detected by neurons with larger receptive fields (a size/disparity correlation). We show that this simple modification succeeds in reconciling the model with human results, confirming that stereoresolution for disparity gratings may indeed be limited by the size of receptive fields in primary visual cortex.

References

[1]  Westheimer G (1975) Editorial: Visual acuity and hyperacuity. Invest Ophthalmol 14: 570–572.
[2]  Tyler CW (1973) Steroscopic vision: cortical limitations and a disparity scaling effect. Science 181: 276–278.
[3]  Tyler CW (1974) Depth perception in disparity gratings. Nature 251: 140–142.
[4]  Tyler CW (1975) Spatial organization of binocular disparity sensitivity. Vision Res 15: 583–590.
[5]  Schade OH Sr (1956) Optical and photoelectric analog of the eye. J Opt Soc Am 46: 721–739.
[6]  Banks MS, Gepshtein S, Landy MS (2004) Why is spatial stereoresolution so low? J Neurosci 24: 2077–2089.
[7]  Bradshaw MF, Rogers BJ (1999) Sensitivity to horizontal and vertical corrugations defined by binocular disparity. Vision Res 39: 3049–3056.
[8]  Filippini HR, Banks MS (2009) Limits of stereopsis explained by local cross-correlation. J Vis 9: 8 1–18.
[9]  Nienborg H, Bridge H, Parker AJ, Cumming BG (2004) Receptive field size in V1 neurons limits acuity for perceiving disparity modulation. J Neurosci 24: 2065–2076.
[10]  Allenmark F, Read JC (2010) Detectability of sine- versus square-wave disparity gratings: A challenge for current models of depth perception. J Vis 10: 1–16.
[11]  Janssen P, Vogels R, Orban GA (1999) Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. Proc Natl Acad Sci U S A 96: 8217–8222.
[12]  Nguyenkim JD, DeAngelis GC (2003) Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J Neurosci 23: 7117–7128.
[13]  Sakata H, Taira M, Kusunoki M, Murata A, Tsutsui K, et al. (1999) Neural representation of three-dimensional features of manipulation objects with stereopsis. Exp Brain Res 128: 160–169.
[14]  Sugihara H, Murakami I, Shenoy KV, Andersen RA, Komatsu H (2002) Response of MSTd neurons to simulated 3D orientation of rotating planes. J Neurophysiol 87: 273–285.
[15]  McKee SP, Verghese P (2002) Stereo transparency and the disparity gradient limit. Vision Res 42: 1963–1977.
[16]  Smallman HS, MacLeod DI (1994) Size-disparity correlation in stereopsis at contrast threshold. J Opt Soc Am A Opt Image Sci Vis 11: 2169–2183.
[17]  Tsirlin I, Allison RS, Wilcox LM (2008) Stereoscopic transparency: Constraints on the perception of multiple surfaces. J Vis 8: 1–10.
[18]  Prince SJ, Cumming BG, Parker AJ (2002) Range and mechanism of encoding of horizontal disparity in macaque V1. J Neurophysiol 87: 209–221.
[19]  Tsai JJ, Victor JD (2003) Reading a population code: a multi-scale neural model for representing binocular disparity. Vision Res 43: 445–466.
[20]  Read JCA (2010) Vertical Binocular Disparity is Encoded Implicitly within a Model Neuronal Population Tuned to Horizontal Disparity and Orientation. PLoS Comput Biol 6: 1–15.
[21]  Read JCA, Cumming BG (2006) Does depth perception require vertical-disparity detectors? J Vis 6: 1323–1355.
[22]  Burt P, Julesz B (1980) A Disparity Gradient Limit for Binocular Fusion. Science 208: 615–617.
[23]  Kanade T, Okutomi M (1994) A Stereo Matching Algorithm with an Adaptive Window - Theory and Experiment. IEEE Trans Pattern Anal Mach Intell 16: 920–932.
[24]  Julesz B (1971) Foundations of cyclopean perception. Chicago: University of Chicago. 406 p.
[25]  Bredfeldt CE, Cumming BG (2006) A simple account of cyclopean edge responses in macaque v2. J Neurosci 26: 7581–7596.
[26]  von der Heydt R, Zhou H, Friedman HS (2000) Representation of stereoscopic edges in monkey visual cortex. Vision Res 40: 1955–1967.
[27]  Harris JM, McKee SP, Smallman HS (1997) Fine-scale processing in human binocular stereopsis. J Opt Soc Am A Opt Image Sci Vis 14: 1673–1683.
[28]  Hannah M (1974) Computer matching of areas in stereo imagery: Stanford University
[29]  Panton DJ (1978) Flexible Approach to Digital Stereo Mapping. Photogramm Eng Remote Sensing 44: 1499–1512.
[30]  Steingrube P, Gehrig SK, Franke U (2009) Performance Evaluation of Stereo Algorithms for Automotive Applications. Computer Vision Systems, Proceedings 5815: 285–294.
[31]  Cormack LK, Stevenson SB, Schor CM (1991) Interocular Correlation, Luminance Contrast and Cyclopean Processing. Vision Res 1: 2195–2207.
[32]  Fleet DJ, Jepson AD, Jenkin MRM (1991) Phase-Based Disparity Measurement. Comput Vis Image Underst 53: 198–210.
[33]  Ohzawa I, DeAngelis GC, Freeman RD (1990) Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. Science 249: 1037–1041.
[34]  Qian N, Mikaelian S (2000) Relationship between phase and energy methods for disparity computation. Neural Comput 12: 279–292.
[35]  Qian N, Zhu YD (1997) Physiological computation of binocular disparity. Vision Res 37: 1811–1827.
[36]  Fleet DJ, Wagner H, Heeger DJ (1996) Neural encoding of binocular disparity: energy models, position shifts and phase shifts. Vision Res 36: 1839–1857.
[37]  Qian N (1994) Computing Stereo Disparity and Motion with Known Binocular Cell Properties. Neural Comput 6: 390–404.
[38]  Read JC, Cumming BG (2007) Sensors for impossible stimuli may solve the stereo correspondence problem. Nat Neurosci 10: 1322–1328.
[39]  Read JC, Cumming BG (2004) Understanding the cortical specialization for horizontal disparity. Neural Comput 16: 1983–2020.
[40]  Read JC (2002) A Bayesian model of stereopsis depth and motion direction discrimination. Biol Cybern 86: 117–136.
[41]  Deangelis GC, Ohzawa I, Freeman RD (1991) Depth Is Encoded in the Visual-Cortex by a Specialized Receptive-Field Structure. Nature 352: 156–159.
[42]  Devalois RL, Albrecht DG, Thorell LG (1982) Spatial-Frequency Selectivity of Cells in Macaque Visual-Cortex. Vision Res 22: 545–559.
[43]  Kulikowski JJ (1978) Limit of single vision in stereopsis depends on contour sharpness. Nature 275: 126–127.
[44]  Schor CM, Wood I (1983) Disparity range for local stereopsis as a function of luminance spatial frequency. Vision Res 23: 1649–1654.
[45]  Cumming BG, Parker AJ (1999) Binocular neurons in V1 of awake monkeys are selective for absolute, not relative, disparity. J Neurosci 19: 5602–5618.
[46]  McKee SP, Verghese P, Farell B (2004) What is the depth of a sinusoidal grating? J Vis 4: 524–538.
[47]  Bradshaw MF, Hibbard PB, Parton AD, Rose D, Langley K (2006) Surface orientation, modulation frequency and the detection and perception of depth defined by binocular disparity and motion parallax. Vision Res 46: 2636–2644.
[48]  Serrano-Pedraza I, Read JCA (2010) Multiple channels for horizontal, but only one for vertical corrugations? A new look at the stereo anisotropy. J Vis 10: 1–11.
[49]  van der Willigen RF, Harmening WM, Vossen S, Wagner H (2010) Disparity sensitivity in man and owl: Psychophysical evidence for equivalent perception of shape-from-stereo. J Vis 10: 1–11.
[50]  Cumming BG (2002) An unexpected specialization for horizontal disparity in primate primary visual cortex. Nature 418: 633–636.
[51]  Parker AJ (2007) Binocular depth perception and the cerebral cortex. Nature Reviews Neuroscience 8: 379–391.
[52]  Snyder LH, Batista AP, Andersen RA (2000) Intention-related activity in the posterior parietal cortex: a review. Vision Res 40: 1433–1441.

Full-Text

comments powered by Disqus