In recent years, our research in computational neuroscience has focused on understanding how populations of neurons encode naturalistic stimuli. In particular, we focused on how populations of neurons use the time domain to encode sensory information. In this focused review, we summarize this recent work from our laboratory. We focus in particular on the mathematical methods that we developed for the quantification of how information is encoded by populations of neurons and on how we used these methods to investigate the encoding of complex naturalistic sounds in auditory cortex. We review how these methods revealed a complementary role of low frequency oscillations and millisecond precise spike patterns in encoding complex sounds and in making these representations robust to imprecise knowledge about the timing of the external stimulus. Further, we discuss challenges in extending this work to understand how large populations of neurons encode sensory information. Overall, this previous work provides analytical tools and conceptual understanding necessary to study the principles of how neural populations reflect sensory inputs and achieve a stable representation despite many uncertainties in the environment. 1. Introduction Our sensory percept and our interaction with the environment arise from neural representations of the external world. An important question is therefore how the characteristics of external events, such as sensory stimuli, are represented by patterns of neural activity in the brain. Answering these questions amounts to determining the neural code [1–3], more formally defined as the smallest set of response patterns capable of encoding relevant stimulus parameters [4]. Two dimensions of neural representations are important for characterizing a neural code. The first is defined by space: sensory processing is based on spatially distributed populations of neurons, ranging from localized groups to populations of neurons spread across brain areas [5, 6]. The second dimension is defined by time: neuronal responses evolve over time, and the temporal structure of neural activity is often required to explain speeded reactions. Under most circumstances, neglecting the temporal dimension of neural activity results in a much impoverished representation of the sensory input [4, 7]. In this review, we focus on the recent work of our laboratory towards understanding the temporal dimension of neural codes in the auditory system. We first discuss our general mathematical approach, based on the principles of information theory, to evaluate the
References
[1]
D. Ferster and N. Spruston, “Cracking the neuronal code,” Science, vol. 270, no. 5237, pp. 756–757, 1995.
[2]
C. F. Stevens and A. Zador, “The enigma of the brain,” Current Biology, vol. 5, no. 12, pp. 1370–1371, 1995.
[3]
F. Rieke, Spikes: Exploring the Neural Code, The MIT Press, Cambridge, Mass, USA, 1999.
[4]
S. Panzeri, N. Brunel, N. K. Logothetis, and C. Kayser, “Sensory neural codes using multiplexed temporal scales,” Trends in Neurosciences, vol. 33, no. 3, pp. 111–120, 2010.
[5]
W. B. Kristan Jr. and B. K. Shaw, “Population coding and behavioral choice,” Current Opinion in Neurobiology, vol. 7, no. 6, pp. 826–831, 1997.
[6]
A. Pouget, P. Dayan, and R. Zemel, “Information processing with population codes,” Nature Reviews Neuroscience, vol. 1, no. 2, pp. 125–132, 2000.
[7]
J. D. Victor, “How the brain uses time to represent and process visual information,” Brain Research, vol. 886, no. 1-2, pp. 33–46, 2000.
[8]
G. Foffani, J. K. Chapin, and K. A. Moxon, “Computational role of large receptive fields in the primary somatosensory cortex,” Journal of Neurophysiology, vol. 100, no. 1, pp. 268–280, 2008.
[9]
T. Gollisch and M. Meister, “Rapid neural coding in the retina with relative spike latencies,” Science, vol. 319, no. 5866, pp. 1108–1111, 2008.
[10]
R. Shusterman, M. C. Smear, A. A. Koulakov, and D. Rinberg, “Precise olfactory responses tile the sniff cycle,” Nature Neuroscience, vol. 14, no. 8, pp. 1039–1044, 2011.
[11]
S. Panzeri, R. Senatore, M. A. Montemurro, and R. S. Petersen, “Correcting for the sampling bias problem in spike train information measures,” Journal of Neurophysiology, vol. 98, no. 3, pp. 1064–1072, 2007.
[12]
R. A. A. Ince, R. Senatore, E. Arabzadeh, F. Montani, M. E. Diamond, and S. Panzeri, “Information-theoretic methods for studying population codes,” Neural Networks, vol. 23, no. 6, pp. 713–727, 2010.
[13]
I. Nelken and G. Chechik, “Information theory in auditory research,” Hearing Research, vol. 229, no. 1-2, pp. 94–105, 2007.
[14]
R. Q. Quiroga and S. Panzeri, “Extracting information from neuronal populations: information theory and decoding approaches,” Nature Reviews Neuroscience, vol. 10, no. 3, pp. 173–185, 2009.
[15]
C. E. Shannon, “The mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379–423, 623–656, 1948.
[16]
J. D. Victor and K. P. Purpura, “Nature and precision of temporal coding in visual cortex: a metric-space analysis,” Journal of Neurophysiology, vol. 76, no. 2, pp. 1310–1326, 1996.
[17]
R. R. D. R. van Steveninck, G. D. Lewen, S. P. Strong, R. Koberle, and W. Bialek, “Reproducibility and variability in neural spike trains,” Science, vol. 275, no. 5307, pp. 1805–1808, 1997.
[18]
C. Kayser, M. A. Montemurro, N. K. Logothetis, and S. Panzeri, “Spike-phase coding boosts and stabilizes the information carried by spatial and temporal spike patterns,” Neuron, vol. 61, no. 4, pp. 597–608, 2009.
[19]
C. Kayser, N. K. Logothetis, and S. Panzeri, “Millisecond encoding precision of auditory cortex neurons,” Proceedings of the National Academy of Sciences of the United States of America, vol. 107, no. 39, pp. 16976–16981, 2010.
[20]
S. Panzeri, R. S. Petersen, S. R. Schultz, M. Lebedev, and M. E. Diamond, “The role of spike timing in the coding of stimulus location in rat somatosensory cortex,” Neuron, vol. 29, no. 3, pp. 769–777, 2001.
[21]
J. A. Garcia-Lazaro, L. A. Belliveau, and N. A. Lesica, “Independent population coding of speech with sub-millisecond precision,” The Journal of Neuroscience, vol. 33, no. 49, pp. 19362–19372, 2013.
[22]
C. A. Perez, C. T. Engineer, V. Jakkamsetti, R. S. Carraway, M. S. Perry, and M. P. Kilgard, “Different timescales for the neural coding of consonant and vowel sounds,” Cerebral Cortex, vol. 23, no. 3, pp. 670–683, 2013.
[23]
M. E. Diamond, M. von Heimendahl, and E. Arabzadeh, “Whisker-mediated texture discrimination,” PLoS Biology, vol. 6, article e220, 2008.
[24]
M. E. Diamond and E. Arabzadeh, “Whisker sensory system—from receptor to decision,” Progress in Neurobiology, vol. 103, pp. 28–40, 2013.
[25]
K. P. Purpura, S. F. Kalik, and N. D. Schiff, “Analysis of perisaccadic field potentials in the occipitotemporal pathway during active vision,” Journal of Neurophysiology, vol. 90, no. 5, pp. 3455–3478, 2003.
[26]
C. E. Schroeder, D. A. Wilson, T. Radman, H. Scharfman, and P. Lakatos, “Dynamics of active sensing and perceptual selection,” Current Opinion in Neurobiology, vol. 20, no. 2, pp. 172–176, 2010.
[27]
J. D. Moore, M. Deschênes, T. Furuta et al., “Hierarchy of orofacial rhythms revealed through whisking and breathing,” Nature, vol. 497, no. 7448, pp. 205–210, 2013.
[28]
R. Brasselet, S. Panzeri, N. K. Logothetis, and C. Kayser, “Neurons with stereotyped and rapid responses provide a reference frame for relative temporal coding in primate auditory cortex,” Journal of Neuroscience, vol. 32, no. 9, pp. 2998–3008, 2012.
[29]
S. Panzeri, R. A. Ince, M. E. Diamond, and C. Kayser, “Reading spike timing without a clock: intrinsic decoding of spike trains,” Philosophical Transactions of the Royal Society of London Series B, Biological Sciences, vol. 369, no. 1637, 2014.
[30]
C. T. Engineer, C. A. Perez, Y. H. Chen et al., “Cortical activity patterns predict speech discrimination ability,” Nature Neuroscience, vol. 11, no. 5, pp. 603–608, 2008.
[31]
S. M. Chase and E. D. Young, “First-spike latency information in single neurons increases when referenced to population onset,” Proceedings of the National Academy of Sciences of the United States of America, vol. 104, no. 12, pp. 5175–5180, 2007.
[32]
C. Kayser, R. A. A. Ince, and S. Panzeri, “Analysis of slow (theta) oscillations as a potential temporal reference frame for information coding in sensory cortices,” PLoS Computational Biology, vol. 8, no. 10, Article ID e1002717, 2012.
[33]
S. A. Shamma, M. Elhilali, and C. Micheyl, “Temporal coherence and attention in auditory scene analysis,” Trends in Neurosciences, vol. 34, no. 3, pp. 114–123, 2011.
[34]
A. Giraud and D. Poeppel, “Cortical oscillations and speech processing: emerging computational principles and operations,” Nature Neuroscience, vol. 15, no. 4, pp. 511–517, 2012.
[35]
J. Lisman, “The theta/gamma discrete phase code occuring during the hippocampal phase precession may be a more general brain coding scheme,” Hippocampus, vol. 15, no. 7, pp. 913–922, 2005.
[36]
H. Luo and D. Poeppel, “Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex,” Neuron, vol. 54, no. 6, pp. 1001–1010, 2007.
[37]
M. A. Montemurro, M. J. Rasch, Y. Murayama, N. K. Logothetis, and S. Panzeri, “Phase-of-firing coding of natural visual stimuli in primary visual cortex,” Current Biology, vol. 18, no. 5, pp. 375–380, 2008.
[38]
T. M. Elliott and F. E. Theunissen, “The modulation transfer function for speech intelligibility,” PLoS Computational Biology, vol. 5, no. 3, Article ID e1000302, 2009.
[39]
C. Chandrasekaran, H. K. Turesson, C. H. Brown, and A. A. Ghazanfar, “The influence of natural scene dynamics on auditory cortical activity,” The Journal of Neuroscience, vol. 30, no. 42, pp. 13919–13931, 2010.
[40]
J. Gross, N. Hoogenboom, G. Thut et al., “Speech rhythms and multiplexed oscillatory sensory coding in the human brain,” PLoS Biology, vol. 11, Article ID e1001752, 2013.
[41]
F. D. Szymanski, N. C. Rabinowitz, C. Magri, S. Panzeri, and J. W. H. Schnupp, “The laminar and temporal structure of stimulus information in the phase of field potentials of auditory cortex,” Journal of Neuroscience, vol. 31, no. 44, pp. 15787–15801, 2011.
[42]
A. Mazzoni, S. Panzeri, N. K. Logothetis, and N. Brunel, “Encoding of naturalistic stimuli by local field potential spectra in networks of excitatory and inhibitory neurons,” PLoS Computational Biology, vol. 4, no. 12, Article ID e1000239, 2008.
[43]
A. Mazzoni, K. Whittingstall, N. Brunel, N. K. Logothetis, and S. Panzeri, “Understanding the relationships between spike rate and delta/gamma frequency bands of LFPs and EEGs using a local cortical network model,” NeuroImage, vol. 52, no. 3, pp. 956–972, 2010.
[44]
S. Cavallari, S. Panzeri, and A. Mazzoni, “Comparison of the dynamics of neural interactions between current-based and conductance-based integrate-and-fire recurrent networks,” Frontiers in Neural Circuits, vol. 8, article 12, 2014.
[45]
D. W. Dong and J. J. Atick, “Statistics of natural time-varying images,” Network: Computation in Neural Systems, vol. 6, pp. 345–358, 1995.
[46]
H. Attias and C. Schreiner, “Temporal low-order statistics of natural sounds,” in Advances in Neural Information Processing Systems, pp. 27–33, MIT Press, Cambridge, Mass, USA, 1997.
[47]
A. T. Schaefer, K. Angelo, H. Spors, and T. W. Margrie, “Neuronal oscillations enhance stimulus discrimination by ensuring action potential precision,” PLoS Biology, vol. 4, no. 6, article e163, 2006.
[48]
J. Kwag, D. McLelland, and O. Paulsen, “Phase of firing as a local window for efficient neuronal computation: tonic and phasic mechanisms in the control of theta spike phase,” Frontiers in Human Neuroscience, vol. 5, article 3, 2011.
[49]
A. Arieli, D. Shoham, R. Hildesheim, and A. Grinvald, “Coherent spatiotemporal patterns of ongoing activity revealed by real-time optical imaging coupled with single-unit recording in the cat visual cortex,” Journal of Neurophysiology, vol. 73, no. 5, pp. 2072–2093, 1995.
[50]
J. B. M. Goense and N. K. Logothetis, “Neurophysiology of the BOLD fMRI signal in awake monkeys,” Current Biology, vol. 18, no. 9, pp. 631–640, 2008.
[51]
M. Okun, A. Naim, and I. Lampl, “The subthreshold relation between cortical local field potential and neuronal firing unveiled by intracellular recordings in awake rats,” Journal of Neuroscience, vol. 30, no. 12, pp. 4440–4448, 2010.
[52]
A. Alenda, M. Molano-Mazón, S. Panzeri, and M. Maravall, “Sensory input drives multiple intracellular information streams in somatosensory cortex,” Journal of Neuroscience, vol. 30, no. 32, pp. 10872–10884, 2010.
[53]
V. Braintenberg and A. Schuetz, Cortex: Statistics and Geometry of Neuronal Connectivity, Springer, Berlin, Germany, 1998.
[54]
J. D. Fitzgerald, R. J. Rowekamp, L. C. Sincich, and T. O. Sharpee, “Second order dimensionality reduction using minimum and maximum mutual information models,” PLoS Computational Biology, vol. 7, no. 10, Article ID e1002249, 2011.
[55]
R. A. Ince, S. Panzeri, and C. Kayser, “Neural codes formed by small and temporally precise populations in auditory cortex,” The Journal of Neuroscience, vol. 33, no. 46, pp. 18277–18287, 2013.
[56]
F. Montani, R. A. Ince, R. Senatore, E. Arabzadeh, M. E. Diamond, and S. Panzeri, “The impact of high-order interactions on the rate of synchronous discharge and information transmission in somatosensory cortex,” Philosophical Transactions A: Mathematical, Physical and Engineering Sciences, vol. 367, no. 1901, pp. 3297–3310, 2009.
[57]
D. D. Lee and H. S. Seung, “Learning the parts of objects by non-negative matrix factorization,” Nature, vol. 401, no. 6755, pp. 788–791, 1999.
[58]
I. Delis, S. Panzeri, T. Pozzo, and B. Berret, “A unifying model of concurrent spatial and temporal modularity in muscle activity,” Journal of Neurophysiology, vol. 111, pp. 675–693, 2014.
[59]
J. H. Macke, P. Berens, A. S. Ecker, A. S. Tolias, and M. Bethge, “Generating spike trains with specified correlation coefficients,” Neural Computation, vol. 21, no. 2, pp. 397–423, 2009.
[60]
A. S. Ecker, P. Berens, A. S. Tolias, and M. Bethge, “The effect of noise correlations in populations of diversely tuned neurons,” The Journal of Neuroscience, vol. 31, no. 40, pp. 14272–14283, 2011.
[61]
A. Onken, V. Dragoi, and K. Obermayer, “A maximum entropy test for evaluating higher-order correlations in spike counts,” PLoS Computational Biology, vol. 8, no. 6, Article ID e1002539, 12 pages, 2012.
[62]
A. Onken, S. Grunewalder, M. H. J. Munk, and K. Obermayer, “Analyzing short-term noise dependencies of spike-counts in Macaque prefrontal cortex using copulas and the flashlight transformation,” PLoS Computational Biology, vol. 5, no. 11, Article ID e1000577, e1000577, 13 pages, 2009.
[63]
B. Staude, S. Rotter, and S. Grün, “CuBIC: cumulant based inference of higher-order correlations in massively parallel spike trains,” Journal of Computational Neuroscience, vol. 29, no. 1-2, pp. 327–350, 2010.
[64]
M. N. Shadlen and J. A. Movshon, “Synchrony unbound: a critical evaluation of the temporal binding hypothesis,” Neuron, vol. 24, no. 1, pp. 67–77, 111–125, 1999.
[65]
C. von der Malsburg, “The what and why of binding: the modeler's perspective,” Neuron, vol. 24, no. 1, pp. 95–104, 1999.
[66]
S. H. Nirenberg and J. D. Victor, “Analyzing the activity of large populations of neurons: how tractable is the problem?” Current Opinion in Neurobiology, vol. 17, no. 4, pp. 397–400, 2007.
[67]
O. Shriki, A. Kohn, and M. Shamir, “Fast coding of orientation in primary visual cortex,” PLoS Computational Biology, vol. 8, no. 6, Article ID e1002536, 2012.
[68]
T. Masquelier, “Neural variability, or lack thereof,” Frontiers in Computational Neuroscience, vol. 7, 2013.
[69]
S. Furukawa, L. Xu, and J. C. Middlebrooks, “Coding of sound-source location by ensembles of cortical neurons,” Journal of Neuroscience, vol. 20, no. 3, pp. 1216–1228, 2000.
[70]
S. Junek, E. Kludt, F. Wolf, and D. Schild, “Olfactory coding with patterns of response latencies,” Neuron, vol. 67, no. 5, pp. 872–884, 2010.
[71]
A. Kumar, S. Rotter, and A. Aertsen, “Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding,” Nature Reviews Neuroscience, vol. 11, no. 9, pp. 615–627, 2010.
[72]
M. V. Tsodyks and H. Markram, “The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability,” Proceedings of the National Academy of Sciences of the United States of America, vol. 94, no. 2, pp. 719–723, 1997.
[73]
R. Gütig and H. Sompolinsky, “The tempotron: a neuron that learns spike timing-based decisions,” Nature Neuroscience, vol. 9, no. 3, pp. 420–428, 2006.
[74]
T. Masquelier, R. Guyonneau, and S. J. Thorpe, “Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains,” PLoS ONE, vol. 3, no. 1, Article ID e1377, 2008.
[75]
A. Luczak, P. Bartho, and K. D. Harris, “Gating of sensory input by spontaneous cortical activity,” Journal of Neuroscience, vol. 33, no. 4, pp. 1684–1695, 2013.
[76]
T. Masquelier, E. Hugues, G. Deco, and S. J. Thorpe, “Oscillations, phase-of-firing coding, and spike timing-dependent plasticity: an efficient learning scheme,” Journal of Neuroscience, vol. 29, no. 43, pp. 13484–13493, 2009.
[77]
D. V. Buonomano and W. Maass, “State-dependent computations: spatiotemporal processing in cortical networks,” Nature Reviews Neuroscience, vol. 10, no. 2, pp. 113–125, 2009.
