A computational model of a self-structuring neuronal net is presented in which repetitively applied pattern sets induce the formation of cortical columns and microcircuits which decode distinct patterns after a learning phase. In a case study, it is demonstrated how specific neurons in a feature classifier layer become orientation selective if they receive bar patterns of different slopes from an input layer. The input layer is mapped and intertwined by self-evolving neuronal microcircuits to the feature classifier layer. In this topical overview, several models are discussed which indicate that the net formation converges in its functionality to a mathematical transform which maps the input pattern space to a feature representing output space. The self-learning of the mathematical transform is discussed and its implications are interpreted. Model assumptions are deduced which serve as a guide to apply model derived repetitive stimuli pattern sets to in vitro cultures of neuron ensembles to condition them to learn and execute a mathematical transform. 1. Introduction It can be said that neuronal networks, whether artificial, in vivo, or in vitro, are capable of information processing if they are able to learn and discriminate between pattern sets [1–3]. The central focus in modeling the information processing of such networks is on the specific neuronal architecture which is trained. This is because the architecture of the network determines the possible pattern discriminations that can be performed between pattern sets. For example, a specific architecture may provide orientation selectivity and thus be capable of discriminating between bars of different slopes. Bioinspired concepts will be introduced in the first section of this review with emphasis on the aspects of the in vivo experiments of orientation selectivity by Hubel [4]. Furthermore, the hypothesis of Blasdel will be revisited in the section on the Hough transform in the neurobiological context. The hypothesis states that the firing of these orientation selective cells can be explained by mapping the input stimuli back to the firing cells using a mathematical Hough transform [5]. To strengthen the plausibility of Blasdel’s hypothesis, the motion-detection experiments of Okamoto et al. are also revisited which investigated the hypothesis under the assumption that the mathematical Hough transformation is functionally used and represented as microcircuitry for bar detection in the medial temporal lobe (MTL) of the brain [6]. The base principle of the mathematical Hough transform will be
References
[1]
D. V. Buonomano and W. Maass, “State-dependent computations: spatiotemporal processing in cortical networks,” Nature Reviews Neuroscience, vol. 10, no. 2, pp. 113–125, 2009.
[2]
S. Sadeghi and K. Ramanathan, “A Hubel Wiesel model of early concept generalization based on local correlation of input features,” in Proceedings of the International Joint Conference on Neural Network (IJCNN '11), pp. 709–716, San Jose, Calif, USA, August 2011.
[3]
R. Q. Quiroga, L. Reddy, G. Kreiman, C. Koch, and I. Fried, “Invariant visual representation by single neurons in the human brain,” Nature, vol. 435, no. 7045, pp. 1102–1107, 2005.
[4]
D. H. Hubel, “Evolution of ideas on the primary visual cortex, 1955–1978: a biased historical account—Nobel lecture, 8 December 1981,” Bioscience Reports, vol. 2, no. 7, pp. 435–469, 1982.
[5]
G. G. Blasdel, “Orientation selectivity, preference, and continuity in monkey striate cortex,” The Journal of Neuroscience, vol. 12, no. 8, pp. 3139–3161, 1992.
[6]
H. Okamoto, S. Kawakami, H.-A. Saito et al., “MT neurons in the macaque exhibited two types of bimodal direction tuning as predicted by a model for visual motion detection,” Vision Research, vol. 39, no. 20, pp. 3465–3479, 1999.
[7]
L. O. Trussell, “Synaptic mechanisms for coding timing in auditory neurons,” Annual Review of Physiology, vol. 61, pp. 477–496, 1999.
[8]
K. Kandler, A. Clause, and J. Noh, “Tonotopic reorganization of developing auditory brainstem circuits,” Nature Neuroscience, vol. 12, no. 6, pp. 711–717, 2009.
[9]
A. G. Habib, M. A. Cameron, G. J. Suaning, N. H. Lovell, and J. W. Morley, “Spatially restricted electrical activation of retinal ganglion cells in the rabbit retina by hexapolar electrode return configuration,” Journal of Neural Engineering, vol. 10, no. 3, Article ID 036013, 2013.
[10]
R. E. Hampson, G. A. Gerhardt, V. Marmarelis, et al., “Facilitation and restoration of cognitive function in primate prefrontal cortex by a neuroprosthesis that utilizes minicolumn-specific neural firing,” Journal of Neural Engineering, vol. 9, no. 5, Article ID 056012, 2012.
[11]
G. Langner and M. Ochse, “The neural basis of pitch and harmony in the auditory system,” Musicae Scientiae, vol. 10, no. 1, pp. 185–208, 2006.
[12]
D. H. Hubel and T. N. Wiesel, “Ferrier lecture. Functional architecture of macaque monkey visual cortex,” Proceedings of the Royal Society B, vol. 190, no. 1130, pp. 1–59, 1977.
[13]
D. R. Moore and R. V. Shannon, “Beyond cochlear implants: awakening the deafened brain,” Nature Neuroscience, vol. 12, no. 6, pp. 686–691, 2009.
[14]
A. J. King and I. Nelken, “Unraveling the principles of auditory cortical processing: can we learn from the visual system?” Nature Neuroscience, vol. 12, no. 6, pp. 698–701, 2009.
[15]
A. Destexhe, “Intracellular and computational evidence for a dominant role of internal network activity in cortical computations,” Current Opinion in Neurobiology, vol. 21, no. 5, pp. 717–725, 2011.
[16]
S. Zeki and J. Stutters, “Functional specialization and generalization for grouping of stimuli based on colour and motion,” NeuroImage, vol. 73, pp. 156–166, 2013.
[17]
D. H. Hubel and T. N. Wiesel, “Receptive fields, binocular interaction and functional architecture in the cat's visual cortex,” The Journal of Physiology, vol. 160, pp. 106–154, 1962.
[18]
M. A. Paradiso, “A theory for the use of visual orientation information which exploits the columnar structure of striate cortex,” Biological Cybernetics, vol. 58, no. 1, pp. 35–49, 1988.
[19]
E. L. Bienenstock, L. N. Cooper, and P. W. Munro, “Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex,” The Journal of Neuroscience, vol. 2, no. 1, pp. 32–48, 1982.
[20]
G. B. Stanley, J. Jin, Y. Wang, et al., “Visual orientation and directional selectivity through thalamic synchrony,” The Journal of Neuroscience, vol. 32, no. 26, pp. 9073–9088, 2012.
[21]
E. Chicca, A. M. Whatley, P. Lichtsteiner et al., “A multichip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity,” IEEE Transactions on Circuits and Systems I, vol. 54, no. 5, pp. 981–993, 2007.
[22]
P. E. Hart, “How the Hough transform was invented [DSP History],” IEEE Signal Processing Magazine, vol. 26, no. 6, pp. 18–22, 2009.
[23]
F. Klefenz, K.-H. Noffz, W. Conen, R. Zoz, A. Kugel, and R. Manner, “Track recognition in 4μs by a systolic trigger processor using a parallel Hough transform,” IEEE Transactions on Nuclear Science, vol. 40, no. 4, pp. 688–691, 1993.
[24]
A. Epstein, G. U. Paul, B. Vettermann, C. Boulin, and F. Klefenz, “A parallel systolic array ASIC for real-time execution of the Hough transform,” IEEE Transactions on Nuclear Science, vol. 49, no. 2, pp. 339–346, 2002.
[25]
S. Kawakami and H. Okamoto, “A cell model for the detection of local image motion on the magnocellular pathway of the visual cortex,” Vision Research, vol. 36, no. 1, pp. 117–147, 1996.
[26]
L. K. Scheffer, “Design tools for artificial nervous systems,” in Proceedings of the 49th ACM/EDAC/IEEE Design Automation Conference, pp. 717–722, San Francisco, Calif, USA, 2012.
[27]
R. Brette, M. Rudolph, T. Carnevale et al., “Simulation of networks of spiking neurons: a review of tools and strategies,” Journal of Computational Neuroscience, vol. 23, no. 3, pp. 349–398, 2007.
[28]
S. Cheirdaris, D. Nikelshpur, C. Tappert et al., “A character recognition study using a biologically plausible neural network of the mammalian visual system,” in Proceedings of the Michael L. Gargano 9th Annual Student/Faculty Research Day, Pace University, May 2011.
[29]
L. N. Long and A. Gupta, “Biologically-inspired spiking neural networks with Hebbian learning for vision processing,” in Proceedings of the 46th AIAA Aerospace Sciences Meeting and Exhibit, pp. 1–17, Reno, Nev, January 2008, AIAA Paper No. 2008-0885.
[30]
Q. Wu, T. M. McGinnity, L. Maguire, G. D. Valderrama-Gonzalez, and J. Cai, “Detection of straight lines using a spiking neural network model,” in Proceedings of the 5th International Conference on Natural Computation (ICNC '09), pp. 385–389, IEEE Computer Society, August 2009.
[31]
A. Brückmann, F. Klefenz, and A. Wünsche, “A neural net for 2D-slope and sinusoidal shape detection,” International Scientific Journal of Computing, vol. 3, no. 1, pp. 21–26, 2004.
[32]
F. Klefenz, P. Husar, D. Krenzer, and A. Hess, “Real-time calibration-free autonomous eye tracker,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '10), pp. 762–765, March 2010.
[33]
J. Conradt, R. Berner, M. Cook, and T. Delbruck, “An embedded AER dynamic vision sensor for low-latency pole balancing,” in Proceedings of the 12th IEEE International Conference on Computer Vision Workshops (ICCV '09), pp. 780–785, October 2009.
[34]
H. Jia, N. L. Rochefort, X. Chen, and A. Konnerth, “Dendritic organization of sensory input to cortical neurons in vivo,” Nature, vol. 464, no. 7293, pp. 1307–1312, 2010.
[35]
M. London and M. H？usser, “Dendritic computation,” Annual Review of Neuroscience, vol. 28, pp. 503–532, 2005.
[36]
X. Chen, U. Leischner, N. L. Rochefort, I. Nelken, and A. Konnerth, “Functional mapping of single spines in cortical neurons in vivo,” Nature, vol. 475, no. 7357, pp. 501–505, 2011.
[37]
T. Branco and M. H？usser, “The single dendritic branch as a fundamental functional unit in the nervous system,” Current Opinion in Neurobiology, vol. 20, no. 4, pp. 494–502, 2010.
[38]
G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, no. 7, pp. 1527–1554, 2006.
[39]
Y. Ikegaya, G. Aaron, R. Cossart et al., “Synfire chains and cortical songs: temporal modules of cortical activity,” Science, vol. 304, no. 5670, pp. 559–564, 2004.
[40]
M. N. Havenith, S. Yu, J. Biederlack, N.-H. Chen, W. Singer, and D. Nikolic, “Synchrony makes neurons fire in sequence, and stimulus properties determine who is ahead,” The Journal of Neuroscience, vol. 31, no. 23, pp. 8570–8584, 2011.
[41]
C. Kayser, M. A. Montemurro, N. K. Logothetis, and S. Panzeri, “Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns,” Neuron, vol. 61, no. 4, pp. 597–608, 2009.
[42]
A. D. Huberman, M. B. Feller, and B. Chapman, “Mechanisms underlying development of visual maps and receptive fields,” Annual Review of Neuroscience, vol. 31, pp. 479–509, 2008.
[43]
A. Nere, U. Olcese, D. Balduzzi, and G. A. Tononi, “Neuromorphic architecture for object recognition and motion anticipation using burst-STDP,” PLoS ONE, vol. 7, no. 5, Article ID e36958, 2012.
[44]
C. Pandarinath, I. Bomash, J. D. Victor, G. T. Prusky, W. W. Tschetter, and S. Nirenberg, “A novel mechanism for switching a neural system from one state to another,” Frontiers in Computational Neuroscience, vol. 4, article 2, pp. 1–18, 2010.
[45]
H. Markram, W. Gerstner, and P. J. Sjostrom, “A history of spike-timing-dependent plasticity,” Frontiers in Synaptic Neuroscience, vol. 3, article 4, 2011.
[46]
H. Markram, J. Lübke, M. Frotscher, and B. Sakmann, “Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs,” Science, vol. 275, no. 5297, pp. 213–215, 1997.
[47]
J. F. Hunzinger, V. H. Chan, and R. C. Froemke, “Learning complex temporal patterns with resource-dependent spike timing-dependent plasticity,” Journal of Neurophysiology, vol. 108, pp. 551–566, 2012.
[48]
A. Williamson, L. Schumann, L. Hiller, et al., “Synaptic behavior and STDP of asymmetric nanoscale memristors in biohybrid systems,” Nanoscale, vol. 5, pp. 7297–7303, 2013.
[49]
S. P. Adhikari, C. Yang, H. Kim, and L. O. Chua, “Memristor bridge synapse-based neural network and its learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 9, pp. 1426–1435, 2012.
[50]
K. Dhoble, N. Nuntalid, G. Indiveri, and N. Kasabov, “Online spatio-temporal pattern recognition with evolving spiking neural networks utilising address event representation, rank order, and temporal spike learning,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '12), pp. 554–560.
[51]
E. M. Izhikevich, “Polychronization: computation with spikes,” Neural Computation, vol. 18, no. 2, pp. 245–282, 2006.
[52]
O. Bichler, D. Querlioz, S. J. Thorpe, J.-P. Bourgoin, and C. Gamrat, “Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity,” Neural Networks, vol. 32, pp. 339–348, 2012.
[53]
A. A. Lazar and E. A. Pnevmatikakis, “Video time encoding machines,” IEEE Transactions on Neural Networks, vol. 22, no. 3, pp. 461–473, 2011.
[54]
D. A. Butts, G. Desbordes, C. Weng, J. Jin, J.-M. Alonso, and G. B. Stanley, “The episodic nature of spike trains in the early visual pathway,” Journal of Neurophysiology, vol. 104, no. 6, pp. 3371–3387, 2010.
[55]
T. Masquelier, “Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model,” Journal of Computational Neuroscience, vol. 32, pp. 425–441, 2012.
[56]
Y. Kozorovitskiy, A. Saunders, C. A. Johnson, B. B. Lowell, and B. L. Sabatini, “Recurrent network activity drives striatal synaptogenesis,” Nature, vol. 485, pp. 646–650, 2012.
[57]
Y. Sasai, “Grow your own eye,” Scientific American, vol. 307, pp. 44–49, 2012.
[58]
J. Mariani, M. V. Simonini, D. Palejev, et al., “Modeling human cortical development in vitro using induced pluripotent stem cells,” Proceedings of the National Academy of Sciences of the United States of America, vol. 109, no. 31, pp. 12770–12775, 2012.
[59]
J. D. Rolston, D. A. Wagenaar, and S. M. Potter, “Precisely timed spatiotemporal patterns of neural activity in dissociated cortical cultures,” Neuroscience, vol. 148, no. 1, pp. 294–303, 2007.
[60]
J.-J. Sun, W. Kilb, and H. J. Luhmann, “Self-organization of repetitive spike patterns in developing neuronal networks in vitro,” European Journal of Neuroscience, vol. 32, no. 8, pp. 1289–1299, 2010.
[61]
P. Bonifazi, M. Goldin, M. A. Picardo et al., “GABAergic hub neurons orchestrate synchrony in developing hippocampal networks,” Science, vol. 326, no. 5958, pp. 1419–1424, 2009.
[62]
R. Min and T. Nevian, “Astrocyte signaling controls spike timing-dependent depression at neocortical synapses,” Nature Neuroscience, vol. 15, no. 5, pp. 746–753, 2012.
[63]
A. Dityatev and D. A. Rusakov, “Molecular signals of plasticity at the tetrapartite synapse,” Current Opinion in Neurobiology, vol. 21, no. 2, pp. 353–359, 2011.
[64]
D. J. Rossi, “Astrocytes join the plasticity party,” Nature Neuroscience, vol. 15, no. 5, pp. 649–651, 2012.
[65]
R. Min, M. Santello, and T. Nevian, “The computational power of astrocyte mediated synaptic plasticity,” Frontiers in Computational Neuroscience, vol. 6, article 93, 2012.
[66]
A. Hierlemann, U. Frey, S. Hafizovic, and F. Heer, “Growing cells atop microelectronic chips: interfacing electrogenic cells in vitro with CMOS-based microelectrode arrays,” Proceedings of the IEEE, vol. 99, no. 2, pp. 252–284, 2011.
[67]
G.-Q. Bi and M.-M. Poo, “Distributed synaptic modification in neural networks induced by patterned stimulation,” Nature, vol. 401, no. 6755, pp. 792–796, 1999.
[68]
H. A. Johnson, A. Goel, and D. V. Buonomano, “Neural dynamics of in vitro cortical networks reflects experienced temporal patterns,” Nature Neuroscience, vol. 13, no. 8, pp. 917–919, 2010.
[69]
A. Vishwanathan, G.-Q. Bi, and H. C. Zeringue, “Ring-shaped neuronal networks: a platform to study persistent activity,” Lab on a Chip, vol. 11, no. 6, pp. 1081–1088, 2011.
[70]
A. N. Ide, A. Andruska, M. Boehler, B. C. Wheeler, and G. J. Brewer, “Chronic network stimulation enhances evoked action potentials,” Journal of Neural Engineering, vol. 7, no. 1, Article ID 016008, 2010.
[71]
M. E. Ruaro, P. Bonifazi, and V. Torre, “Toward the neurocomputer: image processing and pattern recognition with neuronal cultures,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 3, pp. 371–383, 2005.
[72]
A. Williamson, S. Singh, U. Fernekorn, and A. Schober, “The future of the patient-specific Body-on-a-chip,” Lab on a Chip, vol. 13, no. 18, pp. 3471–3480, 2013.
[73]
A. Schober, U. Fernekorn, S. Singh et al., “Mimicking the biological world: methods for the 3D structuring of artificial cellular environments,” Engineering in Life Sciences, vol. 13, no. 4, pp. 352–367, 2013.
