%0 Journal Article
%T Over-Sampling in a Deep Neural Network
%A Andrew J. R. Simpson
%J Computer Science
%D 2015
%I arXiv
%X Deep neural networks (DNN) are the state of the art on many engineering problems such as computer vision and audition. A key factor in the success of the DNN is scalability - bigger networks work better. However, the reason for this scalability is not yet well understood. Here, we interpret the DNN as a discrete system, of linear filters followed by nonlinear activations, that is subject to the laws of sampling theory. In this context, we demonstrate that over-sampled networks are more selective, learn faster and learn more robustly. Our findings may ultimately generalize to the human brain.
%U http://arxiv.org/abs/1502.03648v1