Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

We present a novel deep learning framework for probabilistic pixel-wise semantic segmentation, which we term Bayesian SegNet. Pixel-wise semantic segmentation is an important step for visual scene understanding. It is a complex task requiring knowledge of support relationships and contextual information, as well as visual appearance. Our contribution is a practical system which is able to predict pixel-wise class labels with a measure of model uncertainty. We achieve this by Monte Carlo sampling with dropout at test time to generate a posterior distribution of pixel class labels. We show this Bayesian neural network provides a significant performance improvement in segmentation, with no additional parameterisation. We set a new benchmark with state-of-the-art performance on both the indoor SUN Scene Understanding and outdoor CamVid driving scenes datasets. Bayesian SegNet also performs competitively on Pascal VOC 2012 object segmentation challenge. For our web demo and source code, see http://mi.eng.cam.ac.uk/projects/segnet/