Uncertainty in Deep Learning Models

2020-01-26

This article was published on the Google AI Blog by Google research scientist Jasper Snoek and Google research engineer Zachary Nado, and translated by atyun.


In an ideal world, deploying machine learning (ML) methods like deep learning would allow us to make predictions on data that follows the same distribution as the training data. However, the reality can be quite different: camera lenses get blurry, sensor performance degrades, and popular online topics change, which can lead to differences between the data distribution used to train a model and the data distribution it is applied to, a situation known as covariate shift. For example, it has recently been observed that deep learning models trained to detect pneumonia in chest X-rays can achieve very different accuracies when evaluated on unknown hospital data, partly due to subtle differences in image acquisition and processing.In our work at NeurIPS 2019, we benchmarked the uncertainty of state-of-the-art deep learning models as they are exposed to both changing data distributions and distributional mismatches. In this work, we considered various input modalities, including images, text, and online advertising data, to expose these deep learning models to increasingly diverse test data, while carefully analyzing the behavior of their predictive probabilities. We also compared various methods for improving model uncertainty to see which strategies work best under distributional shifts.



What is Distributional Shift?


Deep learning models provide probabilities for each prediction, representing the model's confidence or uncertainty. Therefore, when data falls outside the range of the original training dataset, they can express what they don't know and accordingly abstain from making predictions. In the case of covariate shift, uncertainty ideally increases proportionally with any decrease in accuracy. A more extreme case is when the data does not have any representation in the training set, known as out-of-distribution (OOD) data. For example, consider what would happen if an animal image classifier is shown an image of an airplane. Would the model confidently predict it incorrectly or assign low probabilities to each class? In a recent article, we discussed methods developed to identify such OOD examples. In contrast, in this work, we analyze the predictive uncertainty of models and provide examples of distributional mismatches and transferred examples to see if model probabilities reflect their ability to make predictions on such data.



Quantifying the Quality of Uncertainty


What does it mean for one model to represent its uncertainty better than another? While this can be a subtle question often defined by downstream tasks, there are still some methods to quantitatively evaluate the overall quality of probability predictions. For example, the meteorological community has carefully considered this question and developed a set of appropriate scoring rules that probabilistic weather forecasts should satisfy in order to be well-calibrated while still improving accuracy. We applied some appropriate scoring rules, such as the Brier score and negative log-likelihood (NLL), as well as more intuitive heuristic methods like expected calibration error (ECE), to understand how different ML models handle uncertainty under dataset shifts.



Experiments


We analyzed the impact of dataset shifts on the uncertainty of various data modalities, including images, text, online advertising data, and genomics. For example, we demonstrated the impact of dataset shifts on ImageNet, a popular image understanding benchmark. ImageNet involves classifying over a million images into 1,000 different categories. Some consider this challenge to be largely solved, and more difficult variants have been developed, such as Corrupted Imagenet (or Imagenet-C), where the data has been augmented with 16 different levels of actual corruption (five different intensities each).


We explore the behavior of model uncertainty as image perturbation intensity increases in Corrupted Imagenet. Shown here are examples of image corruptions for each type in intensity level 3 (out of 5).


We used these corrupted images as examples of shifted data and studied the predictive probabilities of deep learning models when exposed to varying intensities. Below, we show box plots of accuracy and ECE obtained for each corruption level (including the uncorrupted test data), summarizing all corruption types in ImageNet-C. Each color represents a different type of model - a "vanilla" deep neural network used as a baseline, four uncertainty methods (margin, temperature scaling, and our last-layer methods), and the ensemble method.


Accuracy (top) and expected calibration error (ECE) (bottom; lower is better) as the intensity of dataset shift increases on ImageNet-C. We observe that the decrease in accuracy does not reflect an increase in model uncertainty, indicating a degradation in both accuracy and ECE.


As the intensity of shift increases, the accuracy bias of different corruption methods for each model increases (box size increases), as expected, and overall accuracy decreases. Ideally, this would be reflected in an increase in model uncertainty, thus keeping the expected calibration error (ECE) constant. However, looking at the lower graph of ECE, we find that this is not the case, and calibration is often affected as well. We observe similar worsening trends in Brier score and NLL, indicating that models do not become increasingly uncertain with shift, but rather become wrong.


A popular method for improving calibration is called temperature scaling, a variant of Platt scaling that involves smoothing predictions using performance on a held-out validation set after training. We observe that while this improves calibration on standard test data, it often makes shifted data worse! Therefore, practitioners applying this technique should be cautious about assigning shifts.


Fortunately, one method elegantly reduces uncertainty more than others. Deep ensembles (in green) can average the predicted values of selected models, each with different initializations, which is a simple strategy that significantly improves robustness to shifts and outperforms all other tested methods.



Summary and Recommended Best Practices


In our paper, we explore the behavior of state-of-the-art models under dataset shifts across images, text, online advertising data, and genomics. Our findings are generally consistent across these different types of data. The quality of uncertainty decreases with dataset shifts, but there is promising research to mitigate this issue. We hope that deep learning practitioners take away the following from our research:


Consider uncertainty under dataset shifts when training models.


Improving calibration and accuracy on in-distribution test sets often does not translate to improved calibration on shifted data.


Among all the methods we considered, deep ensembles are the most robust to dataset shifts, and a small ensemble size (e.g., 5) is sufficient. The effectiveness of ensembles provides an interesting avenue for improving other methods.


Improving the prediction uncertainty of deep learning models remains an active area of research in machine learning. We have released all the code and model predictions from this benchmark, hoping it will be useful to the community in driving and evaluating future work on this important topic.

RECENT AI NEWS

RECENT AI TOOLS