Class-wise Thresholding for Detecting Out-of-Distribution Data Matteo Guarrera 1,2 , Baihong Jin 1 , Tung-Wei Lin 1 , Maria Zuluaga 2 , Yuxin Chen 3 , Alberto Sangiovanni-Vincentelli 1 1 University of California, Berkeley 2 EURECOM 3 University of Chicago Abstract We consider the problem of detecting Out-of- Distribution (OoD) input data when using deep neural networks, and we propose a simple yet effective way to improve the robustness of several popular OoD detection methods against label shift. Our work is motivated by the observation that most existing OoD detection algorithms consider all training/test data as a whole, regardless of which class entry each input activates (inter-class dif- ferences). Through extensive experimentation, we have found that such practice leads to a detector whose per- formance is sensitive and vulnerable to label shift. To address this issue, we propose a class-wise thresholding scheme that can apply to most existing OoD detection algorithms and can maintain similar OoD detection per- formance even in the presence of label shift in the test distribution. Introduction With the recent advancement in deep learning, image clas- sification has shown great performance improvement under well-controlled settings where the test data are clean and sam- pled from the same distribution as the training data. However, the deployment of deep learning models in the real world is still full of unknowns. More often than not, well-trained models can come across Out-of-Distribution (OoD) data that are sampled from a different distribution than the one used for training. For example, objects that do not belong to any of the classes in the training data (i.e., OoD inputs) can appear at test time. Faced with OoD inputs, deep learning-based classifiers may render unpredictable behaviors and often tend to make overly confident decisions (Nguyen, Yosinski, and Clune 2015); to address this issue, many previous works (Liu et al. 2020; Liang, Li, and Srikant 2020; Hendrycks, Mazeika, and Dietterich 2019; Hendrycks and Gimpel 2018; Lee et al. 2018) have been dedicated to detecting such OoDs inputs. Therefore, in safety-critical applications such as healthcare and autonomous driving (Joseph et al. 2021), the classifier should have the ability to yield control to humans upon com- ing across such inputs, instead of making incorrect predic- tions silently. Copyright © 2022, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Among the plethora of works on OoD detection, almost all previous literature focuses on improving the detection performance on various OoD test sets. Their experiment setups implicitly assume that the training and the test In- Distribution (ID) data follow the same distribution (a.k.a. distribution shift), so that the false alarm rates at test time will stay at the same level. However, it is often not the case in real-world settings, and the distribution shift may result in increased or decreased number of false alarms on the ID data, which can lead to economic loss (additional costs to address these false alarms). Worse still, malicious attackers can exploit this weakness to launch attacks that cause an over- flow of false alarms for certain classes, which eventually can lower the sensitivity of the detection system against actual OoD inputs (due to the excessive number of false alarms in- jected by attackers). The above-mentioned issues are seldom discussed in prior literature on OoD detection but is indeed vital for real-world deployment of such systems. In this paper, we specifically target OoD detection algo- rithms built upon supervised multi-class classifiers, and ad- dress the above-mentioned challenges in the context of label shift (a special type of distribution shift) by using a novel thresholding scheme. Our approach is applicable even if the internal structure and parameters of the classifier are invisible (i.e., black-box models). The contributions of this paper are three-fold: • We identify a problem that make many existing OoD detection algorithms vulnerable to test-time label shift. • We propose a simple yet effective thresholding scheme to address the challenge, and show empirically that our solu- tion can be used as a plugin amendment to any existing OoD systems with a class-wise score function. • Using our novel thresholding scheme, we also assess the performance limit of several learning-based OoD detec- tors, and compare them with non-learning-based ones. The study provides some guidance on how to navigate the design space of OoD detection systems. Preliminaries We consider the OoD detection problem in supervised multi- class classification settings; our goal is to identify whether a data point (image) x comes from the distribution D in-dist which the development set data are sampled from. Let us de-