Predictive Modeling in the Presence of Nuisance-Induced Spurious Correlations Aahlad Puli 1 Lily H. Zhang 2 Eric K. Oermann 2,3,4 Rajesh Ranganath 1,2,5 1 Department of Computer Science, New York University 2 Center for Data Science, New York University 3 Department of Neurosurgery, Langone Health, New York University 4 Department of Radiology, Langone Health, New York University 5 Department of Population Health, Langone Health, New York University Abstract In many prediction problems, spurious correlations are induced by a changing relationship between the label and a nuisance variable that is also correlated with the covariates. For example, in classifying an- imals in natural images, the background, which is the nuisance, can predict the type of animal. This nuisance-label relationship does not always hold, and the performance of a model trained under one such relationship may be poor on data with a different nuisance-label relationship. To build predictive models that perform well regardless of the nuisance-label relationship, we develop Nuisance-Randomized Distillation (NURD). We first define the nuisance-varying family, a set of distributions that differ only in the nuisance-label relationship. We then introduce the nuisance-randomized distribution, a distribution where the nuisance and the label are independent. Under this distribution, we define the set of repre- sentations such that conditioning on any member, the nuisance and the label remain independent. We prove that the representations in this set always perform better than chance, while representations out- side of this set may not. NURD finds a representation from this set that is most informative of the label under the nuisance-randomized distribution, and we prove that this representation achieves the highest performance within the set on every distribution in the nuisance-varying family. We evaluate NURD on several tasks including chest X-ray classification where, using non-lung patches as the nuisance, NURD produces models that predict pneumonia under strong spurious correlations. 1 Introduction Spurious correlations are relationships between the label and the covariates that are prone to change be- tween training and test distributions [1]. Predictive models that exploit spurious correlations can perform worse than even predicting without covariates on the test distribution [2]. Detecting spurious correla- tions requires more than the training distribution because any fixed dataset has a fixed label-covariate relationship. Often, spurious correlations are discovered by noticing different relationships across mul- tiple distributions between the label and nuisance factors correlated with the covariates. We call these nuisance-induced spurious correlations. For example, in classifying cows vs. penguins, typical images have cows appear on grasslands and pen- guins appear near snow due to natural habitats [2, 3], but these animals can be photographed outside their habitats. In classifying hair color from celebrity faces on CelebA [4], gender is correlated with the hair color. This relationship may not hold in different countries [5]. In language, sentiment of a movie review determines the types of words used in the review to convey attitudes and opinions. However, di- rectors’ names appear in the reviews and are correlated with positive sentiment in time periods where Corresponding email: aahlad@nyu.edu 1 arXiv:2107.00520v3 [cs.LG] 18 Oct 2021