On the Equivalence of Bi-Level Optimization and Game-Theoretic Formulations of Invariant Risk Minimization

Abstract

The standard risk minimization paradigm of machine learning is brittle when operating in environments whose test distributions are different from the training distribution due to spurious correlations. Training on data from many environments and finding invariant predictors reduces the effect of spurious features by concentrating models on features that have a causal relationship with the outcome. Such invariant risk minimization was posed as a bi-level optimization problem by Arjovsky et al. (2019). In this work, we pose invariant risk minimization as finding the Nash equilibrium of an ensemble game among several environments and show that the set of Nash equilibria for the proposed game are equivalent to the set of invariant predictors obtained by the bi-level optimization even with nonlinear classifiers and transformations