Better Counterfactual Model Reasoning with Submodular Quadratic Component Models

In this work, we propose a new heuristic for component modeling---the task of building a model that estimates the effect of model components on model behavior. Empirically, we show that even at scale, actively learning a quadratic model instead of a linear model increases accuracy and decreases sample complexity. Prior work has favored a linear model, because outside of accuracy, interpretability is a key desiderata for a component model. By exploiting properties of the degree 2 Fourier representation, we derive an individual influence for each point, that strictly generalizes the coefficient in a linear data model. This notion corresponds to the discrete derivative of the function at a given point, and has the benefit that it incorporates information about the rest of the dataset into the individual influence estimate. We also introduce the idea of enforcing submodularity, which theoretically can allow for better and nontrivial optimization for interesting counterfactual reasoning tasks with set cardinality constraints.