Learning to Act in Noisy Contexts Using Deep Proxy Learning (Invited Talk by Arthur Gretton)

We consider the problem of evaluating the expected outcome of an action or policy, using off-policy observations, where the relevant context is noisy/anonymized. This scenario might arise due to privacy constraints, data bandwidth restrictions, or both. As an example, users might wish to determine the anticipated outcome of an exercise regime, with only an incomplete view available of their fitness levels (for instance, from journaling or wearables). We will employ the recently developed tool of proxy causal learning to address this problem. In brief, two noisy views of the context are used: one prior to the user action, and one subsequent to it, and influenced by the action. This pair of views will allow us to recover the average causal effect of an action under reasonable assumptions. As a key benefit of the proxy approach, we need never explicitly model or recover the hidden context. Our implementation employs learned neural net representations for both the action and context, allowing each to be complex and high dimensional (images, text). We demonstrate the deep proxy learning method in a setting where the action is an image, and show that we outperform an autoencoder-based alternative.