Deep learning and probabilistic programming are domains that have a lot in common in certain respects; both rely on software abstractions to enable iterative model development.

In this talk we discuss how we can integrate techniques from both domains in problems where we would like to use priors to induce structured representations. To do so, we employ reweighted wake-sleep methods, which combine importance sampling methods (which have been operationalized in probabilistic programming) with variational methods for learning proposals.

To enable a more iterative design of these methods, we introduce compositional constructs, which we refer to as combinators, which serve to define both model structure and evaluation strategies that correspond to different importance sampling schemes. Together these constructs define a path towards a more compositional design of variational methods that are correct by construction.