Timezone: »

 
Poster
Neurosymbolic Deep Generative Models for Sequence Data with Relational Constraints
Halley Young · Maxwell Du · Osbert Bastani

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #609
in Poster Session 4 »

There has been significant recent progress designing deep generative models that generate realistic sequence data such as text or music. Nevertheless, it remains difficult to incorporate high-level structure to guide the generative process, and many such models perform well on local coherence, but less so on global coherence. We propose a novel approach for incorporating global structure in the form of relational constraints between different subcomponents of an example (e.g., lines of a poem or measures of music). Our generative model has two parts: (i) one model to generate a realistic set of relational constraints, and (ii) a second model to generate realistic data satisfying these constraints. For model (i), we propose a constrained optimization algorithm that infers the relational constraints present in the training data, and then learn a generative model based on the resulting constraint data. In our experiments, we show that our approach significantly improves over state-of-the-art in terms of capturing high-level structure in the data, while performing comparably or better in terms of low-level structure. We also show that using constrained optimization for part (ii) as well leads to increased controllability with little decrease in quality compared to pure learning-based models.

Keywords: Generative · Program Synthesis · sequence · constraint · music · poetry · neurosymbolic

Author Information

Halley Young (University of Pennsylvania)
Maxwell Du (School of Engineering and Applied Science, University of Pennsylvania)
Osbert Bastani (University of Pennsylvania)

More from the Same Authors