We introduce Logical Neural Networks, a new neuro-symbolic framework which creates a 1-to-1 correspondence between a modified form of the standard differentiable neuron and a logic gate in a weighted form of real-valued logic. The key modifications of the neuron model are a) the addition of an ability to perform inference in the reverse direction in order to perform the equivalent of logical inference rules such as modus ponens, within the message-passing paradigm of neural networks, and b) learning with constraints on the weights in order to enforce logical behavior plus a new kind loss term, contradiction loss, which maximizes logical consistency in the face of imperfect and inconsistent knowledge. The result differs significantly from other neuro-symbolic ideas in that 1) the model is fully disentangled and understandable since every neuron has a meaning, 2) the model can perform both classical logical deduction and its real-valued generalization (which allows for the representation and propagation of uncertainty) exactly, as special cases, as opposed to approximately as in nearly all other approaches, and 3) the model is compositional and modular, allowing for fully reusable knowledge across tasks.

We introduce Logical Neural Networks, a new neuro-symbolic framework which creates a 1-to-1 correspondence between a modified form of the standard differentiable neuron and a logic gate in a weighted form of real-valued logic. The key modifications of the neuron model are a) the addition of an ability to perform inference in the reverse direction in order to perform the equivalent of logical inference rules such as modus ponens, within the message-passing paradigm of neural networks, and b) learning with constraints on the weights in order to enforce logical behavior plus a new kind loss term, contradiction loss, which maximizes logical consistency in the face of imperfect and inconsistent knowledge. The result differs significantly from other neuro-symbolic ideas in that 1) the model is fully disentangled and understandable since every neuron has a meaning, 2) the model can perform both classical logical deduction and its real-valued generalization (which allows for the representation and propagation of uncertainty) exactly, as special cases, as opposed to approximately as in nearly all other approaches, and 3) the model is compositional and modular, allowing for fully reusable knowledge across tasks.

We introduce Logical Neural Networks, a new neuro-symbolic framework which creates a 1-to-1 correspondence between a modified form of the standard differentiable neuron and a logic gate in a weighted form of real-valued logic. The key modifications of the neuron model are a) the addition of an ability to perform inference in the reverse direction in order to perform the equivalent of logical inference rules such as modus ponens, within the message-passing paradigm of neural networks, and b) learning with constraints on the weights in order to enforce logical behavior plus a new kind loss term, contradiction loss, which maximizes logical consistency in the face of imperfect and inconsistent knowledge. The result differs significantly from other neuro-symbolic ideas in that 1) the model is fully disentangled and understandable since every neuron has a meaning, 2) the model can perform both classical logical deduction and its real-valued generalization (which allows for the representation and propagation of uncertainty) exactly, as special cases, as opposed to approximately as in nearly all other approaches, and 3) the model is compositional and modular, allowing for fully reusable knowledge across tasks.

Knowledge base question answering (KBQA) is an important task in Natural Language Processing. Existing approaches face significant challenges including complex question understanding, necessity for reasoning, and lack of large training datasets. In this work, we propose a semantic parsing and reasoning-based Deep Thinking Question Answering (DTQA) system, that leverages (1) Abstract Meaning Representation (AMR) parses for task-independent question understanding; (2) a novel path-based approach to transform AMR parses into candidate logical queries that are aligned to the KB; (3) a neuro-symbolic reasoner called Logical Neural Network (LNN) that executes logical queries and reasons over KB facts to provide an answer; (4) system of systems approach, which integrates multiple, reusable modules that are trained specifically for their individual tasks (e.g. semantic parsing, entity linking, and relationship linking) and do not require end-to-end training data. DTQA achieves state-of-the-art performance on QALD-9 and LC-QuAD 1.0. DTQA's novelty lies in its modular neuro-symbolic architecture and its task-general approach to interpreting natural language questions

Knowledge base question answering (KBQA) is an important task in Natural Language Processing. Existing approaches face significant challenges including complex question understanding, necessity for reasoning, and lack of large training datasets. In this work, we propose a semantic parsing and reasoning-based Deep Thinking Question Answering (DTQA) system, that leverages (1) Abstract Meaning Representation (AMR) parses for task-independent question understanding; (2) a novel path-based approach to transform AMR parses into candidate logical queries that are aligned to the KB; (3) a neuro-symbolic reasoner called Logical Neural Network (LNN) that executes logical queries and reasons over KB facts to provide an answer; (4) system of systems approach, which integrates multiple, reusable modules that are trained specifically for their individual tasks (e.g. semantic parsing, entity linking, and relationship linking) and do not require end-to-end training data. DTQA achieves state-of-the-art performance on QALD-9 and LC-QuAD 1.0. DTQA's novelty lies in its modular neuro-symbolic architecture and its task-general approach to interpreting natural language questions

Humans are capable of learning visual concepts by jointly understanding vision and language. Imagine that someone with no prior knowledge of colors is presented with the images of the red and green objects, paired with descriptions. They can easily identify the difference in objects' visual appearance (in this case, color), and align it to the corresponding words. This intuition motivates the use of image-text pairs to facilitate automated visual concept learning and language acquisition. In the talk, I will present recent progress on neuro-symbolic models for visual concept learning and reasoning. These models learn visual concepts and their association with symbolic representations of language, only by looking at images and reading paired natural language texts.

Humans are capable of reasoning about physical phenomena by inferring laws of physics from a very limited set of observations. The inferred laws can potentially depend on unobserved properties, such as mass, texture, charge, etc. This sample-efficient physical reasoning is considered a core domain of human common-sense knowledge and hints at the existence of a physics engine in the head. In this paper, we propose a Bayesian symbolic framework for learning sample-efficient models of physical reasoning and prediction, which are of special interests in the field of intuitive physics. In our framework, the environment is represented by a top-down generative model with a collection of entities with some known and unknown properties as latent variables to capture uncertainty. The physics engine depends on physical laws which are modeled as interpretable symbolic expressions and are assumed to be functions of the latent properties of the entities interacting under simple Newtonian physics. As such, learning the laws is then reduced to symbolic regression and Bayesian inference methods are used to obtain the distribution of unobserved properties. These inference and regression steps are performed in an iterative manner following the expectation–maximization algorithm to infer the unknown properties and use them to learn the laws from a very small set of observations. We demonstrate that on three physics learning tasks that compared to the existing methods of learning physics, our proposed framework is more data-efficient, accurate and makes joint reasoning and learning possible.

In this talk, I will demonstrate how to combine the power of deep neural networks and classic symbolic AI to deal with challenges in video understanding. I will showcase the application of these methods to problems such as temporal and causal reasoning in videos and music generation from videos.

Automated theorem provers have traditionally relied on manually tuned heuristics to guide how they perform proof search. Deep reinforcement learning has been proposed as a way to obviate the need for such heuristics, however, its deployment in automated theorem proving remains a challenge. We introduce TRAIL, a system that applies deep reinforcement learning to saturation-based theorem proving. TRAIL leverages (a) a novel neural representation of the state of a theorem prover and (b) a novel characterization of the inference selection process in terms of an attention-based action policy. We show through systematic analysis that these mechanisms allow TRAIL to significantly outperform previous reinforcement-learning-based theorem provers on two benchmark datasets for first-order logic automated theorem proving (proving around15% more theorems).

Intuitively, compositional generalization is about combining things you know in new ways to solve a task, with little or no additional training. People can do this very well, even young children, but neural networks struggle. This is important because the cost to generalize without such a mechanism can be exponential in the number of variables in the task representation. Recent work evaluating neural networks for compositional generalization has mostly focused on natural language translation. In this presentation I'll review some of that work and talk about our experiments on composition in a completely geometric domain with no language but rather concepts specified in a first-order logical language which has richer constraints than those imposed by context free grammars. Our preliminary results indicate that neural nets do not generalize compositionally in this setting either.