Timezone: »

 
The Impact of Symbolic Representations on In-context Learning for Few-shot Reasoning
Hanlin Zhang · yifan zhang · Li Erran Li · Eric Xing

@
in Workshop on neuro Causal and Symbolic AI (nCSI) »
Event URL: https://openreview.net/forum?id=qLgQpeQX3x1 »

Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations for in-context learning. On the other hand, those reasoning tasks are usually presumed to be more approachable for symbolic programming. To make progress towards understanding in-context learning, we revisit neuro-symbolic approaches and design a model LMLP that learns from demonstrations containing logic rules and corresponding examples to iteratively reason over knowledge bases (KBs). Such a procedure makes explicit correspondence between LMs' outputs and predicates in the KBs to recover Prolog’s backward chaining algorithm. Comprehensive experiments are included to systematically compare LMLP with their natural language counterparts like ``chain-of-thought'' (CoT) in deductive and inductive reasoning settings, which demonstrates that LMLP enjoys much better efficiency and length generalization in various settings.

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Hanlin Zhang (School of Computer Science, Carnegie Mellon University)
yifan zhang (Institute of Automation, Chinese Academy of Sciences)
Li Erran Li (AWS AI, Amazon)

Li Erran Li is the head of machine learning at Scale and an adjunct professor at Columbia University. Previously, he was chief scientist at Pony.ai. Before that, he was with the perception team at Uber ATG and machine learning platform team at Uber where he worked on deep learning for autonomous driving, led the machine learning platform team technically, and drove strategy for company-wide artificial intelligence initiatives. He started his career at Bell Labs. Li’s current research interests are machine learning, computer vision, learning-based robotics, and their application to autonomous driving. He has a PhD from the computer science department at Cornell University. He’s an ACM Fellow and IEEE Fellow.

Eric Xing (Petuum Inc.)

More from the Same Authors