Timezone: »

 
MWP-BERT: A Numeracy-augmented Pre-trained Encoder for Math Word Problems
Zhenwen Liang · Jipeng ZHANG · Lei Wang · Wei QIN · Jie Shao · Xiangliang Zhang

@
in MATH-AI: Toward Human-Level Mathematical Reasoning »

Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema canprovide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM. We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.

[ Poster] 

Author Information

Zhenwen Liang (University of Notre Dame)
Jipeng ZHANG (University of Electronic Science and Technology of China)
Lei Wang (SMU)
Wei QIN (Hefei University of Technology)
Jie Shao (University of Electronic Science and Technology of China)
Jie Shao

Jie Shao received a PhD degree from The University of Queensland in 2009, and a bachelor degree from Southeast University, China in 2004, both in Computer Science. He joined the University of Electronic Science and Technology of China in June 2014. Before that, he was a Research Fellow in The University of Melbourne and The University of Queensland respectively, working on advanced database systems (spatial databases, multimedia databases, etc.). Form June 2012 to May 2014, he worked as a Research Fellow in National University of Singapore. His current research interests include big data analytics and multimedia information retrieval.

Xiangliang Zhang (University of Notre Dame)

I am an Associate Professor in the department of Computer Science and Engineering at University of Notre Dame, where I am leading a Machine Intelligence and kNowledge Engineering (MINE) group. My research broadly addresses ways that enable ​computer machines to​ learn by the use of diverse types of data. Specifically, I am interested in designing machine learning algorithms for learning from complex and large-scale streaming data and graph data, with applications to recommendation systems, knowledge discovery, and natural language understanding. More information can be found in the publications grouped by research problems, or the full list of over 190 peer-reviewed papers. I was invited to deliver an Early Career Spotlight talk at IJCAI-ECAI 2018. In 2010, I received a Chinese government award for outstanding self-financed students abroad. In 2009, I was awarded the European Research Consortium for Informatics and Mathematics (ERCIM) Alain Bensoussan Fellowship. I regularly serve on the Program Committee for premier conferences like SIGKDD (Senior PC), AAAI (Area Chair, Senior PC), IJCAI (Area Chair, Senior PC), etc. I also serve as Editor-in-Chief of ACM SIGKDD Explorations, associated editor for IEEE Transactions on Dependable and Secure Computing (TDSC) and Information Sciences. Prior to joining the University of Notre Dame, I was an Associate Professor in Computer Science at KAUST, Saudi Arabia. I completed my Ph.D. degree in computer science from INRIA-University Paris-Sud, France, in July 2010. I received my master and bachelor degrees from Xi’an Jiaotong University, China.

More from the Same Authors