Timezone: »
Despite the success achieved in neural abstractive summarization based on pre-trained language models, one unresolved issue is that the generated summaries are not always faithful to the input document.There are two possible causes of the unfaithfulness problem: (1) the summarization model fails to understand or capture the gist of the input text, and (2) the model over-relies on the language model to generate fluent but inadequate words.In this work, we propose a Faithfulness Enhanced Summarization model (FES), which is designed for addressing these two problems and improving faithfulness in abstractive summarization.For the first problem, we propose to use question-answering (QA) to examine whether the encoder fully grasps the input document and can answer the questions on the key information in the input. The QA attention on the proper input words can also be used to stipulate how the decoder should attend to the source.For the second problem, we introduce a max-margin loss defined on the difference between the language and the summarization model, aiming to prevent the overconfidence of the language model.Extensive experiments on two benchmark summarization datasets, CNN/DM and XSum, demonstrate that our model significantly outperforms strong baselines.The evaluation of factual consistency also shows that our model generates more faithful summaries than baselines.
Author Information
Xiuying Chen (KAUST)
Mingzhe Li (Peking University)
Xin Gao (KAUST)
Xiangliang Zhang (University of Notre Dame)
I am an Associate Professor in the department of Computer Science and Engineering at University of Notre Dame, where I am leading a Machine Intelligence and kNowledge Engineering (MINE) group. My research broadly addresses ways that enable computer machines to learn by the use of diverse types of data. Specifically, I am interested in designing machine learning algorithms for learning from complex and large-scale streaming data and graph data, with applications to recommendation systems, knowledge discovery, and natural language understanding. More information can be found in the publications grouped by research problems, or the full list of over 190 peer-reviewed papers. I was invited to deliver an Early Career Spotlight talk at IJCAI-ECAI 2018. In 2010, I received a Chinese government award for outstanding self-financed students abroad. In 2009, I was awarded the European Research Consortium for Informatics and Mathematics (ERCIM) Alain Bensoussan Fellowship. I regularly serve on the Program Committee for premier conferences like SIGKDD (Senior PC), AAAI (Area Chair, Senior PC), IJCAI (Area Chair, Senior PC), etc. I also serve as Editor-in-Chief of ACM SIGKDD Explorations, associated editor for IEEE Transactions on Dependable and Secure Computing (TDSC) and Information Sciences. Prior to joining the University of Notre Dame, I was an Associate Professor in Computer Science at KAUST, Saudi Arabia. I completed my Ph.D. degree in computer science from INRIA-University Paris-Sud, France, in July 2010. I received my master and bachelor degrees from Xi’an Jiaotong University, China.
More from the Same Authors
-
2022 : MWP-BERT: A Numeracy-augmented Pre-trained Encoder for Math Word Problems »
Zhenwen Liang · Jipeng ZHANG · Lei Wang · Wei QIN · Jie Shao · Xiangliang Zhang -
2022 : NOSMOG: Learning Noise-robust and Structure-aware MLPs on Graphs »
Yijun Tian · Chuxu Zhang · Zhichun Guo · Xiangliang Zhang · Nitesh Chawla -
2023 Poster: Lift Yourself Up: Retrieval-augmented Text Generation with Self-Memory »
Xin Cheng · Di Luo · Xiuying Chen · Lemao Liu · Dongyan Zhao · Rui Yan -
2023 Poster: What indeed can GPT models do in chemistry? A comprehensive benchmark on eight tasks »
Taicheng Guo · kehan Guo · Bozhao Nan · Zhenwen Liang · Zhichun Guo · Nitesh Chawla · Olaf Wiest · Xiangliang Zhang -
2020 Poster: Temporal Positive-unlabeled Learning for Biomedical Hypothesis Generation via Risk Estimation »
Uchenna Akujuobi · Jun Chen · Mohamed Elhoseiny · Michael Spranger · Xiangliang Zhang