Timezone: »
In this work, we propose TGLS, a novel framework for unsupervised Text Generation by Learning from Search. We start by applying a strong search algorithm (in particular, simulated annealing) towards a heuristically defined objective that (roughly) estimates the quality of sentences. Then, a conditional generative model learns from the search results, and meanwhile smooth out the noise of search. The alternation between search and learning can be repeated for performance bootstrapping. We demonstrate the effectiveness of TGLS on two real-world natural language generation tasks, unsupervised paraphrasing and text formalization. Our model significantly outperforms unsupervised baseline methods in both tasks. Especially, it achieves comparable performance to strong supervised methods for paraphrase generation.
Author Information
Jingjing Li (The Chinese University of Hong Kong)
Zichao Li (Huawei Noah's Ark Lab)
Lili Mou (University of Alberta)
Xin Jiang (Huawei Noah's Ark Lab)
Michael R Lyu (CUHK)
Irwin King (Chinese University of Hong Kong)
More from the Same Authors
-
2021 : Score-based Graph Generative Model for Neutrino Events Classification and Reconstruction »
Yiming Sun · Zixing Song · Irwin King -
2022 : Individual Fairness in Dynamic Financial Networks »
Zixing Song · Yueen Ma · Irwin King -
2023 Poster: Optimal Block-wise Asymmetric Graph Construction for Graph-based Semi-supervised Learning »
Zixing Song · Yifei Zhang · Irwin King -
2023 Poster: Predicting Global Label Relationship Matrix for Graph Neural Networks under Heterophily »
Langzhang Liang · Xiangjing Hu · Zenglin Xu · Zixing Song · Irwin King -
2023 Poster: Mitigating the Popularity Bias in Graph-based Collaborative Filtering »
Yifei Zhang · Hao Zhu · yankai Chen · Zixing Song · Piotr Koniusz · Irwin King -
2023 Poster: No Change, No Gain: Empowering Graph Neural Networks with Expected Model Change Maximization for Active Learning »
Zixing Song · Yifei Zhang · Irwin King -
2023 Workshop: Third Workshop on Efficient Natural Language and Speech Processing (ENLSP-III): Towards the Future of Large Language Models and their Emerging Descendants »
Mehdi Rezagholizadeh · Peyman Passban · Yue Dong · Yu Cheng · Soheila Samiee · Lili Mou · Qun Liu · Boxing Chen -
2022 Spotlight: A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarization »
Puyuan Liu · Xiang Zhang · Lili Mou -
2022 Spotlight: Lightning Talks 1B-4 »
Andrei Atanov · Shiqi Yang · Wanshan Li · Yongchang Hao · Ziquan Liu · Jiaxin Shi · Anton Plaksin · Jiaxiang Chen · Ziqi Pan · yaxing wang · Yuxin Liu · Stepan Martyanov · Alessandro Rinaldo · Yuhao Zhou · Li Niu · Qingyuan Yang · Andrei Filatov · Yi Xu · Liqing Zhang · Lili Mou · Ruomin Huang · Teresa Yeo · kai wang · Daren Wang · Jessica Hwang · Yuanhong Xu · Qi Qian · Hu Ding · Michalis Titsias · Shangling Jui · Ajay Sohmshetty · Lester Mackey · Joost van de Weijer · Hao Li · Amir Zamir · Xiangyang Ji · Antoni Chan · Rong Jin -
2022 Spotlight: Teacher Forcing Recovers Reward Functions for Text Generation »
Yongchang Hao · Yuxin Liu · Lili Mou -
2022 Workshop: Second Workshop on Efficient Natural Language and Speech Processing (ENLSP-II) »
Mehdi Rezagholizadeh · Peyman Passban · Yue Dong · Lili Mou · Pascal Poupart · Ali Ghodsi · Qun Liu -
2022 Poster: A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarization »
Puyuan Liu · Xiang Zhang · Lili Mou -
2022 Poster: Towards Efficient Post-training Quantization of Pre-trained Language Models »
Haoli Bai · Lu Hou · Lifeng Shang · Xin Jiang · Irwin King · Michael R Lyu -
2022 Poster: Teacher Forcing Recovers Reward Functions for Text Generation »
Yongchang Hao · Yuxin Liu · Lili Mou -
2020 Poster: Revisiting Parameter Sharing for Automatic Neural Channel Number Search »
Jiaxing Wang · Haoli Bai · Jiaxiang Wu · Xupeng Shi · Junzhou Huang · Irwin King · Michael R Lyu · Jian Cheng -
2020 Poster: DynaBERT: Dynamic BERT with Adaptive Width and Depth »
Lu Hou · Zhiqi Huang · Lifeng Shang · Xin Jiang · Xiao Chen · Qun Liu -
2020 Spotlight: DynaBERT: Dynamic BERT with Adaptive Width and Depth »
Lu Hou · Zhiqi Huang · Lifeng Shang · Xin Jiang · Xiao Chen · Qun Liu -
2018 Poster: Almost Optimal Algorithms for Linear Stochastic Bandits with Heavy-Tailed Payoffs »
Han Shao · Xiaotian Yu · Irwin King · Michael R Lyu -
2018 Spotlight: Almost Optimal Algorithms for Linear Stochastic Bandits with Heavy-Tailed Payoffs »
Han Shao · Xiaotian Yu · Irwin King · Michael R Lyu -
2014 Poster: Combinatorial Pure Exploration of Multi-Armed Bandits »
Shouyuan Chen · Tian Lin · Irwin King · Michael R Lyu · Wei Chen -
2014 Oral: Combinatorial Pure Exploration of Multi-Armed Bandits »
Shouyuan Chen · Tian Lin · Irwin King · Michael R Lyu · Wei Chen -
2013 Poster: Exact and Stable Recovery of Pairwise Interaction Tensors »
Shouyuan Chen · Michael R Lyu · Irwin King · Zenglin Xu -
2013 Spotlight: Exact and Stable Recovery of Pairwise Interaction Tensors »
Shouyuan Chen · Michael R Lyu · Irwin King · Zenglin Xu -
2010 Workshop: Machine Learning for Social Computing »
Zenglin Xu · Irwin King · Shenghuo Zhu · Yuan Qi · Rong Yan · John Yen -
2009 Poster: Adaptive Regularization for Transductive Support Vector Machine »
Zenglin Xu · Rong Jin · Jianke Zhu · Irwin King · Michael R Lyu · Zhirong Yang -
2009 Spotlight: Adaptive Regularization for Transductive Support Vector Machine »
Zenglin Xu · Rong Jin · Jianke Zhu · Irwin King · Michael R Lyu · Zhirong Yang -
2009 Poster: Heavy-Tailed Symmetric Stochastic Neighbor Embedding »
Zhirong Yang · Irwin King · Zenglin Xu · Erkki Oja -
2009 Spotlight: Heavy-Tailed Symmetric Stochastic Neighbor Embedding »
Zhirong Yang · Irwin King · Zenglin Xu · Erkki Oja -
2008 Poster: Learning with Consistency between Inductive Functions and Kernels »
Haixuan Yang · Irwin King · Michael R Lyu -
2008 Spotlight: Learning with Consistency between Inductive Functions and Kernels »
Haixuan Yang · Irwin King · Michael R Lyu -
2008 Poster: An Extended Level Method for Efficient Multiple Kernel Learning »
Zenglin Xu · Rong Jin · Irwin King · Michael R Lyu -
2007 Poster: Efficient Convex Relaxation for Transductive Support Vector Machine »
Zenglin Xu · Rong Jin · Jianke Zhu · Irwin King · Michael R Lyu