Timezone: »

Teacher Forcing Recovers Reward Functions for Text Generation
Yongchang Hao · Yuxin Liu · Lili Mou

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #339

Reinforcement learning (RL) has been widely used in text generation to alleviate the exposure bias issue or to utilize non-parallel datasets. The reward function plays an important role in making RL training successful. However, previous reward functions are typically task-specific and sparse, restricting the use of RL. In our work, we propose a task-agnostic approach that derives a step-wise reward function directly from a model trained with teacher forcing. We additionally propose a simple modification to stabilize the RL training on non-parallel datasets with our induced reward function. Empirical results show that our method outperforms self-training and reward regression methods on several text generation tasks, confirming the effectiveness of our reward function.

Author Information

Yongchang Hao (University of Alberta)
Yuxin Liu (University of Alberta)
Lili Mou (University of Alberta)

More from the Same Authors