Timezone: »

 
Spotlight
Neural Voice Cloning with a Few Samples
Sercan Arik · Jitong Chen · Kainan Peng · Wei Ping · Yanqi Zhou

Tue Dec 04 12:30 PM -- 12:35 PM (PST) @ Room 220 CD
in Tue Track 1 -- Session 2 »

Voice cloning is a highly desired feature for personalized speech interfaces. We introduce a neural voice cloning system that learns to synthesize a person's voice from only a few audio samples. We study two approaches: speaker adaptation and speaker encoding. Speaker adaptation is based on fine-tuning a multi-speaker generative model. Speaker encoding is based on training a separate model to directly infer a new speaker embedding, which will be applied to a multi-speaker generative model. In terms of naturalness of the speech and similarity to the original speaker, both approaches can achieve good performance, even with a few cloning audios. While speaker adaptation can achieve slightly better naturalness and similarity, cloning time and required memory for the speaker encoding approach are significantly less, making it more favorable for low-resource deployment.

Author Information

Sercan Arik (Google)
Jitong Chen (ByteDance)
Kainan Peng (Baidu Research)
Wei Ping (Baidu Research)
Yanqi Zhou (Baidu Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors