Invited Talk
in
Workshop: Machine Learning for Audio Signal Processing (ML4Audio)

Polyphonic piano transcription using deep neural networks

Douglas Eck

2017 Invited Talk
in
Workshop: Machine Learning for Audio Signal Processing (ML4Audio)

Project Page

Abstract

I'll discuss the problem of transcribing polyphonic piano music with an emphasis on generalizing to unseen instruments. We optimize for two objectives. We first predict pitch onset events and then conditionally predict pitch at the frame level. I'll discuss the model architecture, which combines CNNs and LSTMs. I'll also discuss challenges faced in robust piano transcription, such as obtaining enough data to train a good model I'll also provide some demos and links to working code. This collaboration was led by Curtis Hawthorne, Erich Elsen and Jialin Song (https://arxiv.org/abs/1710.11153).

Douglas Eck works at the Google Brain team on the Magenta project, an effort to generate music, video, images and text using machine intelligence. He also worked on music search and recommendation for Google Play Music. I was an Associate Professor in Computer Science at University of Montreal in the BRAMS research center. He also worked on music performance modeling.

Speaker

Douglas Eck

I am a Senior Research Director at Google, and lead research efforts at Google DeepMind in Generative Media, including image, video, 3D, music and audio generation. My own research lies at the intersection of machine learning and human-computer interaction (HCI). In 2015, I created [Magenta](https://magenta.withgoogle.com/), an ongoing research project exploring the role of AI in art and music creation. Before joining Google in 2010, I carried out research in music perception, aspects of music performance, machine learning for large audio datasets and music recommendation. I completed my PhD in Computer Science and Cognitive Science at Indiana University in 2000 and went on to a postdoctoral fellowship with Juergen Schmidhuber at IDSIA in Lugano Switzerland. From 2003-2010, I was faculty in Computer Science in the University of Montreal machine learning group (now [MILA machine learning lab](https://mila.quebec/en), where he became Associate Professor. For more information see [http://g.co/research/douglaseck](http://g.co/research/douglaseck).

Chat is not available.