Timezone: »
This paper tackles the complex problem of visually matching people in similar pose but with different clothes, background, and other appearance changes. We achieve this with a novel method for learning a nonlinear embedding based on several extensions to the Neighborhood Component Analysis (NCA) framework. Our method is convolutional, enabling it to scale to realistically-sized images. By cheaply labeling the head and hands in large video databases through Amazon Mechanical Turk (a crowd-sourcing service), we can use the task of localizing the head and hands as a proxy for determining body pose. We apply our method to challenging real-world data and show that it can generalize beyond hand localization to infer a more general notion of body pose. We evaluate our method quantitatively against other embedding methods. We also demonstrate that real-world performance can be improved through the use of synthetic data.
Author Information
Graham Taylor (University of Guelph)
Rob Fergus (DeepMind / NYU)
Rob Fergus is an Associate Professor of Computer Science at the Courant Institute of Mathematical Sciences, New York University. He received a Masters in Electrical Engineering with Prof. Pietro Perona at Caltech, before completing a PhD with Prof. Andrew Zisserman at the University of Oxford in 2005. Before coming to NYU, he spent two years as a post-doc in the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT, working with Prof. William Freeman. He has received several awards including a CVPR best paper prize, a Sloan Fellowship & NSF Career award and the IEEE Longuet-Higgins prize.
George Williams (New York University)
Ian Spiro (New York University)
Christoph Bregler (New York University)
More from the Same Authors
-
2020 Poster: Instance Selection for GANs »
Terrance DeVries · Michal Drozdzal · Graham Taylor -
2020 Session: Orals & Spotlights Track 08: Deep Learning »
Graham Taylor · Mario Lucic -
2019 Poster: Understanding Attention and Generalization in Graph Neural Networks »
Boris Knyazev · Graham Taylor · Mohamed Amer -
2016 Poster: Learning Multiagent Communication with Backpropagation »
Sainbayar Sukhbaatar · arthur szlam · Rob Fergus -
2014 Poster: Depth Map Prediction from a Single Image using a Multi-Scale Deep Network »
David Eigen · Christian Puhrsch · Rob Fergus -
2014 Poster: Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation »
Emily Denton · Wojciech Zaremba · Joan Bruna · Yann LeCun · Rob Fergus -
2014 Poster: Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation »
Jonathan J Tompson · Arjun Jain · Yann LeCun · Christoph Bregler -
2014 Spotlight: Depth Map Prediction from a Single Image using a Multi-Scale Deep Network »
David Eigen · Christian Puhrsch · Rob Fergus -
2014 Poster: Learning to Discover Efficient Mathematical Identities »
Wojciech Zaremba · Karol Kurach · Rob Fergus -
2014 Spotlight: Learning to Discover Efficient Mathematical Identities »
Wojciech Zaremba · Karol Kurach · Rob Fergus -
2013 Tutorial: Deep Learning for Computer Vision »
Rob Fergus -
2011 Workshop: Machine Learning meets Computational Photography »
Michael Hirsch · Stefan Harmeling · Rob Fergus · Peyman Milanfar -
2011 Workshop: Big Learning: Algorithms, Systems, and Tools for Learning at Scale »
Joseph E Gonzalez · Sameer Singh · Graham Taylor · James Bergstra · Alice Zheng · Misha Bilenko · Yucheng Low · Yoshua Bengio · Michael Franklin · Carlos Guestrin · Andrew McCallum · Alexander Smola · Michael Jordan · Sugato Basu -
2011 Poster: Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines »
Matthew D Zeiler · Graham Taylor · Leonid Sigal · Iain Matthews · Rob Fergus -
2011 Session: Spotlight Session 1 »
Rob Fergus -
2010 Session: Oral Session 17 »
Rob Fergus -
2009 Poster: Fast Image Deconvolution using Hyper-Laplacian Priors »
Dilip Krishnan · Rob Fergus -
2009 Spotlight: Fast Image Deconvolution using Hyper-Laplacian Priors »
Dilip Krishnan · Rob Fergus -
2009 Poster: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2009 Oral: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2008 Poster: The Recurrent Temporal Restricted Boltzmann Machine »
Ilya Sutskever · Geoffrey E Hinton · Graham Taylor -
2008 Poster: Spectral Hashing »
Yair Weiss · Antonio Torralba · Rob Fergus -
2007 Spotlight: Object Recognition by Scene Alignment »
Bryan C Russell · Antonio Torralba · Ce Liu · Rob Fergus · William Freeman -
2007 Poster: Object Recognition by Scene Alignment »
Bryan C Russell · Antonio Torralba · Ce Liu · Rob Fergus · William Freeman -
2006 Poster: Learning Motion Style Synthesis from Perceptual Observations »
Lorenzo Torresani · Peggy Hackney · Christoph Bregler -
2006 Poster: Modeling Human Motion Using Binary Latent Variables »
Graham Taylor · Geoffrey E Hinton · Sam T Roweis -
2006 Spotlight: Modeling Human Motion Using Binary Latent Variables »
Graham Taylor · Geoffrey E Hinton · Sam T Roweis -
2006 Talk: Learning Motion Style Synthesis from Perceptual Observations »
Lorenzo Torresani · Peggy Hackney · Christoph Bregler