Workshop
Second Workshop on Transfer and Multi-Task Learning: Theory meets Practice
Urun Dogan · Tatiana Tommasi · Yoshua Bengio · Francesco Orabona · Marius Kloft · Andres Munoz · Gunnar Rätsch · Hal Daumé III · Mehryar Mohri · Xuezhi Wang · Daniel Hernández-lobato · Song Liu · Thomas Unterthiner · Pascal Germain · Vinay P Namboodiri · Michael Goetz · Christopher Berlind · Sigurd Spieckermann · Marta Soare · Yujia Li · Vitaly Kuznetsov · Wenzhao Lian · Daniele Calandriello · Emilie Morvant

Sat Dec 13th 08:30 AM -- 06:30 PM @ Level 5; room 510 d
Event URL: https://sites.google.com/site/multitaskwsnips2014/ »

Transfer, domain adaptation and multi-task learning methods have been developed to better exploit the available data at training time, originally moved by the need to deal with a reduced amount of information. Nowadays, gathering data is much easier than in the past thanks to the low price of different acquisition devices (e.g. cameras) and to the World Wide Web that connects million of devices users. Existing methods must embrace new challenges to manage large scale data that do not lack anymore in size but may lack in quality or may continuously change over time. All this comes with several open questions, for instance:

- what are the limits of existing multi-task learning methods when the number of tasks grows while each task is described by only a small bunch of samples (“big T, small n”)?
- theory vs. practice: can multi-task learning for very big data (n>10^7) be performed with extremely randomized trees?
- what is the right way to leverage over noisy data gathered from the Internet as reference for a new task?

- can we get an advantage by overcoming the dataset bias and aligning multiple existing but partially related data collections before using them as source knowledge for a new target problem?

- how can an automatic system process a continuous stream of information in time and progressively adapt for life-long learning?
- since deep learning has demonstrated high performance on large scale data, is it possible to combine it with transfer and multiple kernel learning in a principled manner?
- can deep learning help to learn the right representation (e.g., task similarity matrix) in kernel-based transfer and multi-task learning?
- How can notions from reinforcement learning such as source task selection be connected to notions from convex multi-task learning such as the task similarity matrix?
- How can similarities across languages help us adapt to different domains in natural language processing tasks?

After the first workshop edition where we investigated new directions for learning across domains, we want now to call the attention of the machine learning community on the emerging problem of big data and its particular challenges regarding multi-task and transfer learning and its practical effects in many application areas like computer vision, robotics, medicine, bioinformatics etc. where transfer, domain adaptation and multi-task learning have been previously used with success. We will encourage applied researchers to contribute to the workshop in order to create a synergy with theoreticians and lead to a global advancement of the field.

A selection of the papers accepted to the workshop and voted by the reviewers will be re-evaluated also as invited contributions to the planned JMLR special topic on Domain Adaptation, Multi-task and Transfer Learning. The proposal for this special topic is currently under evaluation.

References:
[1] I. Kuzborskij, F. Orabona. Stability and Hypothesis Transfer Learning. ICML 2013
[2] T. Tommasi, F. Orabona, B. Caputo. Learning Categories from few Examples with Multi Model Knowledge Transfer. PAMI 36(5), 2014.
[3] U. Rückert, M. Kloft. Transfer Learning with Adaptive Regularizers. ECML 2011.
[4] A. Pentina, C. H. Lampert. A PAC-Bayesian bound for Lifelong Learning. ICML 2014.
[5] X. Glorot , A. Bordes , Y. Bengio. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach. ICML 2011.
[6] A. Kumar, A. Saha, H. Daumé III. A Co-regularization Based Semi-supervised Domain Adaptation. NIPS 2010.
[7] Cortes, Corinna, and Mehryar Mohri. Domain adaptation and sample bias correction theory and algorithm for regression. In Theoretical Computer Science 519 (2014): 103-126.
[8] C. Widmer, M. Kloft, G. Rätsch. Multi-task Learning for Computational Biology: Overview and Outlook. In Schölkopf et al: Festschrift in Honor of Vladimir Vapnik, Spinger 2013.

Author Information

Urun Dogan (Microsoft)
Tatiana Tommasi (KU Leuven)
Yoshua Bengio (University of Montreal)

Yoshua Bengio (PhD'1991 in Computer Science, McGill University). After two post-doctoral years, one at MIT with Michael Jordan and one at AT&T Bell Laboratories with Yann LeCun, he became professor at the department of computer science and operations research at Université de Montréal. Author of two books (a third is in preparation) and more than 200 publications, he is among the most cited Canadian computer scientists and is or has been associate editor of the top journals in machine learning and neural networks. Since '2000 he holds a Canada Research Chair in Statistical Learning Algorithms, since '2006 an NSERC Chair, since '2005 his is a Senior Fellow of the Canadian Institute for Advanced Research and since 2014 he co-directs its program focused on deep learning. He is on the board of the NIPS foundation and has been program chair and general chair for NIPS. He has co-organized the Learning Workshop for 14 years and co-created the International Conference on Learning Representations. His interests are centered around a quest for AI through machine learning, and include fundamental questions on deep learning, representation learning, the geometry of generalization in high-dimensional spaces, manifold learning and biologically inspired learning algorithms.

Francesco Orabona (Stony Brook University)
Marius Kloft (TU Kaiserslautern)
Andres Munoz (Google)
Gunnar Rätsch (ETH Zürich)
Hal Daumé III (Microsoft Research & University of Maryland)

Hal Daumé III wields a professor appointment in Computer Science and Language Science at the University of Maryland, and spends time as a principal researcher in the machine learning group and fairness group at Microsoft Research in New York City. He and his wonderful advisees study questions related to how to get machines to become more adept at human language, by developing models and algorithms that allow them to learn from data. The two major questions that really drive their research these days are: (1) how can we get computers to learn language through natural interaction with people/users? and (2) how can we do this in a way that promotes fairness, transparency and explainability in the learned models?

Mehryar Mohri (Courant Inst. of Math. Sciences & Google Research)
Xuezhi Wang (Google AI)
Daniel Hernández-lobato (Universidad Autonoma de Madrid)
Song Liu (Tokyo Institute of Technology)
Tom Unterthiner (LIT AI Lab / University Linz)
Pascal Germain (INRIA Paris)
Vinay P Namboodiri (IIT Kanpur)
Michael Goetz (DKFZ Heidelberg)
Christopher Berlind (Georgia Institute of Technology)
Sigurd Spieckermann (Siemens)
Marta Soare (INRIA Lille - Nord Europe)
Yujia Li (University of Toronto)
Vitaly Kuznetsov (HRT)

Vitaly Kuznetsov is a research scientist at Google. Prior to joining Google Research, Vitaly received his Ph.D. in mathematics from the Courant Institute of Mathematical Sciences at New York University. Vitaly has contributed to a number of different areas in machine learning, in particular the development of the theory and algorithms for forecasting non-stationary time series. At Google, his work is focused on the design and implementation of large-scale machine learning tools and algorithms for time series modeling, forecasting and anomaly detection. His current research interests include all aspects of applied and theoretical time series analysis, in particular, in non-stationary environments.

Wenzhao Lian (Duke University)
Daniele Calandriello (LCSL IIT/MIT)
Emilie Morvant (LaHC, University of Saint-Etienne)

More from the Same Authors