Second Workshop on Transfer and Multi-Task Learning: Theory meets Practice

Transfer, domain adaptation and multi-task learning methods have been developed to better exploit the available data at training time, originally moved by the need to deal with a reduced amount of information. Nowadays, gathering data is much easier than in the past thanks to the low price of different acquisition devices (e.g. cameras) and to the World Wide Web that connects million of devices users. Existing methods must embrace new challenges to manage large scale data that do not lack anymore in size but may lack in quality or may continuously change over time. All this comes with several open questions, for instance:

- what are the limits of existing multi-task learning methods when the number of tasks grows while each task is described by only a small bunch of samples (“big T, small n”)?
- theory vs. practice: can multi-task learning for very big data (n>10^7) be performed with extremely randomized trees?
- what is the right way to leverage over noisy data gathered from the Internet as reference for a new task?

- can we get an advantage by overcoming the dataset bias and aligning multiple existing but partially related data collections before using them as source knowledge for a new target problem?

- how can an automatic system process a continuous stream of information in time and progressively adapt for life-long learning?
- since deep learning has demonstrated high performance on large scale data, is it possible to combine it with transfer and multiple kernel learning in a principled manner?
- can deep learning help to learn the right representation (e.g., task similarity matrix) in kernel-based transfer and multi-task learning?
- How can notions from reinforcement learning such as source task selection be connected to notions from convex multi-task learning such as the task similarity matrix?
- How can similarities across languages help us adapt to different domains in natural language processing tasks?

After the first workshop edition where we investigated new directions for learning across domains, we want now to call the attention of the machine learning community on the emerging problem of big data and its particular challenges regarding multi-task and transfer learning and its practical effects in many application areas like computer vision, robotics, medicine, bioinformatics etc. where transfer, domain adaptation and multi-task learning have been previously used with success. We will encourage applied researchers to contribute to the workshop in order to create a synergy with theoreticians and lead to a global advancement of the field.

A selection of the papers accepted to the workshop and voted by the reviewers will be re-evaluated also as invited contributions to the planned JMLR special topic on Domain Adaptation, Multi-task and Transfer Learning. The proposal for this special topic is currently under evaluation.

References:
[1] I. Kuzborskij, F. Orabona. Stability and Hypothesis Transfer Learning. ICML 2013
[2] T. Tommasi, F. Orabona, B. Caputo. Learning Categories from few Examples with Multi Model Knowledge Transfer. PAMI 36(5), 2014.
[3] U. Rückert, M. Kloft. Transfer Learning with Adaptive Regularizers. ECML 2011.
[4] A. Pentina, C. H. Lampert. A PAC-Bayesian bound for Lifelong Learning. ICML 2014.
[5] X. Glorot , A. Bordes , Y. Bengio. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach. ICML 2011.
[6] A. Kumar, A. Saha, H. Daumé III. A Co-regularization Based Semi-supervised Domain Adaptation. NIPS 2010.
[7] Cortes, Corinna, and Mehryar Mohri. Domain adaptation and sample bias correction theory and algorithm for regression. In Theoretical Computer Science 519 (2014): 103-126.
[8] C. Widmer, M. Kloft, G. Rätsch. Multi-task Learning for Computational Biology: Overview and Outlook. In Schölkopf et al: Festschrift in Honor of Vladimir Vapnik, Spinger 2013.