Output Representation Learning

Modern data analysis is increasingly facing prediction problems that have complex and high dimensional output spaces. For example, document tagging problems regularly consider large (and sometimes hierarchical) sets of output tags; image tagging problems regularly consider tens of thousands of possible output labels; natural language processing tasks have always considered complex output spaces. In such complex and high dimensional output spaces the candidate labels are often too specialized---leading to sparse data for individual labels---or too generalized---leading to complex prediction maps being required. In such cases, it is essential to identify an alternative output representation that can provide latent output categories that abstract overly specialized labels, specialize overly abstract labels, or reveal the latent dependence between labels.

There is a growing body of work on learning output representations, distinct from current work on learning input representations. For example, in machine learning, work on multi-label learning, and particularly output dimensionality reduction in high dimensional label spaces, has begun to address the specialized label problem, while work on output kernel learning has begun to address the abstracted label problem. In computer vision, work on image categorization and tagging has begun to investigate simple forms of latent output representation learning to cope with abstract semantic labels and large label sets. In speech recognition, dimensionality reduction has been used to identify abstracted outputs, while hidden CRFs have been used to identify specialized latent outputs. In information retrieval and natural language processing, discovering latent output specializations in complex domains has been an ongoing research topic for the past half decade.

The aim of this workshop is to bring these relevant research communities together to identify fundamental strategies, highlight differences, and identify the prospects for developing a set of systematic theory and methods for output representation learning. The target communities include researchers working on image tagging, document categorization, natural language processing, large vocabulary speech recognition, deep learning, latent variable modeling, and large scale multi-label learning.