Analysis Operator Learning vs. Dictionary Learning: Fraternal Twins in Sparse Modeling

Exploiting structure in data is crucial for the success of many techniques in neuroscience, machine learning, signal processing, and statistics. In this context, the fact that data of interest can be modeled via sparsity has been proven extremely valuable. As a consequence, numerous algorithms either aiming at learning sparse representations of data, or exploiting sparse representations in applications have been proposed within the machine learning and signal processing communities over the last few years.
The most common way to model sparsity in data is via the so called synthesis model, also known as sparse coding. Therein, the underlying assumption is that the data can be decomposed into a linear combination of very few atoms of some dictionary. Various previous workshops and special sessions at machine learning conferences have focused on this model and its applications, as well as on algorithms for learning suitable dictionaries.

In contrast to this, considerably less attention has been drawn up to now to an interesting alternative, the so called analysis model. Here, the data is mapped to a higher dimensional space by an analysis operator and the image of this mapping is assumed to be sparse. One of the most prominent examples of analysis sparsity is the total variation model in image processing.
Both analysis operators and dictionaries can either be defined analytically, or learned using training samples drawn from the considered data. Learning sparse models is important since they outperform analytic ones in terms of optimal sparse representation, and allow sparse representations for classes of data where no analytical model is available. For the challenge of learning, unsupervised techniques are of major interest as they do not require labeled ground-truth data and are independent of a specific task. There are theoretical results for the synthesis model that mathematically justify constraints on the structure of dictionaries and thus help to design learning algorithms. Nevertheless, many theoretical questions associated with learning sparse models remain, in particular for the analysis case, which is far from being fully understood.
Clearly, synthesis modeling has big impact on machine learning problems like detection, classification or recognition tasks and has mainly influenced the areas of e.g. Deep Learning, or Multimodal Learning. Although the analysis model have proven advantageous over the synthesis model in regularizing inverse problems, its applicability to the aforementioned data analysis tasks has much less been investigated.
The proposed workshop aims at highlighting the differences, commonalities, advantages and disadvantages of the analysis and synthesis data models. The workshop will provide a venue for discussing pros and cons of the two approaches in terms of scalability, ease of learning, and most importantly, applicability to problems in machine learning such as classification, recognition, data completion, source separation, etc. The targeted group of participants ranges from researchers in machine learning and signal processing to mathematicians. All participants of the workshop will gain a deeper understanding of the duality of the two approaches for modeling data and a clear view of which model suits best for certain applications. Moreover, further research directions will be identified that adress the issue of usability of the analysis operator approach for problems arising in Machine Learning as well as important theoretical questions related to the connection of the two fraternal twins in sparse modeling.