Timezone: »

 
Poster
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
Aapo Hyvarinen · Hiroshi Morioka

Tue Dec 06 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #188
in Tuesday Posters »

Nonlinear independent component analysis (ICA) provides an appealing framework for unsupervised feature learning, but the models proposed so far are not identifiable. Here, we first propose a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data. Our learning principle, time-contrastive learning (TCL), finds a representation which allows optimal discrimination of time segments (windows). Surprisingly, we show how TCL can be related to a nonlinear ICA model, when ICA is redefined to include temporal nonstationarities. In particular, we show that TCL combined with linear ICA estimates the nonlinear ICA model up to point-wise transformations of the sources, and this solution is unique --- thus providing the first identifiability result for nonlinear ICA which is rigorous, constructive, as well as very general.

Keywords: Deep Learning or Neural Networks · (Other) Unsupervised Learning Methods · Component Analysis (ICA,PCA,CCA, FLDA)
[ PDF] 

Author Information

Aapo Hyvarinen (University of Helsinki)
Hiroshi Morioka (University of Helsinki)

More from the Same Authors