Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
Nonlinear independent component analysis (ICA) provides an appealing framework for unsupervised feature learning, but the models proposed so far are not identifiable. Here, we first propose a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data. Our learning principle, time-contrastive learning (TCL), finds a representation which allows optimal discrimination of time segments (windows). Surprisingly, we show how TCL can be related to a nonlinear ICA model, when ICA is redefined to include temporal nonstationarities. In particular, we show that TCL combined with linear ICA estimates the nonlinear ICA model up to point-wise transformations of the sources, and this solution is unique --- thus providing the first identifiability result for nonlinear ICA which is rigorous, constructive, as well as very general.
Code (2)
Tasks
Contrastive LearningTime SeriesTime Series AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unsupervised Feature Learning by Autoencoder and Prototypical Contrastive Learning for Hyperspectral Classification
Unsupervised learning methods for feature extraction are becoming more and more popular. We combine the popular contrastive learning method (prototypical contrastive learning) and the classic representation learning meth…
Contrastive LearningGeneral ClassificationRepresentation LearningUnified Framework for Feature Extraction based on Contrastive Learning
Feature extraction is an efficient approach for alleviating the issue of dimensionality in high-dimensional data. As a popular self-supervised learning method, contrastive learning has recently garnered considerable atte…
Contrastive LearningGraph EmbeddingSelf-Supervised LearningCone: Unsupervised Contrastive Opinion Extraction
Contrastive opinion extraction aims to extract a structured summary or key points organised as positive and negative viewpoints towards a common aspect or topic. Most recent works for unsupervised key point extraction is…
ClusteringContrastive LearningSentenceHiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction
Unsupervised relation extraction aims to extract the relationship between entities from natural language sentences without prior information on relational scope or distribution. Existing works either utilize self-supervi…
ClusteringContrastive LearningRelationRelation Extraction+1HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction
Unsupervised relation extraction aims to extract the relationship between entities from natural language sentences without prior information on relational scope or distribution. Existing works either utilize self-supervi…
ClusteringContrastive LearningRelationRelation Extraction+1