Self-supervised contrastive learning performs non-linear system identification
Self-supervised learning (SSL) approaches have brought tremendous success across many tasks and domains. It has been argued that these successes can be attributed to a link between SSL and identifiable representation learning: Temporal structure and auxiliary variables ensure that latent representations are related to the true underlying generative factors of the data. Here, we deepen this connection and show that SSL can perform system identification in latent space. We propose DynCL, a framework to uncover linear, switching linear and non-linear dynamics under a non-linear observation model, give theoretical guarantees and validate them empirically.
Code (1)
Tasks
Contrastive LearningRepresentation LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Multi-network Contrastive Learning Based on Global and Local Representations
The popularity of self-supervised learning has made it possible to train models without relying on labeled data, which saves expensive annotation costs. However, most existing self-supervised contrastive learning methods…
Contrastive LearningLinear evaluationSelf-Supervised LearningUnderstanding self-supervised Learning Dynamics without Contrastive Pairs
While contrastive approaches of self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs) and maximizing views from different data…
Self-Supervised LearningThe Power of Contrast for Feature Learning: A Theoretical Analysis
Contrastive learning has achieved state-of-the-art performance in various self-supervised learning tasks and even outperforms its supervised counterpart. Despite its empirical success, theoretical understanding of the su…
Contrastive LearningSelf-Supervised LearningTransfer LearningSelf-Supervised and Invariant Representations for Wireless Localization
In this work, we present a wireless localization method that operates on self-supervised and unlabeled channel estimates. Our self-supervising method learns general-purpose channel features robust to fading and system im…
Linear evaluationA Simple Framework for Contrastive Learning of Visual Representations
This paper presents SimCLR: a simple framework for contrastive learning of visual representations. We simplify recently proposed contrastive self-supervised learning algorithms without requiring specialized architectures…
Contrastive LearningImage ClassificationObject RecognitionPerson Re-Identification+4