Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering
In this paper, we explore the problem of deep multi-view subspace clustering framework from an information-theoretic point of view. We extend the traditional information bottleneck principle to learn common information among different views in a self-supervised manner, and accordingly establish a new framework called Self-supervised Information Bottleneck based Multi-view Subspace Clustering (SIB-MSC). Inheriting the advantages from information bottleneck, SIB-MSC can learn a latent space for each view to capture common information among the latent representations of different views by removing superfluous information from the view itself while retaining sufficient information for the latent representations of other views. Actually, the latent representation of each view provides a kind of self-supervised signal for training the latent representations of other views. Moreover, SIB-MSC attempts to learn the other latent space for each view to capture the view-specific information by introducing mutual information based regularization terms, so as to further improve the performance of multi-view subspace clustering. To the best of our knowledge, this is the first work to explore information bottleneck for multi-view subspace clustering. Extensive experiments on real-world multi-view data demonstrate that our method achieves superior performance over the related state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringMulti-view Subspace ClusteringSimilar Papers 제목 키워드 기반
MVEB: Self-Supervised Learning with Multi-View Entropy Bottleneck
Self-supervised learning aims to learn representation that can be effectively generalized to downstream tasks. Many self-supervised approaches regard two views of an image as both the input and the self-supervised signal…
Linear evaluationSelf-Supervised LearningTo Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review
Deep neural networks excel in supervised learning tasks but are constrained by the need for extensive labeled data. Self-supervised learning emerges as a promising alternative, allowing models to learn without explicit l…
Self-Supervised LearningPre-training Graph Neural Networks on 2D and 3D Molecular Structures by using Multi-View Conditional Information Bottleneck
Recent pre-training strategies for molecular graphs have attempted to use 2D and 3D molecular views as both inputs and self-supervised signals, primarily aligning graph-level representations. However, existing studies re…
Representation LearningBottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
The principle of the Information Bottleneck (Tishby et al. 1999) is to produce a summary of information X optimized to predict some other relevant information Y. In this paper, we propose a novel approach to unsupervised…
Abstractive Text SummarizationExtractive SummarizationLanguage ModelingLanguage Modelling+4Understanding Self-supervised Learning via Information Bottleneck Principle
Self-supervised learning alleviates the massive demands for annotations in deep learning, and recent advances are mainly dominated by contrastive learning. Existed contrastive learning methods narrows the distance betwee…
Contrastive LearningSelf-Supervised Learning