Towards Mitigating Dimensional Collapse of Representations in Collaborative Filtering
Contrastive Learning (CL) has shown promising performance in collaborative filtering. The key idea is to generate augmentation-invariant embeddings by maximizing the Mutual Information between different augmented views of the same instance. However, we empirically observe that existing CL models suffer from the \textsl{dimensional collapse} issue, where user/item embeddings only span a low-dimension subspace of the entire feature space. This suppresses other dimensional information and weakens the distinguishability of embeddings. Here we propose a non-contrastive learning objective, named nCL, which explicitly mitigates dimensional collapse of representations in collaborative filtering. Our nCL aims to achieve geometric properties of \textsl{Alignment} and \textsl{Compactness} on the embedding space. In particular, the alignment tries to push together representations of positive-related user-item pairs, while compactness tends to find the optimal coding length of user/item embeddings, subject to a given distortion. More importantly, our nCL does not require data augmentation nor negative sampling during training, making it scalable to large datasets. Experimental results demonstrate the superiority of our nCL.
Code (0)
등록된 구현이 없습니다.
Tasks
Collaborative FilteringContrastive LearningData AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning
Federated learning aims to train models collaboratively across different clients without the sharing of data for privacy considerations. However, one major challenge for this learning paradigm is the {\em data heterogene…
Federated LearningStabilizing Self-Consuming Diffusion Models with Latent Space Filtering
As synthetic data proliferates across the Internet, it is often reused to train successive generations of generative models. This creates a ``self-consuming loop" that can lead to training instability or \textit{model co…
Mitigating the Popularity Bias of Graph Collaborative Filtering: A Dimensional Collapse Perspective
Graph-based Collaborative Filtering (GCF) is widely used in personalized recommendation systems. However, GCF suffers from a fundamental problem where features tend to occupy the embedding space inefficiently (by spannin…
Collaborative FilteringRecommendation SystemsMitigating the Popularity Bias of Graph Collaborative Filtering: A Dimensional Collapse Perspective
Graph-based Collaborative Filtering (GCF) is widely used in personalized recommendation systems. However, GCF suffers from a fundamental problem where features tend to occupy the embedding space inefficiently (by spannin…
Collaborative FilteringDiversityRecommendation SystemsNeural Fair Collaborative Filtering
A growing proportion of human interactions are digitized on social media platforms and subjected to algorithmic decision-making, and it has become increasingly important to ensure fair treatment from these algorithms. In…
Collaborative FilteringDecision MakingRecommendation Systems