SimCE: Simplifying Cross-Entropy Loss for Collaborative Filtering

2024-06-23 · Xiaodong Yang, Huiyuan Chen, Yuchen Yan, Yuxin Tang, Yuying Zhao, Eric Xu, Yiwei Cai, Hanghang Tong

The learning objective is integral to collaborative filtering systems, where the Bayesian Personalized Ranking (BPR) loss is widely used for learning informative backbones. However, BPR often experiences slow convergence and suboptimal local optima, partially because it only considers one negative item for each positive item, neglecting the potential impacts of other unobserved items. To address this issue, the recently proposed Sampled Softmax Cross-Entropy (SSM) compares one positive sample with multiple negative samples, leading to better performance. Our comprehensive experiments confirm that recommender systems consistently benefit from multiple negative samples during training. Furthermore, we introduce a \underline{Sim}plified Sampled Softmax \underline{C}ross-\underline{E}ntropy Loss (SimCE), which simplifies the SSM using its upper bound. Our validation on 12 benchmark datasets, using both MF and LightGCN backbones, shows that SimCE significantly outperforms both BPR and SSM.

📄 PDF Abstract BibTeX arXiv:2406.16170

Code (1)

kevinc2015/simce 공식 구현 pytorch

Tasks

Collaborative Filtering Recommendation Systems

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

LightGCN LightGCN is a type of graph convolutional neural network (GCN), including only the most essential component in GCN (neighborhood…

SimCert: Probabilistic Certification for Behavioral Similarity in Deep Neural Network Compression

2026-03-16 · Jingyang Li, Fu Song, Guoqiang Li arxiv

Deploying Deep Neural Networks (DNNs) on resource-constrained embedded systems requires aggressive model compression techniques like quantization and pruning. However, ensuring that the compressed model preserves the beh…

Neural Network Compression Model Compression

Role of Orthogonality Constraints in Improving Properties of Deep Networks for Image Classification

2020-09-22 · Hongjun Choi, Anirudh Som, Pavan Turaga

Standard deep learning models that employ the categorical cross-entropy loss are known to perform well at image classification tasks. However, many standard models thus obtained often exhibit issues like feature redundan…

General Classification image-classification Image Classification

Optimized learned entropy coding parameters for practical neural-based image and video compression

2023-01-20 · Amir Said, Reza Pourreza, Hoang Le

Neural-based image and video codecs are significantly more power-efficient when weights and activations are quantized to low-precision integers. While there are general-purpose techniques for reducing quantization effect…

Quantization Video Compression

Attention-aware Semantic Communications for Collaborative Inference

2024-02-23 · Jiwoong Im, Nayoung Kwon, Taewoo Park, Jiheon Woo 외

We propose a communication-efficient collaborative inference framework in the domain of edge inference, focusing on the efficient use of vision transformer (ViT) models. The partitioning strategy of conventional collabor…

Collaborative Inference

Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization

2026-04-13 · Jiashu Yao, Heyan Huang, Daiqing Wu, Zeming Liu 외 arxiv

To encourage diverse exploration in reinforcement learning (RL) for large language models (LLMs) without compromising accuracy, we propose Policy Split, a novel paradigm that bifurcates the policy into normal and high-en…

Reinforcement Learning