FINED: Feed Instance-Wise Information Need with Essential and Disentangled Parametric Knowledge from the Past
Recommender models play a vital role in various industrial scenarios, while often faced with the catastrophic forgetting problem caused by the fast shifting data distribution. To alleviate this problem, a common approach is to reuse knowledge from the historical data. However, preserving the vast and fast-accumulating data is hard, which causes dramatic storage overhead. Memorizing old data through a parametric knowledge base is then proposed, which compresses the vast amount of raw data into model parameters. Despite the flexibility, how to improve the memorization and generalization capabilities of the parametric knowledge base and suit the flexible information need of each instance are challenging. In this paper, we propose FINED to Feed INstance-wise information need with Essential and Disentangled parametric knowledge from past data for recommendation enhancement. Concretely, we train a knowledge extractor that extracts knowledge patterns of arbitrary order from past data and a knowledge encoder that memorizes the arbitrary order patterns, which serves as the retrieval key generator and memory network respectively in the following knowledge reusing phase. The whole process is regularized by the proposed two constraints, which improve the capabilities of the parametric knowledge base without increasing the size of it. The essential principle helps to compress the input into representative vectors that capture the task-relevant information and filter out the noisy information. The disentanglement principle reduces the redundancy of stored information and pushes the knowledge base to focus on capturing the disentangled invariant patterns. These two rules together promote rational compression of information for robust and generalized knowledge representations. Extensive experiments on two datasets justify the effectiveness of the proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementMemorizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Neural network-based clustering using pairwise constraints
This paper presents a neural network-based end-to-end clustering framework. We design a novel strategy to utilize the contrastive criteria for pushing data-forming clusters directly from raw data, in addition to learning…
ClusteringDebiased Pairwise Learning from Positive-Unlabeled Implicit Feedback
Learning contrastive representations from pairwise comparisons has achieved remarkable success in various fields, such as natural language processing, computer vision, and information retrieval. Collaborative filtering a…
Collaborative FilteringInformation RetrievalRetrievalCombinatorial Bandits with Relative Feedback
We consider combinatorial online learning with subset choices when only relative feedback information from subsets is available, instead of bandit or semi-bandit feedback which is absolute. Specifically, we study two reg…
From PAC to Instance-Optimal Sample Complexity in the Plackett-Luce Model
We consider PAC-learning a good item from $k$-subsetwise feedback information sampled from a Plackett-Luce probability model, with instance-dependent sample complexity performance. In the setting where subsets of a fixed…
PAC learningDistributed Pinning Control Design for Probabilistic Boolean Networks
This paper investigates the stabilization of probabilistic Boolean networks (PBNs) via a novel pinning control strategy based on network structure. In a PBN, each node needs to choose a Boolean function from candidate Bo…