Causality-Enhanced Behavior Sequence Modeling in LLMs for Personalized Recommendation
Recent advancements in recommender systems have focused on leveraging Large Language Models (LLMs) to improve user preference modeling, yielding promising outcomes. However, current LLM-based approaches struggle to fully leverage user behavior sequences, resulting in suboptimal preference modeling for personalized recommendations. In this study, we propose a novel Counterfactual Fine-Tuning (CFT) method to address this issue by explicitly emphasizing the role of behavior sequences when generating recommendations. Specifically, we employ counterfactual reasoning to identify the causal effects of behavior sequences on model output and introduce a task that directly fits the ground-truth labels based on these effects, achieving the goal of explicit emphasis. Additionally, we develop a token-level weighting mechanism to adjust the emphasis strength for different item tokens, reflecting the diminishing influence of behavior sequences from earlier to later tokens during predicting an item. Extensive experiments on real-world datasets demonstrate that CFT effectively improves behavior sequence modeling. Our codes are available at https://github.com/itsmeyjt/CFT.
Code (1)
Tasks
counterfactualCounterfactual ReasoningRecommendation SystemsSimilar Papers 제목 키워드 기반
Probing Causality Manipulation of Large Language Models
Large language models (LLMs) have shown various ability on natural language processing, including problems about causality. It is not intuitive for LLMs to command causality, since pretrained models usually work on stati…
In-Context LearningRAGRetrievalRetrieval-augmented Generation+1LIBER: Lifelong User Behavior Modeling Based on Large Language Models
CTR prediction plays a vital role in recommender systems. Recently, large language models (LLMs) have been applied in recommender systems due to their emergence abilities. While leveraging semantic information from LLMs …
Click-Through Rate PredictionMusic RecommendationRecommendation SystemsLearning interaction rules from multi-animal trajectories via augmented behavioral models
Extracting the interaction rules of biological agents from movement sequences pose challenges in various domains. Granger causality is a practical framework for analyzing the interactions from observed time-series data; …
Time SeriesTime Series AnalysisBreaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
With the rise of large language models (LLMs), recent works have leveraged LLMs to improve the performance of click-through rate (CTR) prediction. However, we argue that a critical obstacle remains in deploying LLMs for …
Click-Through Rate PredictionVAC2: Visual Analysis of Combined Causality in Event Sequences
Identifying causality behind complex systems plays a significant role in different domains, such as decision making, policy implementations, and management recommendations. However, existing causality studies on temporal…
Causal DiscoveryDecision MakingManagementPoint Processes