A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedback
Interactive search can provide a better experience by incorporating interaction feedback from the users. This can significantly improve search accuracy as it helps avoid irrelevant information and captures the users' search intents. Existing state-of-the-art (SOTA) systems use reinforcement learning (RL) models to incorporate the interactions but focus on item-level feedback, ignoring the fine-grained information found in sentence-level feedback. Yet such feedback requires extensive RL action space exploration and large amounts of annotated data. This work addresses these challenges by proposing a new deep Q-learning (DQ) approach, DQrank. DQrank adapts BERT-based models, the SOTA in natural language processing, to select crucial sentences based on users' engagement and rank the items to obtain more satisfactory responses. We also propose two mechanisms to better explore optimal actions. DQrank further utilizes the experience replay mechanism in DQ to store the feedback sentences to obtain a better initial ranking performance. We validate the effectiveness of DQrank on three search datasets. The results show that DQRank performs at least 12% better than the previous SOTA RL approaches. We also conduct detailed ablation studies. The ablation results demonstrate that each model component can efficiently extract and accumulate long-term engagement effects from the users' sentence-level feedback. This structure offers new technologies with promised performance to construct a search system with sentence-level interaction.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningQ-LearningReinforcement Learning (RL)SentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
We propose an interactive-predictive neural machine translation framework for easier model personalization using reinforcement and imitation learning. During the interactive translation process, the user is asked for fee…
FormImitation LearningMachine TranslationTranslationReinforcement Learning with Token-level Feedback for Controllable Text Generation
To meet the requirements of real-world applications, it is essential to control generations of large language models (LLMs). Prior research has tried to introduce reinforcement learning (RL) into controllable text genera…
Attributereinforcement-learningReinforcement LearningReinforcement Learning (RL)+2Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment
Human-centered AI considers human experiences with AI performance. While abundant research has been helping AI achieve superhuman performance either by fully automatic or weak supervision learning, fewer endeavors are ex…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks
Reinforcement learning (RL) has become widely adopted in robot control. Despite many successes, one major persisting problem can be very low data efficiency. One solution is interactive feedback, which has been shown to …
reinforcement-learningReinforcement Learning (RL)Facial Feedback for Reinforcement Learning: A Case Study and Offline Analysis Using the TAMER Framework
Interactive reinforcement learning provides a way for agents to learn to solve tasks from evaluative feedback provided by a human user. Previous research showed that humans give copious feedback early in training but ver…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)