Identifying Breakdowns in Conversational Recommender Systems using User Simulation
We present a methodology to systematically test conversational recommender systems with regards to conversational breakdowns. It involves examining conversations generated between the system and simulated users for a set of pre-defined breakdown types, extracting responsible conversational paths, and characterizing them in terms of the underlying dialogue intents. User simulation offers the advantages of simplicity, cost-effectiveness, and time efficiency for obtaining conversations where potential breakdowns can be identified. The proposed methodology can be used as diagnostic tool as well as a development tool to improve conversational recommendation systems. We apply our methodology in a case study with an existing conversational recommender system and user simulator, demonstrating that with just a few iterations, we can make the system more robust to conversational breakdowns.
Code (1)
Tasks
Conversational RecommendationDiagnosticUser SimulationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Customized Conversational Recommender Systems
Conversational recommender systems (CRS) aim to capture user's current intentions and provide recommendations through real-time multi-turn conversational interactions. As a human-machine interactive system, it is essenti…
Meta-LearningRecommendation SystemsTowards Fair Conversational Recommender Systems
Conversational recommender systems have demonstrated great success. They can accurately capture a user's current detailed preference -- through a multi-round interaction cycle -- to effectively guide users to a more pers…
FairnessRecommendation SystemsComparison-based Conversational Recommender System with Relative Bandit Feedback
With the recent advances of conversational recommendations, the recommender system is able to actively and dynamically elicit user preference via conversational interactions. To achieve this, the system periodically quer…
Recommendation SystemsLearning to Ask Appropriate Questions in Conversational Recommendation
Conversational recommender systems (CRSs) have revolutionized the conventional recommendation paradigm by embracing dialogue agents to dynamically capture the fine-grained user preference. In a typical conversational rec…
Conversational RecommendationQuestion GenerationQuestion-GenerationRecommendation SystemsEvaluating Conversational Recommender Systems with Large Language Models: A User-Centric Evaluation Framework
Conversational recommender systems (CRS) involve both recommendation and dialogue tasks, which makes their evaluation a unique challenge. Although past research has analyzed various factors that may affect user satisfact…
Recommendation Systems