Papers User Simulation
“User Simulation” 태그가 달린 논문 64편 · 필터 해제
USimAgent: Large Language Models for Simulating Search Users
Due to the advantages in the cost-efficiency and reproducibility, user simulation has become a promising solution to the user-centric evaluation of information retrieval systems. Nonetheless, accurately simulating user s…
Information RetrievalUser SimulationStrength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation
We investigate non-collaborative dialogue agents, which are expected to engage in strategic conversations with diverse users, for securing a mutual agreement that leans favorably towards the system's objectives. This pos…
User SimulationBASES: Large-scale Web Search User Simulation with Large Language Model based Agents
Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation. Considering the scarcity and limit (e.g., privacy issues) of real user data, …
Information RetrievalLanguage ModelingLanguage ModellingLarge Language Model+2Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation
Social media has emerged as a cornerstone of social movements, wielding significant influence in driving societal change. Simulating the response of the public and forecasting the potential impact has become increasingly…
User SimulationReliable LLM-based User Simulator for Task-Oriented Dialogue Systems
In the realm of dialogue systems, user simulation techniques have emerged as a game-changer, redefining the evaluation and enhancement of task-oriented dialogue (TOD) systems. These methods are crucial for replicating re…
Data AugmentationTask-Oriented Dialogue SystemsUser SimulationLeveraging User Simulation to Develop and Evaluate Conversational Information Access Agents
We observe a change in the way users access information, that is, the rise of conversational information access (CIA) agents. However, the automatic evaluation of these agents remains an open challenge. Moreover, the tra…
Conversational Information AccessReinforcement LearningUser SimulationCIRCLE: Multi-Turn Query Clarifications with Reinforcement Learning
Users often have trouble formulating their information needs into words on the first try when searching online. This can lead to frustration, as they may have to reformulate their queries when retrieved information is no…
Language ModelingLanguage Modellingreinforcement-learningReinforcement Learning+1User Simulation with Large Language Models for Evaluating Task-Oriented Dialogue
One of the major impediments to the development of new task-oriented dialogue (TOD) systems is the need for human evaluation at multiple stages and iterations of the development process. In an effort to move toward autom…
In-Context LearningUser SimulationUser Simulation for Evaluating Information Access Systems
Information access systems, such as search engines, recommender systems, and conversational assistants, have become integral to our daily lives as they help us satisfy our information needs. However, evaluating the effec…
Recommendation SystemsUser SimulationUser Behavior Simulation with Large Language Model based Agents
Simulating high quality user behavior data has always been a fundamental problem in human-centered applications, where the major difficulty originates from the intricate mechanism of human decision process. Recently, sub…
Language ModelingLanguage ModellingLarge Language ModelRecommendation Systems+1In-Context Learning User Simulators for Task-Oriented Dialog Systems
This paper presents a novel application of large language models in user simulation for task-oriented dialog systems, specifically focusing on an in-context learning approach. By harnessing the power of these models, the…
Goal-Oriented Dialogue SystemsIn-Context LearningInteractive Evaluation of DialogTask-Oriented Dialogue Systems+1Theoretical Analysis on the Efficiency of Interleaved Comparisons
This study presents a theoretical analysis on the efficiency of interleaving, an efficient online evaluation method for rankings. Although interleaving has already been applied to production systems, the source of its hi…
User SimulationAn In-depth Investigation of User Response Simulation for Conversational Search
Conversational search has seen increased recent attention in both the IR and NLP communities. It seeks to clarify and solve users' search needs through multi-turn natural language interactions. However, most existing sys…
Conversational SearchText GenerationUser SimulationMultimodal Interactive Lung Lesion Segmentation: A Framework for Annotating PET/CT Images based on Physiological and Anatomical Cues
Recently, deep learning enabled the accurate segmentation of various diseases in medical imaging. These performances, however, typically demand large amounts of manual voxel annotations. This tedious process for volumetr…
Interactive SegmentationLesion SegmentationSegmentationUser SimulationUserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender Systems
We present an extensible user simulation toolkit to facilitate automatic evaluation of conversational recommender systems. It builds on an established agenda-based approach and extends it with several novel elements, inc…
Recommendation SystemsText GenerationUser SimulationBotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systems
We introduce BotSIM, a modular, open-source Bot SIMulation environment with dialog generation, user simulation and conversation analytics capabilities. BotSIM aims to serve as a one-stop solution for large-scale data-eff…
User SimulationLeveraging Centric Data Federated Learning Using Blockchain For Integrity Assurance
Machine learning abilities have become a vital component for various solutions across industries, applications, and sectors. Many organizations seek to leverage AI-based solutions across their business services to unlock…
Federated LearningUser SimulationInteractive Query Clarification and Refinement via User Simulation
When users initiate search sessions, their queries are often unclear or might lack of context; this resulting in inefficient document ranking. Multiple approaches have been proposed by the Information Retrieval community…
Document RankingInformation RetrievalRetrievalUser SimulationAnalyzing and Simulating User Utterance Reformulation in Conversational Recommender Systems
User simulation has been a cost-effective technique for evaluating conversational recommender systems. However, building a human-like simulator is still an open challenge. In this work, we focus on how users reformulate …
Recommendation SystemsUser SimulationEvaluating Mixed-initiative Conversational Search Systems via User Simulation
Clarifying the underlying user information need by asking clarifying questions is an important feature of modern conversational search system. However, evaluation of such systems through answering prompted clarifying que…
Conversational SearchText GenerationUser Simulation