Simulating Users in Interactive Web Table Retrieval
Considering the multimodal signals of search items is beneficial for retrieval effectiveness. Especially in web table retrieval (WTR) experiments, accounting for multimodal properties of tables boosts effectiveness. However, it still remains an open question how the single modalities affect user experience in particular. Previous work analyzed WTR performance in ad-hoc retrieval benchmarks, which neglects interactive search behavior and limits the conclusion about the implications for real-world user environments. To this end, this work presents an in-depth evaluation of simulated interactive WTR search sessions as a more cost-efficient and reproducible alternative to real user studies. As a first of its kind, we introduce interactive query reformulation strategies based on Doc2Query, incorporating cognitive states of simulated user knowledge. Our evaluations include two perspectives on user effectiveness by considering different cost paradigms, namely query-wise and time-oriented measures of effort. Our multi-perspective evaluation scheme reveals new insights about query strategies, the impact of modalities, and different user types in simulated WTR search sessions.
Code (1)
Tasks
RetrievalTable RetrievalSimilar Papers 제목 키워드 기반
ADEPT: An Entropy-Driven Dual-Strategy Agent for Interactive Video Retrieval
This research aims to solve the challenge of video retrieval from massive datasets, caused by ambiguous user queries. Prevailing single-round retrieval paradigms face a performance bottleneck, as they lack effective feed…
Video RetrievalJoint Learning of Interactive Spoken Content Retrieval and Trainable User Simulator
User-machine interaction is crucial for information retrieval, especially for spoken content retrieval, because spoken content is difficult to browse, and speech recognition has a high degree of uncertainty. In interacti…
Information RetrievalQ-LearningRetrievalspeech-recognition+1Revisiting Relevance Feedback for CLIP-based Interactive Image Retrieval
Many image retrieval studies use metric learning to train an image encoder. However, metric learning cannot handle differences in users' preferences, and requires data to train an image encoder. To overcome these limitat…
Image RetrievalMetric LearningRetrievalVisual-Interactive Similarity Search for Complex Objects by Example of Soccer Player Analysis
The definition of similarity is a key prerequisite when analyzing complex data types in data mining, information retrieval, or machine learning. However, the meaningful definition is often hampered by the complexity of d…
Active LearningInformation RetrievalRetrievalCheerBots: Chatbots toward Empathy and Emotionusing Reinforcement Learning
Apart from the coherence and fluency of responses, an empathetic chatbot emphasizes more on people's feelings. By considering altruistic behaviors between human interaction, empathetic chatbots enable people to get a bet…
ChatbotDeep Reinforcement Learningreinforcement-learningReinforcement Learning+2