Dealing with Uncertainties in User Feedback: Strategies Between Denying and Accepting
Latest research revealed a considerable lack of reliability within user feedback and discussed striking impacts for the assessment of adaptive web systems and content personalisation approaches, e.g. ranking errors, systematic biases to accuracy metrics as well as its natural offset (the magic barrier). In order to perform holistic assessments and to improve web systems, a variety of strategies have been proposed to deal with this so-called human uncertainty. In this contribution we discuss the most relevant strategies to handle uncertain feedback and demonstrate that these approaches are more or less ineffective to fulfil their objectives. In doing so, we consider human uncertainty within a purely probabilistic framework and utilise hypothesis testing as well as a generalisation of the magic barrier to compare the effects of recently proposed algorithms. On this basis we recommend a novel strategy of acceptance which turns away from mere filtering and discuss potential benefits for the community of the WWW.
Code (0)
등록된 구현이 없습니다.
Tasks
Two-sample testingSimilar Papers 제목 키워드 기반
Feedback linearization through the lens of data
Controlling nonlinear systems, especially when data are being used to offset uncertainties in the model, is hard. A natural approach when dealing with the challenges of nonlinear control is to reduce the system to a line…
validModeling and Counteracting Exposure Bias in Recommender Systems
What we discover and see online, and consequently our opinions and decisions, are becoming increasingly affected by automated machine learned predictions. Similarly, the predictive accuracy of learning machines heavily d…
BIG-bench Machine LearningDecision MakingRecommendation SystemsPosition: Uncertainty Quantification Needs Reassessment for Large-language Model Agents
Large-language models (LLMs) and chatbot agents are known to provide wrong outputs at times, and it was recently found that this can never be fully prevented. Hence, uncertainty quantification plays a crucial role, aimin…
ChatbotLanguage ModelingLanguage ModellingLarge Language Model+2NeuralSentinel: Safeguarding Neural Network Reliability and Trustworthiness
The usage of Artificial Intelligence (AI) systems has increased exponentially, thanks to their ability to reduce the amount of data to be analyzed, the user efforts and preserving a high rate of accuracy. However, introd…
Feedback strategies in the market with uncertainties
We explore how dynamic entry deterrence operates through feedback strategies in markets experiencing stochastic demand fluctuations. The incumbent firm, aware of its own cost structure, can deter a potential competitor b…