Assessing Objective Recommendation Quality through Political Forecasting
Recommendations are often rated for their subjective quality, but few researchers have studied comment quality in terms of objective utility. We explore recommendation quality assessment with respect to both subjective (i.e. users{'} ratings) and objective (i.e., did it influence? did it improve decisions?) metrics in a massive online geopolitical forecasting system, ultimately comparing linguistic characteristics of each quality metric. Using a variety of features, we predict all types of quality with better accuracy than the simple yet strong baseline of comment length. Looking at the most predictive content illustrates rater biases; for example, forecasters are subjectively biased in favor of comments mentioning business transactions or dealings as well as material things, even though such comments do not indeed prove any more useful objectively. Additionally, more complex sentence constructions, as evidenced by subordinate conjunctions, are characteristic of comments leading to objective improvements in forecasting.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentiment AnalysisSimilar Papers 제목 키워드 기반
Explainable Recommendation with Comparative Constraints on Product Aspects
To aid users in choice-making, explainable recommendation models seek to provide not only accurate recommendations but also accompanying explanations that help to make sense of those recommendations. Most of the previous…
Explainable RecommendationAssessing Political Prudence of Open-domain Chatbots
Politically sensitive topics are still a challenge for open-domain chatbots. However, dealing with politically sensitive content in a responsible, non-partisan, and safe behavior way is integral for these chatbots. Curre…
ChatbotCross-Cutting Political Awareness through Diverse News Recommendations
The suggestions generated by most existing recommender systems are known to suffer from a lack of diversity, and other issues like popularity bias. As a result, they have been observed to promote well-known "blockbuster"…
DiversityRecommendation SystemsRecommender system in X inadvertently profiles ideological positions of users
Studies on recommendations in social media have mainly analyzed the quality of recommended items (e.g., their diversity or biases) and the impact of recommendation policies (e.g., in comparison with purely chronological …
Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset
The political biases of Large Language Models (LLMs) are usually assessed by simulating their answers to English surveys. In this work, we propose an alternative framing of political biases, relying on principles of fair…