Implementing Evaluation Metrics Based on Theories of Democracy in News Comment Recommendation (Hackathon Report)
Diversity in news recommendation is important for democratic debate. Current recommendation strategies, as well as evaluation metrics for recommender systems, do not explicitly focus on this aspect of news recommendation. In the 2021 Embeddia Hackathon, we implemented one novel, normative theory-based evaluation metric, “activation”, and use it to compare two recommendation strategies of New York Times comments, one based on user likes and another on editor picks. We found that both comment recommendation strategies lead to recommendations consistently less activating than the available comments in the pool of data, but the editor’s picks more so. This might indicate that New York Times editors’ support a deliberative democratic model, in which less activation is deemed ideal for democratic debate.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityNews RecommendationRecommendation SystemsSimilar Papers 제목 키워드 기반
Fake News: A Survey of Research, Detection Methods, and Opportunities
The explosive growth in fake news and its erosion to democracy, justice, and public trust has increased the demand for fake news analysis, detection and intervention. This survey comprehensively and systematically review…
SurveyFake News Early Detection: An Interdisciplinary Study
Massive dissemination of fake news and its potential to erode democracy has increased the demand for accurate fake news detection. Recent advancements in this area have proposed novel techniques that aim to detect fake n…
Fake News DetectionFeature EngineeringNo NLP Task Should be an Island: Multi-disciplinarity for Diversity in News Recommender Systems
Natural Language Processing (NLP) is defined by specific, separate tasks, with each their own literature, benchmark datasets, and definitions. In this position paper, we argue that for a complex problem such as the threa…
DiversityEthicsNews RecommendationPosition+1Using NLP to measure democracy
This paper uses natural language processing to create the first machine-coded democracy index, which I call Automated Democracy Scores (ADS). The ADS are based on 42 million news articles from 6,043 different sources and…
ArticlesregressionBiasScanner: Automatic Detection and Classification of News Bias to Strengthen Democracy
The increasing consumption of news online in the 21st century coincided with increased publication of disinformation, biased reporting, hate speech and other unwanted Web content. We describe BiasScanner, an application …
ArticlesBias DetectionLanguage ModelingLanguage Modelling+2