Entity-Based Evaluation of Political Bias in Automatic Summarization
Growing literature has shown that NLP systems may encode social biases; however, the political bias of summarization models remains relatively unknown. In this work, we use an entity replacement method to investigate the portrayal of politicians in automatically generated summaries of news articles. We develop an entity-based computational framework to assess the sensitivities of several extractive and abstractive summarizers to the politicians Donald Trump and Joe Biden. We find consistent differences in these summaries upon entity replacement, such as reduced emphasis of Trump's presence in the context of the same article and a more individualistic representation of Trump with respect to the collective US government (i.e., administration). These summary dissimilarities are most prominent when the entity is heavily featured in the source article. Our characterization provides a foundation for future studies of bias in summarization and for normative discussions on the ideal qualities of automatic summaries.
Code (1)
Tasks
Abstractive Text SummarizationArticlesSimilar Papers 제목 키워드 기반
Polarity Calibration for Opinion Summarization
Opinion summarization is automatically generating summaries from a variety of subjective information, such as product reviews or political opinions. The challenge of opinions summarization lies in presenting divergent or…
ArticlesOpinion SummarizationNeuS: Neutral Multi-News Summarization for Framing Bias Mitigation
Media framing bias can lead to increased political polarization, and thus, the need for automatic mitigation methods is growing. We propose a new task, a \textit{neutral} summary generation from multiple news articles of…
ArticlesNews SummarizationShaping Political Discourse using multi-source News Summarization
Multi-document summarization is the process of automatically generating a concise summary of multiple documents related to the same topic. This summary can help users quickly understand the key information from a large c…
Document SummarizationMulti-Document SummarizationNews SummarizationUncovering Political Bias in Large Language Models using Parliamentary Voting Records
As large language models (LLMs) become deeply embedded in digital platforms and decision-making systems, concerns about their political biases have grown. While substantial work has examined social biases such as gender …
NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias
Media news framing bias can increase political polarization and undermine civil society. The need for automatic mitigation methods is therefore growing. We propose a new task, a neutral summary generation from multiple n…
ArticlesDecoderMulti-Task LearningNews Summarization