When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries
In an era where societal narratives are increasingly shaped by algorithmic curation, investigating the political neutrality of LLMs is an important research question. This study presents a fresh perspective on quantifying the political neutrality of LLMs through the lens of abstractive text summarization of polarizing news articles. We consider five pressing issues in current US politics: abortion, gun control/rights, healthcare, immigration, and LGBTQ+ rights. Via a substantial corpus of 20,344 news articles, our study reveals a consistent trend towards pro-Democratic biases in several well-known LLMs, with gun control and healthcare exhibiting the most pronounced biases (max polarization differences of -9.49% and -6.14%, respectively). Further analysis uncovers a strong convergence in the vocabulary of the LLM outputs for these divisive topics (55% overlap for Democrat-leaning representations, 52% for Republican). Being months away from a US election of consequence, we consider our findings important.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationArticlesText SummarizationSimilar Papers 제목 키워드 기반
NeutraSum: A Language Model can help a Balanced Media Diet by Neutralizing News Summaries
Media bias in news articles arises from the political polarisation of media outlets, which can reinforce societal stereotypes and beliefs. Reporting on the same event often varies significantly between outlets, reflectin…
ArticlesLanguage ModelingLanguage ModellingPolitical Neutrality in AI Is Impossible- But Here Is How to Approximate It
AI systems often exhibit political bias, influencing users' opinions and decisions. While political neutrality-defined as the absence of bias-is often seen as an ideal solution for fairness and safety, this position pape…
FairnessApolitical Intelligence? Auditing Delphi's responses on controversial political issues in the US
As generative language models are deployed in ever-wider contexts, concerns about their political values have come to the forefront with critique from all parts of the political spectrum that the models are biased and la…
EthicsLanguage ModelingLanguage ModellingLarge Language ModelRevealing Political Bias in LLMs through Structured Multi-Agent Debate
Large language models (LLMs) are increasingly used to simulate social behaviour, yet their political biases and interaction dynamics in debates remain underexplored. We investigate how LLM type and agent gender attribute…
Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini
As leading examples of large language models, ChatGPT and Gemini claim to provide accurate and unbiased information, emphasizing their commitment to political neutrality and avoidance of personal bias. This research inve…