Assessing the Verifiability of Attributions in News Text
When reporting the news, journalists rely on the statements of stakeholders, experts, and officials. The attribution of such a statement is verifiable if its fidelity to the source can be confirmed or denied. In this paper, we develop a new NLP task: determining the verifiability of an attribution based on linguistic cues. We operationalize the notion of verifiability as a score between 0 and 1 using human judgments in a comparison-based approach. Using crowdsourcing, we create a dataset of verifiability-scored attributions, and demonstrate a model that achieves an RMSE of 0.057 and Spearman{'}s rank correlation of 0.95 to human-generated scores. We discuss the application of this technique to the analysis of mass media.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringSimilar Papers 제목 키워드 기반
Assessing Partisan Traits of News Text Attributions
On the topic of journalistic integrity, the current state of accurate, impartial news reporting has garnered much debate in context to the 2016 US Presidential Election. In pursuit of computational evaluation of news tex…
Approaches for Enriching and Improving Textual Knowledge Bases
Verifiability is one of the core editing principles in Wikipedia, where editors are encouraged to provide citations for the added statements. Statements can be any arbitrary piece of text, ranging from a sentence up to a…
ArticlesSentenceUnifying Corroborative and Contributive Attributions in Large Language Models
As businesses, products, and services spring up around large language models, the trustworthiness of these models hinges on the verifiability of their outputs. However, methods for explaining language model outputs large…
Language ModelingLanguage ModellingLarge Language ModelMedical Question Answering+1GenerationPrograms: Fine-grained Attribution with Executable Programs
Recent large language models (LLMs) achieve impressive performance in source-conditioned text generation but often fail to correctly provide fine-grained attributions for their outputs, undermining verifiability and trus…
Document SummarizationLong Form Question AnsweringMulti-Document SummarizationQuestion Answering+1Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching ability
This paper addresses the critical challenge of assessing the representativeness of news thumbnail images, which often serve as the first visual engagement for readers when an article is disseminated on social media. We f…
Contrastive Learningcounterfactual