Contextualizing Argument Quality Assessment with Relevant Knowledge
Automatic assessment of the quality of arguments has been recognized as a challenging task with significant implications for misinformation and targeted speech. While real-world arguments are tightly anchored in context, existing computational methods analyze their quality in isolation, which affects their accuracy and generalizability. We propose SPARK: a novel method for scoring argument quality based on contextualization via relevant knowledge. We devise four augmentations that leverage large language models to provide feedback, infer hidden assumptions, supply a similar-quality argument, or give a counter-argument. SPARK uses a dual-encoder Transformer architecture to enable the original argument and its augmentation to be considered jointly. Our experiments in both in-domain and zero-shot setups show that SPARK consistently outperforms existing techniques across multiple metrics.
Code (1)
Tasks
MisinformationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Quality Assessment of Online Reviews UsingFormal Argumentation Theory and Knowledge Graphs
In this extended abstract, we introduce a previously developed framework for the assessment of the quality of product reviews by using formal argumentation theory and machine learning. Then, we outline current developme…
BIG-bench Machine LearningKnowledge GraphsArgument Quality Assessment in the Age of Instruction-Following Large Language Models
The computational treatment of arguments on controversial issues has been subject to extensive NLP research, due to its envisioned impact on opinion formation, decision making, writing education, and the like. A critical…
Decision MakingDiversityInstruction FollowingIntrinsic Quality Assessment of Arguments
Several quality dimensions of natural language arguments have been investigated. Some are likely to be reflected in linguistic features (e.g., an argument's arrangement), whereas others depend on context (e.g., relevance…
Employing Argumentation Knowledge Graphs for Neural Argument Generation
Generating high-quality arguments, while being challenging, may benefit a wide range of downstream applications, such as writing assistants and argument search engines. Motivated by the effectiveness of utilizing knowled…
Knowledge GraphsText GenerationSimilarity-weighted Construction of Contextualized Commonsense Knowledge Graphs for Knowledge-intense Argumentation Tasks
Arguments often do not make explicit how a conclusion follows from its premises. To compensate for this lack, we enrich arguments with structured background knowledge to support knowledge-intense argumentation tasks. We …
Knowledge GraphsSemantic SimilaritySemantic Textual SimilarityTriplet