Representation Problems in Linguistic Annotations: Ambiguity, Variation, Uncertainty, Error and Bias
The development of linguistic corpora is fraught with various problems of annotation and representation. These constitute a very real challenge for the development and use of annotated corpora, but as yet not much literature exists on how to address the underlying problems. In this paper, we identify and discuss five sources of representation problems, which are independent though interrelated: ambiguity, variation, uncertainty, error and bias. We outline and characterize these sources, discussing how their improper treatment can have stark consequences for research outcomes. Finally, we discuss how an adequate treatment can inform corpus-related linguistic research, both computational and theoretical, improving the reliability of research results and NLP models, as well as informing the more general reproducibility issue.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations
Named Entity Recognition (NER) is a key information extraction task with a long-standing tradition. While recent studies address and aim to correct annotation errors via re-labeling efforts, little is known about the sou…
Key Information Extractionnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1Representation of ambiguity in pretrained models and the problem of domain specificity
Recent developments in pretrained language models have led to many advances in NLP. These models have excelled at learning powerful contextual representations from very large corpora. Fine-tuning these models for downstr…
SpecificityVariational Adapter for Cross-modal Similarity Representation
The core of vision-language models lies in measuring cross-modal similarity within a unified representation space. However, most image-text matching or multi-class image classification datasets lack fine-grained cross-mo…
Domain GeneralizationBinary ClassificationImage ClassificationImage-text matchingRecursive Preferences and Ambiguity Attitudes
We study the implications of recursivity and state monotonicity in intertemporal consumption problems under ambiguity. We show that monotone recursive preferences admit a recursive and ex-ante representation, both with t…
TranslationZero and Few-shot Semantic Parsing with Ambiguous Inputs
Despite the frequent challenges posed by ambiguity when representing meaning via natural language, it is often ignored or deliberately removed in tasks mapping language to formally-designed representations, which general…
Semantic Parsing