paper-with-me

홈 › Papers

DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning

2025-09-11 · Daniil Ignatev, Nan Li, Hugh Mee Wong, Anh Dang, Shane Kaszefski Yaschuk arxiv

This system paper presents the DeMeVa team's approaches to the third edition of the Learning with Disagreements shared task (LeWiDi 2025; Leonardelli et al., 2025). We explore two directions: in-context learning (ICL) with large language models, where we compare example sampling strategies; and label distribution learning (LDL) methods with RoBERTa (Liu et al., 2019b), where we evaluate several fine-tuning methods. Our contributions are twofold: (1) we show that ICL can effectively predict annotator-specific annotations (perspectivist annotations), and that aggregating these predictions into soft labels yields competitive performance; and (2) we argue that LDL methods are promising for soft label predictions and merit further exploration by the perspectivist community.

📄 PDF Abstract BibTeX arXiv:2509.09524

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

LeWiDi-2025 at NLPerspectives: Third Edition of the Learning with Disagreements Shared Task

2025-10-09 · Elisa Leonardelli, Silvia Casola, Siyao Peng, Giulia Rizzi 외 arxiv

Many researchers have reached the conclusion that AI models should be trained to be aware of the possibility of variation and disagreement in human judgments, and evaluated as per their ability to recognize such variatio…

Natural Language InferenceParaphrase IdentificationSarcasm Detection

LPI-RIT at LeWiDi-2025: Improving Distributional Predictions via Metadata and Loss Reweighting with DisCo

2025-08-11 · Mandira Sawkar, Samay U. Shetty, Deepak Pandita, Tharindu Cyril Weerasooriya 외 arxiv

The Learning With Disagreements (LeWiDi) 2025 shared task aims to model annotator disagreement through soft label distribution prediction and perspectivist evaluation, which focuses on modeling individual annotators. We …

Opt-ICL at LeWiDi-2025: Maximizing In-Context Signal from Rater Examples via Meta-Learning

2025-10-08 · Taylor Sorensen, Yejin Choi arxiv

Many natural language processing (NLP) tasks involve subjectivity, ambiguity, or legitimate disagreement between annotators. In this paper, we outline our system for modeling human variation. Our system leverages languag…

SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)

2023-04-28 · Elisa Leonardelli, Alexandra Uma, Gavin Abercrombie, Dina Almanea 외

NLP datasets annotated with human judgments are rife with disagreements between the judges. This is especially true for tasks depending on subjective judgments such as sentiment analysis or offensive language detection. …

Sentiment Analysis

BoN Appetit Team at LeWiDi-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet)

2025-10-14 · Tomas Ruiz, Siyao Peng, Barbara Plank, Carsten Schwemmer arxiv

Test-time scaling is a family of techniques to improve LLM outputs at inference time by performing extra computation. To the best of our knowledge, test-time scaling has been limited to domains with verifiably correct an…