BrainStorm @ iREL at #SMM4H 2024: Leveraging Translation and Topical Embeddings for Annotation Detection in Tweets
The proliferation of LLMs in various NLP tasks has sparked debates regarding their reliability, particularly in annotation tasks where biases and hallucinations may arise. In this shared task, we address the challenge of distinguishing annotations made by LLMs from those made by human domain experts in the context of COVID-19 symptom detection from tweets in Latin American Spanish. This paper presents BrainStorm @ iRELs approach to the SMM4H 2024 Shared Task, leveraging the inherent topical information in tweets, we propose a novel approach to identify and classify annotations, aiming to enhance the trustworthiness of annotated data.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings
Recent research has demonstrated that vector space models of semantics can reflect undesirable biases in human culture. Our investigation of crosslinguistic word embeddings reveals that topical gender bias interacts with…
Cultural Vocal Bursts Intensity PredictionLemmatizationMachine TranslationTranslation+1LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback
Recent large language models (LLM) are leveraging human feedback to improve their generation quality. However, human feedback is costly to obtain, especially during inference. In this work, we propose LLMRefine, an infer…
Long Form Question AnsweringMachine TranslationQuestion AnsweringText Generation+1Topical: Learning Repository Embeddings from Source Code using Attention
This paper presents Topical, a novel deep neural network for repository level embeddings. Existing methods, reliant on natural language documentation or naive aggregation techniques, are outperformed by Topical's utiliza…
Semantic-Driven Topic Modeling for Analyzing Creativity in Virtual Brainstorming
Virtual brainstorming sessions have become a central component of collaborative problem solving, yet the large volume and uneven distribution of ideas often make it difficult to extract valuable insights efficiently. Man…
Dimensionality ReductionSemantic SimilarityJOINTLY LEARNING TOPIC SPECIFIC WORD AND DOCUMENT EMBEDDING
Document embedding generally ignores underlying topics, which fails to capture polysemous terms that can mislead to improper thematic representation. Moreover, embedding a new document during the test process needs a com…
Document ClassificationDocument EmbeddingWord Embeddings