Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection
Sarcasm fundamentally alters meaning through tone and context, yet detecting it in speech remains a challenge due to data scarcity. In addition, existing detection systems often rely on multimodal data, limiting their applicability in contexts where only speech is available. To address this, we propose an annotation pipeline that leverages large language models (LLMs) to generate a sarcasm dataset. Using a publicly available sarcasm-focused podcast, we employ GPT-4o and LLaMA 3 for initial sarcasm annotations, followed by human verification to resolve disagreements. We validate this approach by comparing annotation quality and detection performance on a publicly available sarcasm dataset using a collaborative gating architecture. Finally, we introduce PodSarc, a large-scale sarcastic speech dataset created through this pipeline. The detection model achieves a 73.63% F1 score, demonstrating the dataset's potential as a benchmark for sarcasm detection research.
Code (0)
등록된 구현이 없습니다.
Tasks
Sarcasm DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesis
Sarcastic speech synthesis, which involves generating speech that effectively conveys sarcasm, is essential for enhancing natural interactions in applications such as entertainment and human-computer interaction. However…
Sarcasm DetectionTransfer LearningSpeech SynthesisModeling Sarcastic Speech: Semantic and Prosodic Cues in a Speech Synthesis Framework
Sarcasm is a pragmatic phenomenon in which speakers convey meanings that diverge from literal content, relying on an interaction between semantics and prosodic expression. However, how these cues jointly contribute to th…
Speech SynthesisMarSan at SemEval-2022 Task 6: iSarcasm Detection via T5 and Sequence Learners
The paper describes SemEval-2022’s shared task “Intended Sarcasm Detection in English and Arabic.” This task includes English and Arabic tweets with sarcasm and non-sarcasm samples and irony speech labels.The first two s…
Sarcasm DetectionOptimism, Expectation, or Sarcasm? Multi-Class Hope Speech Detection in Spanish and English
Hope is a complex and underexplored emotional state that plays a significant role in education, mental health, and social interaction. Unlike basic emotions, hope manifests in nuanced forms ranging from grounded optimism…
Emotion RecognitionHope Speech DetectionMultiSoc-4D: A Benchmark for Diagnosing Instruction-Induced Label Collapse in Closed-Set LLM Annotation of Bengali Social Media
Annotation automation via Large Language Models (LLMs) is the core approach for scaling NLP datasets; however, LLM behavior with respect to closed-set instructions in low-resource languages has not been well studied. We …
Sarcasm Detection