SMILE: Evaluation and Domain Adaptation for Social Media Language Understanding

2023-06-30 · Vasilisa Bashlovkina, Riley Matthews, Zhaobin Kuang, Simon Baumgartner, Michael Bendersky

We study the ability of transformer-based language models (LMs) to understand social media language. Social media (SM) language is distinct from standard written language, yet existing benchmarks fall short of capturing LM performance in this socially, economically, and politically important domain. We quantify the degree to which social media language differs from conventional language and conclude that the difference is significant both in terms of token distribution and rate of linguistic shift. Next, we introduce a new benchmark for Social MedIa Language Evaluation (SMILE) that covers four SM platforms and eleven tasks. Finally, we show that learning a tokenizer and pretraining on a mix of social media and conventional language yields an LM that outperforms the best similar-sized alternative by 4.2 points on the overall SMILE score.

📄 PDF Abstract BibTeX arXiv:2307.00135

Code (0)

등록된 구현이 없습니다.

Tasks

Domain Adaptation

Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media

2021-04-16 · Findings (EMNLP) 2021 11 · Paul Röttger, Janet B. Pierrehumbert

Language use differs between domains and even within a domain, language use changes over time. For pre-trained language models like BERT, domain adaptation through continued pre-training has been shown to improve perform…

Document Classification Domain Adaptation General Classification Language Modelling

Real Time Monitoring of Social Media and Digital Press

2018-09-28 · Iñaki San Vicente, Xabier Saralegi, Rodrigo Agerri

Talaia is a platform for monitoring social media and digital press. A configurable crawler gathers content with respect to user defined domains or topics. Crawled data is processed by means of the EliXa Sentiment Analysi…

Data Visualization Domain Adaptation Sentiment Analysis

HadaSmileNet: Hadamard fusion of handcrafted and deep-learning features for enhancing facial emotion recognition of genuine smiles

2025-09-23 · Mohammad Junayed Hasan, Nabeel Mohammed, Shafin Rahman, Philipp Koehn arxiv

The distinction between genuine and posed emotions represents a fundamental pattern recognition challenge with significant implications for data mining applications in social sciences, healthcare, and human-computer inte…

Facial Emotion Recognition Computational Efficiency Multi-Task Learning

SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognition

2024-09-16 · Ming-Hao Hsu, Hung-Yi Lee

Automatic Speech Recognition (ASR) models demonstrate outstanding performance on high-resource languages but face significant challenges when applied to low-resource languages due to limited training data and insufficien…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Data Augmentation In-Context Learning+4

SoCRATES: Towards Reliable Automated Evaluation of Proactive LLM Mediation across Domains and Socio-cognitive Variations

2026-06-04 · Taewon Yun, Hyeonseong Park, Jeonghwan Choi, Hayoon Park 외 arxiv

Evaluating LLM mediators remains challenging, as mediation unfolds as a real-time trajectory shaped by disputants' shifting emotions, intentions, and context. Existing testbeds rely on a few expert-authored domains, vary…