SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research
Despite its relevance, the maturity of NLP for social media pales in comparison with general-purpose models, metrics and benchmarks. This fragmented landscape makes it hard for the community to know, for instance, given a task, which is the best performing model and how it compares with others. To alleviate this issue, we introduce a unified benchmark for NLP evaluation in social media, SuperTweetEval, which includes a heterogeneous set of tasks and datasets combined, adapted and constructed from scratch. We benchmarked the performance of a wide range of models on SuperTweetEval and our results suggest that, despite the recent advances in language modelling, social media remains challenging.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
When Heterophily Meets Heterogeneity: Challenges and a New Large-Scale Graph Benchmark
Graph mining has become crucial in fields such as social science, finance, and cybersecurity. Many large-scale real-world networks exhibit both heterogeneity, where multiple node and edge types exist in the graph, and he…
BenchmarkingGraph LearningGraph MiningModel Selection+1Heterogeneous Social Event Detection via Hyperbolic Graph Representations
Social events reflect the dynamics of society and, here, natural disasters and emergencies receive significant attention. The timely detection of these events can provide organisations and individuals with valuable infor…
Contrastive LearningEvent DetectionGraph Learning Augmented Heterogeneous Graph Neural Network for Social Recommendation
Social recommendation based on social network has achieved great success in improving the performance of recommendation system. Since social network (user-user relations) and user-item interactions are both naturally rep…
Graph LearningGraph Neural NetworkBenchmarking Multi-Modal Graph-based Social Media Popularity Prediction
Social media popularity prediction aims to forecast the future reach or influence of online content from early-stage observations. Accurate prediction enables key downstream applications, such as advertising optimization…
Social Media Popularity PredictionKnowledge-Grounded Dialogue Generation with a Unified Knowledge Representation
Knowledge-grounded dialogue systems are challenging to build due to the lack of training data and heterogeneous knowledge sources. Existing systems perform poorly on unseen topics due to limited topics covered in the tra…
Dialogue GenerationLanguage ModelingLanguage Modelling