TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification
The experimental landscape in natural language processing for social media is too fragmented. Each year, new shared tasks and datasets are proposed, ranging from classics like sentiment analysis to irony detection or emoji prediction. Therefore, it is unclear what the current state of the art is, as there is no standardized evaluation protocol, neither a strong set of baselines trained on such domain-specific data. In this paper, we propose a new evaluation framework (TweetEval) consisting of seven heterogeneous Twitter-specific classification tasks. We also provide a strong set of baselines as starting point, and compare different language modeling pre-training strategies. Our initial experiments show the effectiveness of starting off with existing pre-trained generic language models, and continue training them on Twitter corpora.
Code (2)
Tasks
ClassificationGeneral ClassificationLanguage ModelingLanguage ModellingSentiment AnalysisSimilar Papers 제목 키워드 기반
SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research
Despite its relevance, the maturity of NLP for social media pales in comparison with general-purpose models, metrics and benchmarks. This fragmented landscape makes it hard for the community to know, for instance, given …
Language ModellingUnderstanding Textual Emotion Through Emoji Prediction
This project explores emoji prediction from short text sequences using four deep learning architectures: a feed-forward network, CNN, transformer, and BERT. Using the TweetEval dataset, we address class imbalance through…
Two-in-One: A Model Hijacking Attack Against Text Generation Models
Machine learning has progressed significantly in various applications ranging from face recognition to text generation. However, its success has been accompanied by different attacks. Recently a new attack has been propo…
ClassificationFace Recognitionimage-classificationImage Classification+8Cross-seed explainability using Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoders
We present a Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoder (SAE) for extracting cross-seed universal features from independently trained BERT models. Cross-seed feature universality is a fundamental ch…
Mutual Clustering on Comparative Texts via Heterogeneous Information Networks
Currently, many intelligence systems contain the texts from multi-sources, e.g., bulletin board system (BBS) posts, tweets and news. These texts can be ``comparative'' since they may be semantically correlated and thus p…
ClusteringText ClusteringTransfer Learning