Sentiment analysis is not solved! Assessing and probing sentiment classification
Neural methods for SA have led to quantitative improvements over previous approaches, but these advances are not always accompanied with a thorough analysis of the qualitative differences. Therefore, it is not clear what outstanding conceptual challenges for sentiment analysis remain. In this work, we attempt to discover what challenges still prove a problem for sentiment classifiers for English and to provide a challenging dataset. We collect the subset of sentences that an (oracle) ensemble of state-of-the-art sentiment classifiers misclassify and then annotate them for 18 linguistic and paralinguistic phenomena, such as negation, sarcasm, modality, etc. The dataset is available at https://github.com/ltgoslo/assessing_and_probing_sentiment. Finally, we provide a case study that demonstrates the usefulness of the dataset to probe the performance of a given sentiment classifier with respect to linguistic phenomena.
Code (1)
Tasks
ClassificationGeneral ClassificationNegationSentiment AnalysisSentiment ClassificationSimilar Papers 제목 키워드 기반
Reassessing the Role of Chain-of-Thought in Sentiment Analysis: Insights and Limitations
The relationship between language and thought remains an unresolved philosophical issue. Existing viewpoints can be broadly categorized into two schools: one asserting their independence, and another arguing that languag…
counterfactualSentiment AnalysisLLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing
Large Language Models (LLMs) have rapidly become central to NLP, demonstrating their ability to adapt to various tasks through prompting techniques, including sentiment analysis. However, we still have a limited understa…
Sentiment AnalysisA New Approach To Text Rating Classification Using Sentiment Analysis
Typical use cases of sentiment analysis usually revolve around assessing the probability of a text belonging to a certain sentiment and deriving insight concerning it; little work has been done to explore further use cas…
ClassificationGeneral ClassificationSentiment AnalysisTasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis
Aspect-based sentiment analysis (ABSA) aims to predict the sentiment towards a specific aspect in the text. However, existing ABSA test sets cannot be used to probe whether a model can distinguish the sentiment of the ta…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Sentiment AnalysisMulti-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis
Related tasks often have inter-dependence on each other and perform better when solved in a joint framework. In this paper, we present a deep multi-task learning framework that jointly performs sentiment and emotion anal…
Decision MakingEmotion RecognitionMulti-Task LearningSentiment Analysis