Best Preprocessing Techniques for Sentiment Analysis
Sentiment analysis in Twitter datasets is important because it enables monitoring public opinion on products and analysis of political and social movements. One critical step is preprocessing: the automated processing of text for machine learning algorithms. Preprocessing plays a critical role in reducing noise and improving efficiency. However, little research has systematically examined the order in which preprocessing techniques are implemented. We find that, when accounting for order, spelling correction is the least impactful preprocessing technique, whereas tokenisation is the most impactful. Stemming and stop-word removal are interchangeable, and it is better to remove stop words without removing negation. The best order for applying the preprocessing techniques was tokenisation, text cleaning, stemming, and then stopword removal. Our results provide a systematic approach for practitioners to deploy preprocessing to improve model output without the costly preprocessing exploratory phase.
Code (0)
등록된 구현이 없습니다.
Tasks
Sentiment AnalysisSimilar Papers 제목 키워드 기반
On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysis
Text preprocessing is often the first step in the pipeline of a Natural Language Processing (NLP) system, with potential impact in its final performance. Despite its importance, text preprocessing has not received much a…
Sentiment AnalysisText CategorizationText ClassificationWord EmbeddingsLexicon-based Sentiment Analysis in German: Systematic Evaluation of Resources and Preprocessing Techniques
IUST at SemEval-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text using Deep Neural Networks and Linear Baselines
Sentiment Analysis is a well-studied field of Natural Language Processing. However, the rapid growth of social media and noisy content within them poses significant challenges in addressing this problem with well-establi…
Sentiment AnalysisSentiment Analysis in SemEval: A Review of Sentiment Identification Approaches
Social media platforms are becoming the foundations of social interactions including messaging and opinion expression. In this regard, Sentiment Analysis techniques focus on providing solutions to ensure the retrieval an…
Sentiment AnalysisWord EmbeddingsA Comprehensive Analysis of Preprocessing for Word Representation Learning in Affective Tasks
Affective tasks such as sentiment analysis, emotion classification, and sarcasm detection have been popular in recent years due to an abundance of user-generated data, accurate computational linguistic models, and a broa…
Emotion ClassificationRepresentation LearningSarcasm DetectionSentiment Analysis