Stopwords in Technical Language Processing
There are increasingly applications of natural language processing techniques for information retrieval, indexing and topic modelling in the engineering contexts. A standard component of such tasks is the removal of stopwords, which are uninformative components of the data. While researchers use readily available stopword lists which are derived for general English language, the technical jargon of engineering fields contains their own highly frequent and uninformative words and there exists no standard stopword list for technical language processing applications. Here we address this gap by rigorously identifying generic, insignificant, uninformative stopwords in engineering texts beyond the stopwords in general texts, based on the synthesis of alternative data-driven approaches, and curating a stopword list ready for technical language processing applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalRetrievalSimilar Papers 제목 키워드 기반
Text Categorization Can Enhance Domain-Agnostic Stopword Extraction
This paper investigates the role of text categorization in streamlining stopword extraction in natural language processing (NLP), specifically focusing on nine African languages alongside French. By leveraging the Masakh…
Text CategorizationCurating Stopwords in Marathi: A TF-IDF Approach for Improved Text Analysis and Information Retrieval
Stopwords are commonly used words in a language that are often considered to be of little value in determining the meaning or significance of a document. These words occur frequently in most texts and don't provide much …
Information RetrievalRetrievalSentiment Analysistext-classification+1Non-Zipfian Distribution of Stopwords or Function Words and Subset Selection Models
Stopwords and function words are relatively less informative for the content of a language and more often play a structural role in a sentence. Stopwords are ubiquitous words and may contain verbs, adjectives and adverbs…
ABARUAH at SemEval-2019 Task 5 : Bi-directional LSTM for Hate Speech Detection
In this paper, we present the results obtained using bi-directional long short-term memory (BiLSTM) with and without attention and Logistic Regression (LR) models for SemEval-2019 Task 5 titled {''}HatEval: Multilingual …
Hate Speech DetectionAn Assessment of Language Identification Methods on Tweets and Wikipedia Articles
Language identification is the task of determining the language which a given text is written. This task is important for Natural Language Processing and Information Retrieval activities. Two popular approaches for langu…
ArticlesInformation RetrievalLanguage IdentificationRetrieval