Non-Standard Words as Features for Text Categorization
This paper presents categorization of Croatian texts using Non-Standard Words (NSW) as features. Non-Standard Words are: numbers, dates, acronyms, abbreviations, currency, etc. NSWs in Croatian language are determined according to Croatian NSW taxonomy. For the purpose of this research, 390 text documents were collected and formed the SKIPEZ collection with 6 classes: official, literary, informative, popular, educational and scientific. Text categorization experiment was conducted on three different representations of the SKIPEZ collection: in the first representation, the frequencies of NSWs are used as features; in the second representation, the statistic measures of NSWs (variance, coefficient of variation, standard deviation, etc.) are used as features; while the third representation combines the first two feature sets. Naive Bayes, CN2, C4.5, kNN, Classification Trees and Random Forest algorithms were used in text categorization experiments. The best categorization results are achieved using the first feature set (NSW frequencies) with the categorization accuracy of 87%. This suggests that the NSWs should be considered as features in highly inflectional languages, such as Croatian. NSW based features reduce the dimensionality of the feature space without standard lemmatization procedures, and therefore the bag-of-NSWs should be considered for further Croatian texts categorization experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
LemmatizationText CategorizationSimilar Papers 제목 키워드 기반
Leap-LSTM: Enhancing Long Short-Term Memory for Text Categorization
Recurrent Neural Networks (RNNs) are widely used in the field of natural language processing (NLP), ranging from text categorization to question answering and machine translation. However, RNNs generally read the whole t…
General ClassificationMachine TranslationQuestion AnsweringSentiment Analysis+3Text Categorization by Learning Predominant Sense of Words as Auxiliary Task
Distributions of the senses of words are often highly skewed and give a strong influence of the domain of a document. This paper follows the assumption and presents a method for text categorization by leveraging the pred…
Multi-Task LearningText CategorizationMultivariate Gaussian Document Representation from Word Embeddings for Text Categorization
Recently, there has been a lot of activity in learning distributed representations of words in vector spaces. Although there are models capable of learning high-quality distributed representations of words, how to genera…
Text CategorizationWord EmbeddingsArabic Text Categorization Algorithm using Vector Evaluation Method
Text categorization is the process of grouping documents into categories based on their contents. This process is important to make information retrieval easier, and it became more important due to the huge textual infor…
General ClassificationInformation RetrievalRetrievalText CategorizationImproving Human Needs Categorization of Events with Semantic Classification
Human Needs categories have been used to characterize the reason why an affective event is positive or negative. For example, {``}I got the flu{''} and {``}I got fired{''} are both negative (undesirable) events, but gett…
ClassificationGeneral Classification