FastText.zip: Compressing text classification models
We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory. After considering different solutions inspired by the hashing literature, we propose a method built upon product quantization to store word embeddings. While the original technique leads to a loss in accuracy, we adapt this method to circumvent quantization artefacts. Our experiments carried out on several benchmarks show that our approach typically requires two orders of magnitude less memory than fastText while being only slightly inferior with respect to accuracy. As a result, it outperforms the state of the art by a good margin in terms of the compromise between memory usage and accuracy.
Code (44)
Tasks
General ClassificationQuantizationText ClassificationWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HyperText: Endowing FastText with Hyperbolic Geometry
Natural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym relations in WordNet. FastText, as the state-of-the-art text classifier based on shallow neural network in Euclidean space, may…
ClassificationGeneral ClassificationText ClassificationAnalysis and Optimization of fastText Linear Text Classifier
The paper [1] shows that simple linear classifier can compete with complex deep learning algorithms in text classification applications. Combining bag of words (BoW) and linear classification techniques, fastText [1] att…
ClassificationDeep LearningGeneral Classificationtext-classification+1Bag of Tricks for Efficient Text Classification
This paper explores a simple and efficient baseline for text classification. Our experiments show that our fast text classifier fastText is often on par with deep learning classifiers in terms of accuracy, and many order…
CPUEmotion Recognition in ConversationGeneral ClassificationSentiment Analysis+1Comparative Analysis of Multilingual Text Classification & Identification through Deep Learning and Embedding Visualization
This research conducts a comparative study on multilingual text classification methods, utilizing deep learning and embedding visualization. The study employs LangDetect, LangId, FastText, and Sentence Transformer on a d…
ClassificationClusteringMultilingual text classificationSentence+2An Analysis of Hierarchical Text Classification Using Word Embeddings
Efficient distributed numerical word representation models (word embeddings) combined with modern machine learning algorithms have recently yielded considerable improvement on automatic document classification tasks. How…
BIG-bench Machine LearningClassificationDocument ClassificationGeneral Classification+3