Smaller Text Classifiers with Discriminative Cluster Embeddings
Word embedding parameters often dominate overall model sizes in neural methods for natural language processing. We reduce deployed model sizes of text classifiers by learning a hard word clustering in an end-to-end manner. We use the Gumbel-Softmax distribution to maximize over the latent clustering while minimizing the task loss. We propose variations that selectively assign additional parameters to words, which further improves accuracy while still remaining parameter-efficient.
Code (1)
Tasks
ClusteringSimilar Papers 제목 키워드 기반
An Empirical Study on Clustering Pretrained Embeddings: Is Deep Strictly Better?
Recent research in clustering face embeddings has found that unsupervised, shallow, heuristic-based methods -- including $k$-means and hierarchical agglomerative clustering -- underperform supervised, deep, inductive met…
ClusteringLearning Embeddings for Image Clustering: An Empirical Study of Triplet Loss Approaches
In this work, we evaluate two different image clustering objectives, k-means clustering and correlation clustering, in the context of Triplet Loss induced feature space embeddings. Specifically, we train a convolutional …
Clusteringimage-classificationImage ClassificationImage Clustering+1Neurally boosted supervised spectral clustering
Network embedding methods compute geometric representations of graphs that render various prediction problems amenable to machine learning techniques. Spectral network embeddings are based on the computation of eigenve…
ClusteringNetwork EmbeddingNode ClassificationShort Text Clustering with Transformers
Recent techniques for the task of short text clustering often rely on word embeddings as a transfer learning component. This paper shows that sentence vector representations from Transformers in conjunction with differen…
ClusteringSentenceShort Text ClusteringText Clustering+2Discriminative Embeddings of Latent Variable Models for Structured Data
Kernel classifiers and regressors designed for structured data, such as sequences, trees and graphs, have significantly advanced a number of interdisciplinary areas such as computational biology and drug design. Typicall…
Drug Design