Aggregated Learning: A Vector-Quantization Approach to Learning Neural Network Classifiers
We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call "IB learning". We show that IB learning is, in fact, equivalent to a special class of the quantization problem. The classical results in rate-distortion theory then suggest that IB learning can benefit from a "vector quantization" approach, namely, simultaneously learning the representations of multiple input objects. Such an approach assisted with some variational techniques, result in a novel learning framework, "Aggregated Learning", for classification with neural network models. In this framework, several objects are jointly classified by a single neural network. The effectiveness of this framework is verified through extensive experiments on standard image recognition and text classification tasks.
Code (1)
Tasks
ClassificationGeneral ClassificationQuantizationRepresentation Learningtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A review of learning vector quantization classifiers
In this work we present a review of the state of the art of Learning Vector Quantization (LVQ) classifiers. A taxonomy is proposed which integrates the most relevant LVQ approaches to date. The main concepts associated w…
QuantizationAggregated Learning: A Deep Learning Framework Based on Information-Bottleneck Vector Quantization
Based on the notion of information bottleneck (IB), we formulate a quantization problem called "IB quantization". We show that IB quantization is equivalent to learning based on the IB principle. Under this equivalence, …
Image ClassificationQuantizationtext-classificationText ClassificationAdditive Quantization for Extreme Vector Compression
We introduce a new compression scheme for high-dimensional vectors that approximates the vectors using sums of M codewords coming from M different codebooks. We show that the proposed scheme permits efficient distance an…
General Classificationimage-classificationImage ClassificationQuantizationGaussian Mixture Vector Quantization with Aggregated Categorical Posterior
The vector quantization is a widely used method to map continuous representation to discrete space and has important application in tokenization for generative mode, bottlenecking information and many other tasks in mach…
QuantizationExplaining Reject Options of Learning Vector Quantization Classifiers
While machine learning models are usually assumed to always output a prediction, there also exist extensions in the form of reject options which allow the model to reject inputs where only a prediction with an unacceptab…
counterfactualQuantization