Ternary and Binary Quantization for Improved Classification
Dimension reduction and data quantization are two important methods for reducing data complexity. In the paper, we study the methodology of first reducing data dimension by random projection and then quantizing the projections to ternary or binary codes, which has been widely applied in classification. Usually, the quantization will seriously degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could provide comparable and often superior accuracy, as the data to be quantized are sparse features generated with common filters. Furthermore, this quantization property could be maintained in the random projections of sparse features, if both the features and random projection matrices are sufficiently sparse. By conducting extensive experiments, we validate and analyze this intriguing property.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationDimensionality ReductionQuantizationSimilar Papers 제목 키워드 기반
Smart Ternary Quantization
Neural network models are resource hungry. Low bit quantization such as binary and ternary quantization is a common approach to alleviate this resource requirements. Ternary quantization provides a more flexible model an…
image-classificationImage ClassificationQuantizationThe Binary and Ternary Quantization Can Improve Feature Discrimination
In machine learning, quantization is widely used to simplify data representation and facilitate algorithm deployment on hardware. Given the fundamental role of classification in machine learning, it is crucial to investi…
ClassificationQuantizationAdaptive Binary-Ternary Quantization
Neural network models are resource hungry. It is difficult to deploy such deep networks on devices with limited resources, like smart wearables, cellphones, drones, and autonomous vehicles. Low bit quantization such as b…
Autonomous VehiclesImage ClassificationQuantizationBinaryBERT: Pushing the Limit of BERT Quantization
The rapid development of large pre-trained language models has greatly increased the demand for model compression techniques, among which quantization is a popular solution. In this paper, we propose BinaryBERT, which pu…
BinarizationModel CompressionQuantizationTernary Quantization: A Survey
Inference time, model size, and accuracy are critical for deploying deep neural network models. Numerous research efforts have been made to compress neural network models with faster inference and higher accuracy. Prunin…
QuantizationSurvey