Learning Product Codebooks using Vector Quantized Autoencoders for Image Retrieval
Vector-Quantized Variational Autoencoders (VQ-VAE)[1] provide an unsupervised model for learning discrete representations by combining vector quantization and autoencoders. In this paper, we study the use of VQ-VAE for representation learning for downstream tasks, such as image retrieval. We first describe the VQ-VAE in the context of an information-theoretic framework. We show that the regularization term on the learned representation is determined by the size of the embedded codebook before the training and it affects the generalization ability of the model. As a result, we introduce a hyperparameter to balance the strength of the vector quantizer and the reconstruction error. By tuning the hyperparameter, the embedded bottleneck quantizer is used as a regularizer that forces the output of the encoder to share a constrained coding space such that learned latent features preserve the similarity relations of the data space. In addition, we provide a search range for finding the best hyperparameter. Finally, we incorporate the product quantization into the bottleneck stage of VQ-VAE and propose an end-to-end unsupervised learning model for the image retrieval task. The product quantizer has the advantage of generating large-size codebooks. Fast retrieval can be achieved by using the lookup tables that store the distance between any pair of sub-codewords. State-of-the-art retrieval results are achieved by the learned codebooks.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalQuantizationRepresentation LearningRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Group-Wise Optimization for Self-Extensible Codebooks in Vector Quantized Models
Vector Quantized Variational Autoencoders (VQ-VAEs) leverage self-supervised learning through reconstruction tasks to represent continuous vectors using the closest vectors in a codebook. However, issues such as codebook…
Self-Supervised LearningImage ReconstructionMGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental models that compress continuous visual data into discrete tokens. Existing methods have tried to improve the quantization strategy for better reconstruc…
2kImage GenerationImage ReconstructionQuantizationMGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental models that compress continuous visual data into discrete tokens. Existing methods have tried to improve the quantization strategy for better reconstruc…
2kQuantizationRobust Vector Quantized-Variational Autoencoder
Image generative models can learn the distributions of the training data and consequently generate examples by sampling from these distributions. However, when the training dataset is corrupted with outliers, generative …
DecoderQuantizationProbing neural audio codecs for distinctions among English nuclear tunes
State-of-the-art spoken dialogue models (Défossez et al. 2024; Schalkwyk et al. 2025) use neural audio codecs to "tokenize" audio signals into a lower-frequency stream of vectorial latent representations, each quantized …