BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities
We introduce BiGR, a novel conditional image generation model using compact binary latent codes for generative training, focusing on enhancing both generation and representation capabilities. BiGR is the first conditional generative model that unifies generation and discrimination within the same framework. BiGR features a binary tokenizer, a masked modeling mechanism, and a binary transcoder for binary code prediction. Additionally, we introduce a novel entropy-ordered sampling method to enable efficient image generation. Extensive experiments validate BiGR's superior performance in generation quality, as measured by FID-50k, and representation capabilities, as evidenced by linear-probe accuracy. Moreover, BiGR showcases zero-shot generalization across various vision tasks, enabling applications such as image inpainting, outpainting, editing, interpolation, and enrichment, without the need for structural modifications. Our findings suggest that BiGR unifies generative and discriminative tasks effectively, paving the way for further advancements in the field. We further enable BiGR to perform text-to-image generation, showcasing its potential for broader applications.
Code (1)
Tasks
Conditional Image GenerationImage GenerationImage InpaintingText to Image GenerationText-to-Image GenerationZero-shot GeneralizationSimilar Papers 제목 키워드 기반
A stability theorem for bigraded persistence barcodes
We define bigraded persistent homology modules and bigraded barcodes of a finite pseudo-metric space X using the ordinary and double homology of the moment-angle complex associated with the Vietoris-Rips filtration of X.…
Scalable Gaussian Processes for Supervised Hashing
We propose a flexible procedure for large-scale image search by hash functions with kernels. Our method treats binary codes and pairwise semantic similarity as latent and observed variables, respectively, in a probabilis…
Binary ClassificationGaussian ProcessesGeneral ClassificationImage Retrieval+2CASC: Condition-Aware Semantic Communication with Latent Diffusion Models
Diffusion-based semantic communication methods have shown significant advantages in image transmission by harnessing the generative power of diffusion models. However, they still face challenges, including generation ran…
DecoderImage ReconstructionSemantic CommunicationSupervised Learning of Semantics-Preserving Hash via Deep Convolutional Neural Networks
This paper presents a simple yet effective supervised deep hash approach that constructs binary hash codes from labeled data for large-scale image search. We assume that the semantic labels are governed by several latent…
AttributeClassificationDeep HashingGeneral Classification+2Cross-Modal Deep Variational Hashing
In this paper, we propose a cross-modal deep variational hashing (CMDVH) method to learn compact binary codes for cross-modality multimedia retrieval. Unlike most existing cross-modal hashing methods which learn a single…
Retrieval