Online Exemplar Fine-Tuning for Image-to-Image Translation
Existing techniques to solve exemplar-based image-to-image translation within deep convolutional neural networks (CNNs) generally require a training phase to optimize the network parameters on domain-specific and task-specific benchmarks, thus having limited applicability and generalization ability. In this paper, we propose a novel framework, for the first time, to solve exemplar-based translation through an online optimization given an input image pair, called online exemplar fine-tuning (OEFT), in which we fine-tune the off-the-shelf and general-purpose networks to the input image pair themselves. We design two sub-networks, namely correspondence fine-tuning and multiple GAN inversion, and optimize these network parameters and latent codes, starting from the pre-trained ones, with well-defined loss functions. Our framework does not require the off-line training phase, which has been the main challenge of existing methods, but the pre-trained networks to enable optimization in online. Experimental results prove that our framework is effective in having a generalization power to unseen image pairs and clearly even outperforms the state-of-the-arts needing the intensive training phase.
Code (0)
등록된 구현이 없습니다.
Tasks
Image-to-Image TranslationTranslationSimilar Papers 제목 키워드 기반
Cos R-CNN for Online Few-shot Object Detection
We propose Cos R-CNN, a simple exemplar-based R-CNN formulation that is designed for online few-shot object detection. That is, it is able to localise and classify novel object categories in images with few examples with…
Few-Shot Object DetectionObjectobject-detectionObject DetectionOVMR: Open-Vocabulary Recognition with Multi-Modal References
The challenge of open-vocabulary recognition lies in the model has no clue of new categories it is applied to. Existing works have proposed different methods to embed category cues into the model, \eg, through few-shot f…
Open Vocabulary Object DetectionExemplar Masking for Multimodal Incremental Learning
Multimodal incremental learning needs to digest the information from multiple modalities while concurrently learning new knowledge without forgetting the previously learned information. There are numerous challenges for …
Data AugmentationIncremental LearningPrompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning
Replay-based methods in class-incremental learning~(CIL) have attained remarkable success. Despite their effectiveness, the inherent memory restriction results in saving a limited number of exemplars with poor diversity.…
class-incremental learningClass Incremental LearningData AugmentationDiversity+1Random Representations Outperform Online Continually Learned Representations
Continual learning has primarily focused on the issue of catastrophic forgetting and the associated stability-plasticity tradeoffs. However, little attention has been paid to the efficacy of continually learned represent…
Continual LearningExemplar-FreeRepresentation Learning