FRIH: Fine-grained Region-aware Image Harmonization
Image harmonization aims to generate a more realistic appearance of foreground and background for a composite image. Existing methods perform the same harmonization process for the whole foreground. However, the implanted foreground always contains different appearance patterns. All the existing solutions ignore the difference of each color block and losing some specific details. Therefore, we propose a novel global-local two stages framework for Fine-grained Region-aware Image Harmonization (FRIH), which is trained end-to-end. In the first stage, the whole input foreground mask is used to make a global coarse-grained harmonization. In the second stage, we adaptively cluster the input foreground mask into several submasks by the corresponding pixel RGB values in the composite image. Each submask and the coarsely adjusted image are concatenated respectively and fed into a lightweight cascaded module, adjusting the global harmonization performance according to the region-aware local feature. Moreover, we further designed a fusion prediction module by fusing features from all the cascaded decoder layers together to generate the final result, which could utilize the different degrees of harmonization results comprehensively. Without bells and whistles, our FRIH algorithm achieves the best performance on iHarmony4 dataset (PSNR is 38.19 dB) with a lightweight model. The parameters for our model are only 11.98 M, far below the existing methods.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderImage HarmonizationSimilar Papers 제목 키워드 기반
AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages
Hate speech and abusive language are global phenomena that need socio-cultural background knowledge to be understood, identified, and moderated. However, in many regions of the Global South, there have been several docum…
Abusive LanguageKeyword SpottingAfriHG: News headline generation for African Languages
This paper introduces AfriHG -- a news headline generation dataset created by combining from XLSum and MasakhaNEWS datasets focusing on 16 languages widely spoken by Africa. We experimented with two seq2eq models (mT5-ba…
Headline GenerationAfriHuBERT: A self-supervised speech representation model for African languages
In this work, we present AfriHuBERT, an extension of mHuBERT-147, a compact self-supervised learning (SSL) model pretrained on 147 languages. While mHuBERT-147 covered 16 African languages, we expand this to 1,226 throug…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Cross-corpusLanguage Identification+4Cross Modal Fine-Grained Alignment via Granularity-Aware and Region-Uncertain Modeling
Fine-grained image-text alignment is a pivotal challenge in multimodal learning, underpinning key applications such as visual question answering, image captioning, and vision-language navigation. Unlike global alignment,…
Vision-Language NavigationVisual Question AnsweringImage CaptioningFireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model
Currently, instruction-based image editing methods have made significant progress by leveraging the powerful cross-modal understanding capabilities of vision language models (VLMs). However, they still face challenges in…
DenoisingLanguage ModelingLanguage Modelling