Back-Modality: Leveraging Modal Transformation for Data Augmentation
We introduce Back-Modality, a novel data augmentation schema predicated on modal transformation. Data from an initial modality undergoes transformation to an intermediate modality, followed by a reverse transformation. This framework serves dual roles. On one hand, it operates as a general data augmentation strategy. On the other hand, it allows for other augmentation techniques, suitable for the intermediate modality, to enhance the initial modality. For instance, data augmentation methods applicable to pure text can be employed to augment images, thereby facilitating the cross-modality of data augmentation techniques. To validate the viability and efficacy of our framework, we proffer three instantiations of Back-Modality: back-captioning, back-imagination, and back-speech. Comprehensive evaluations across tasks such as image classification, sentiment classification, and textual entailment demonstrate that our methods consistently enhance performance under data-scarce circumstances.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusion
Modality gap significantly restricts the effectiveness of multimodal fusion. Previous methods often use techniques such as diffusion models and adversarial learning to reduce the modality gap, but they typically focus on…
Diff$^2$I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior
Learning cross-modal correspondences is essential for image-to-point cloud (I2P) registration. Existing methods achieve this mostly by utilizing metric learning to enforce feature alignment across modalities, disregardin…
Point Cloud RegistrationMetric LearningFIRE: Unsupervised bi-directional inter-modality registration using deep networks
Inter-modality image registration is an critical preprocessing step for many applications within the routine clinical pathway. This paper presents an unsupervised deep inter-modality registration network that can learn t…
Image RegistrationHow Image Generation Helps Visible-to-Infrared Person Re-Identification?
Compared to visible-to-visible (V2V) person re-identification (ReID), the visible-to-infrared (V2I) person ReID task is more challenging due to the lack of sufficient training samples and the large cross-modality discrep…
Image GenerationPerson Re-IdentificationGeneralizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization
For medical image segmentation, imagine if a model was only trained using MR images in source domain, how about its performance to directly segment CT images in target domain? This setting, namely generalizable cross-mod…
Domain AdaptationDomain GeneralizationImage SegmentationMedical Image Segmentation+2