TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models

2023-04-18 · Yuwei Yin, Jean Kaddour, Xiang Zhang, Yixin Nie, Zhenguang Liu, Lingpeng Kong, Qi Liu

Data augmentation has been established as an efficacious approach to supplement useful information for low-resource datasets. Traditional augmentation techniques such as noise injection and image transformations have been widely used. In addition, generative data augmentation (GDA) has been shown to produce more diverse and flexible data. While generative adversarial networks (GANs) have been frequently used for GDA, they lack diversity and controllability compared to text-to-image diffusion models. In this paper, we propose TTIDA (Text-to-Text-to-Image Data Augmentation) to leverage the capabilities of large-scale pre-trained Text-to-Text (T2T) and Text-to-Image (T2I) generative models for data augmentation. By conditioning the T2I model on detailed descriptions produced by T2T models, we are able to generate photo-realistic labeled images in a flexible and controllable manner. Experiments on in-domain classification, cross-domain classification, and image captioning tasks show consistent improvements over other data augmentation baselines. Analytical studies in varied settings, including few-shot, long-tail, and adversarial, further reinforce the effectiveness of TTIDA in enhancing performance and increasing robustness.

📄 PDF Abstract BibTeX arXiv:2304.08821

Code (1)

yuweiyin/ttida 공식 구현 pytorch

Tasks

Data AugmentationDiversitydomain classificationImage Captioning

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Semantically Controllable Augmentations for Generalizable Robot Learning

2024-09-02 · Zoey Chen, Zhao Mandi, Homanga Bharadhwaj, Mohit Sharma 외

Generalization to unseen real-world scenarios for robot manipulation requires exposure to diverse datasets during training. However, collecting large real-world datasets is intractable due to high operational costs. For …

Data Augmentation Robot Manipulation

Enhanced Generative Data Augmentation for Semantic Segmentation via Stronger Guidance

2024-09-09 · Quang-Huy Che, Duc-Tri Le, Bich-Nga Pham, Duc-Khai Lam 외

Data augmentation is crucial for pixel-wise annotation tasks like semantic segmentation, where labeling requires significant effort and intensive labor. Traditional methods, involving simple transformations such as rotat…

Data Augmentation Segmentation Semantic Segmentation

ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detection

2025-10-17 · Haowei Zhu, Tianxiang Pan, Rui Qin, Jun-Hai Yong 외 arxiv

The scale and quality of datasets are crucial for training robust perception models. However, obtaining large-scale annotated data is both costly and time-consuming. Generative models have emerged as a powerful tool for …

Data Augmentation Object Detection

Enhancing Domain Generalization in 3D Human Pose Estimation through Controllable Generative Augmentation

2026-05-12 · Xinhao Hu, Yiyi Zhang, Liqing Zhang, Jianfu Zhang arxiv

Pedestrian motion, due to its causal nature, is strongly influenced by domain gaps arising from discrepancies between training and testing data distributions. Focusing on 3D human pose estimation, this work presents a co…

3D Human Pose Estimation Domain Generalization Video Generation

Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models

2026-07-02 · Stanislav Panev, Minhyek Jeon, Vaishnavi Khindkar, Ahish Deshpande 외 arxiv

Recent advances in large-scale image generative models enable photorealistic scene synthesis with controllable attributes. Beyond data augmentation, their potential as diagnostic tools for trained vision systems remains …

Data Augmentation