MixPro: Simple yet Effective Data Augmentation for Prompt-based Learning

2023-04-19 · Bohan Li, Longxu Dou, Yutai Hou, Yunlong Feng, Honglin Mu, Qingfu Zhu, Qinghua Sun, Wanxiang Che

Prompt-based learning has shown considerable promise in reformulating various downstream tasks as cloze problems by combining original input with a predetermined template. This approach demonstrates its effectiveness, especially in few-shot learning scenarios, where the model is trained on a scarce amount of data. Despite its successes, the limited templates and text in few-shot prompt-based learning scenarios leave significant room for performance improvement. Moreover, existing methods sometimes resort to model ensembles, which, while effective, could potentially hamper model efficiency due to increased computational demands. To address these issues, we introduce MixPro, an augmentation method designed to augment both the vanilla input text and the templates. We implement this through the token-level, the sentence-level, and the template-level Mixup strategies. The experimental results on five few-shot datasets show that MixPro outperforms other augmentation baselines, improving model performance by an average of 5.08% compared to before augmentation.

📄 PDF Abstract BibTeX arXiv:2304.09402

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation Few-Shot LearningSentence

Methods 이 논문이 사용한 방법론

Mixup Mixup is a data augmentation technique that generates a weighted combination of random image pairs from the training data. Given two images and their ground truth labels:…

MixPro: Data Augmentation with MaskMix and Progressive Attention Labeling for Vision Transformer

2023-04-24 · QiHao Zhao, Yangyu Huang, Wei Hu, Fan Zhang 외

The recently proposed data augmentation TransMix employs attention labels to help visual transformers (ViT) achieve better robustness and performance. However, TransMix is deficient in two aspects: 1) The image cropping …

Data Augmentation Image Augmentation Image ClassiData Augmentationfication Image Classification+5

Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings

2023-05-23 · Yihao Xue, Ali Payani, Yu Yang, Baharan Mirzasoleiman

Pretrained machine learning models need to be adapted to distribution shifts when deployed in new target environments. When obtaining labeled data from the target distribution is expensive, few-shot adaptation with only …

SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Data

2023-11-06 · Ruoxi Sun, Sercan Ö. Arik, Rajarishi Sinha, Hootan Nakhost 외

Text-to-SQL aims to automate the process of generating SQL queries on a database from natural language text. In this work, we propose "SQLPrompt", tailored to improve the few-shot prompting capabilities of Text-to-SQL fo…

In-Context Learning Text to SQL Text-To-SQL

SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection

2026-03-23 · Kexian Tang, Jiani Wang, Shaowen Wang, Kaifeng Lyu arxiv

While large language models (LLMs) are pretrained on massive amounts of data, their knowledge coverage remains incomplete in specialized, data-scarce domains, motivating extensive efforts to study synthetic data generati…

Synthetic Data Generation Data Augmentation

CoDa: Constrained Generation based Data Augmentation for Low-Resource NLP

2024-03-30 · Chandra Kiran Reddy Evuru, Sreyan Ghosh, Sonal Kumar, Ramaneswaran S 외

We present CoDa (Constrained Generation based Data Augmentation), a controllable, effective, and training-free data augmentation technique for low-resource (data-scarce) NLP. Our approach is based on prompting off-the-sh…

Data Augmentation Instruction Following