Retrieval-guided Counterfactual Generation for QA
Deep NLP models have been shown to learn spurious correlations, leaving them brittle to input perturbations. Recent work has shown that counterfactual or contrastive data -- i.e. minimally perturbed inputs -- can reveal these weaknesses, and that data augmentation using counterfactuals can help ameliorate them. Proposed techniques for generating counterfactuals rely on human annotations, perturbations based on simple heuristics, and meaning representation frameworks. We focus on the task of creating counterfactuals for question answering, which presents unique challenges related to world knowledge, semantic diversity, and answerability. To address these challenges, we develop a Retrieve-Generate-Filter(RGF) technique to create counterfactual evaluation and training data with minimal human supervision. Using an open-domain QA framework and question generation model trained on original task data, we create counterfactuals that are fluent, semantically diverse, and automatically labeled. Data augmentation with RGF counterfactuals improves performance on out-of-domain and challenging evaluation sets over and above existing methods, in both the reading comprehension and open-domain QA settings. Moreover, we find that RGF data leads to significant improvements in a model's robustness to local perturbations.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualData AugmentationDiversityQuestion AnsweringQuestion GenerationQuestion-GenerationReading ComprehensionRetrievalWorld KnowledgeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Retrieval-guided Counterfactual Generation for QA
Deep NLP models have been shown to be brittle to input perturbations. Recent work has shown that data augmentation using counterfactuals --- i.e. minimally perturbed inputs --- can help ameliorate this weakness. We focus…
counterfactualData AugmentationDiversityQuestion Answering+5CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation
Counterfactual data augmentation (CDA) -- i.e., adding minimally perturbed inputs during training -- helps reduce model reliance on spurious correlations and improves generalization to out-of-distribution (OOD) data. Pri…
counterfactualData AugmentationDiversityFew-Shot Learning+7Causal-Counterfactual RAG: The Integration of Causal-Counterfactual Reasoning into RAG
Large language models (LLMs) have transformed natural language processing (NLP), enabling diverse applications by integrating large-scale pre-trained knowledge. However, their static knowledge limits dynamic reasoning ov…
Semantic SimilarityTriplet Synthesis For Enhancing Composed Image Retrieval via Counterfactual Image Generation
Composed Image Retrieval (CIR) provides an effective way to manage and access large-scale visual data. Construction of the CIR model utilizes triplets that consist of a reference image, modification text describing desir…
counterfactualImage GenerationImage RetrievalRetrieval+1Positional Segmentor-Guided Counterfactual Fine-Tuning for Spatially Localized Image Synthesis
Counterfactual image generation enables controlled data augmentation, bias mitigation, and disease modeling. However, existing methods guided by external classifiers or regressors are limited to subject-level factors (e.…
Data AugmentationImage Generation