Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). Such representations are compositional and it is costly to collect responses for all possible combinations of atomic meaning schemata, thereby necessitating few-shot generalization to novel MRs. In this work, we systematically study the compositional generalization of the state-of-the-art T5 models in few-shot data-to-text tasks. We show that T5 models fail to generalize to unseen MRs, and we propose a template-based input representation that considerably improves the model's generalization capability. To further improve the model's performance, we propose an approach based on self-training using fine-tuned BLEURT for pseudo-response selection. On the commonly-used SGD and Weather benchmarks, the proposed self-training approach improves tree accuracy by $46\%+$ and reduces the slot error rates by $73\%+$ over the strong T5 baselines in few-shot settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Data-to-Text GenerationText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). Such representations are compositional and it is costly to collect responses for all possible…
Data-to-Text GenerationText GenerationCompositional Generalization from First Principles
Leveraging the compositional nature of our world to expedite learning and facilitate generalization is a hallmark of human perception. In machine learning, on the other hand, achieving compositional generalization has pr…
SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval
Compositional image retrieval (CIR) is a multimodal learning task where a model combines a query image with a user-provided text modification to retrieve a target image. CIR finds applications in a variety of domains inc…
Image RetrievalRetrievalTripletZero-Shot Composed Image Retrieval (ZS-CIR)Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation
Compositional generalization, representing the model's ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect con…
AttributeBenchmarkingMeta-LearningText GenerationHow Do In-Context Examples Affect Compositional Generalization?
Compositional generalization--understanding unseen combinations of seen primitives--is an essential reasoning capability in human intelligence. The AI community mainly studies this capability by fine-tuning neural networ…
In-Context Learning