ProQA: Structural Prompt-based Pre-training for Unified Question Answering
Question Answering (QA) is a longstanding challenge in natural language processing. Existing QA works mostly focus on specific question types, knowledge domains, or reasoning skills. The specialty in QA research hinders systems from modeling commonalities between tasks and generalization for wider applications. To address this issue, we present ProQA, a unified QA paradigm that solves various tasks through a single model. ProQA takes a unified structural prompt as the bridge and improves the QA-centric ability by structural prompt-based pre-training. Through a structurally designed prompt-based input schema, ProQA concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task. Furthermore, ProQA is pre-trained with structural prompt-formatted large-scale synthesized corpus, which empowers the model with the commonly-required QA ability. Experimental results on 11 QA benchmarks demonstrate that ProQA consistently boosts performance on both full data fine-tuning, few-shot learning, and zero-shot testing scenarios. Furthermore, ProQA exhibits strong ability in both continual learning and transfer learning by taking the advantages of the structural prompt.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningFew-Shot LearningQuestion AnsweringTransfer LearningSimilar Papers 제목 키워드 기반
ProQA: Structural Prompt-based Pre-training for Unified Question Answering
Question Answering (QA) is a longstanding challenge in natural language processing. Existing QA works mostly focus on specific question types, knowledge domains, or reasoning skills. The specialty in QA research hinders …
Continual LearningFew-Shot LearningQuestion AnsweringTransfer LearningLiTransProQA: an LLM-based Literary Translation evaluation metric with Professional Question Answering
The impact of Large Language Models (LLMs) has extended into literary domains. However, existing evaluation metrics prioritize mechanical accuracy over artistic expression and tend to overrate machine translation (MT) as…
Machine TranslationQuestion AnsweringTranslationFew-shot Unified Question Answering: Tuning Models or Prompts?
Question-answering (QA) tasks often investigate specific question types, knowledge domains, or reasoning skills, leading to specialized models catering to specific categories of QA tasks. While recent research has explor…
Question AnsweringTransfer LearningGeological Everything Model 3D: A Promptable Foundation Model for Unified and Zero-Shot Subsurface Understanding
Understanding Earth's subsurface is critical for energy transition, natural hazard mitigation, and planetary science. Yet subsurface analysis remains fragmented, with separate models required for structural interpretatio…
Zero-shot GeneralizationRepresentation LearningSAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation
Graphs are able to model interconnected entities in many online services, supporting a wide range of applications on the Web. This raises an important question: How can we train a graph foundational model on multiple sou…
Domain Adaptation