A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations
Recently, many studies are emerging towards building a retrieval-based dialogue system that is able to effectively leverage background knowledge (e.g., documents) when conversing with humans. However, it is non-trivial to collect large-scale dialogues that are naturally grounded on the background documents, which hinders the effective and adequate training of knowledge selection and response matching. To overcome the challenge, we consider decomposing the training of the knowledge-grounded response selection into three tasks including: 1) query-passage matching task; 2) query-dialogue history matching task; 3) multi-turn response matching task, and joint learning all these tasks in a unified pre-trained language model. The former two tasks could help the model in knowledge selection and comprehension, while the last task is designed for matching the proper response with the given query and background knowledge (dialogue history). By this means, the model can be learned to select relevant knowledge and distinguish proper response, with the help of ad-hoc retrieval corpora and a large number of ungrounded multi-turn dialogues. Experimental results on two benchmarks of knowledge-grounded response selection indicate that our model can achieve comparable performance with several existing methods that rely on crowd-sourced data for training.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModellingRetrievalTask 2Similar Papers 제목 키워드 기반
ZRIGF: An Innovative Multimodal Framework for Zero-Resource Image-Grounded Dialogue Generation
Image-grounded dialogue systems benefit greatly from integrating visual information, resulting in high-quality response generation. However, current models struggle to effectively utilize such information in zero-resourc…
Dialogue GenerationResponse GenerationReSLLM: Large Language Models are Strong Resource Selectors for Federated Search
Federated search, which involves integrating results from multiple independent search engines, will become increasingly pivotal in the context of Retrieval-Augmented Generation pipelines empowering LLM-based applications…
Retrieval-augmented GenerationGlobal Policy-Space Response Oracles for Two-Player Zero-Sum Games
The Policy-Space Response Oracles (PSRO) framework scales equilibrium computation to large zero-sum games by iteratively expanding a restricted strategy set using deep reinforcement learning (DRL). A central challenge is…
Reinforcement LearningSoft Layer Selection with Meta-Learning for Zero-Shot Cross-Lingual Transfer
Multilingual pre-trained contextual embedding models (Devlin et al., 2019) have achieved impressive performance on zero-shot cross-lingual transfer tasks. Finding the most effective fine-tuning strategy to fine-tune thes…
Cross-Lingual Natural Language InferenceCross-Lingual TransferMeta-LearningNatural Language Inference+1Collaborative planning and optimization for electric-thermal-hydrogen-coupled energy systems with portfolio selection of the complete hydrogen energy chain
Under the global low-carbon target, the uneven spatiotemporal distribution of renewable energy resources exacerbates the uncertainty and seasonal power imbalance. Additionally, the issue of an incomplete hydrogen energy …