Hierarchical Inductive Transfer for Continual Dialogue Learning
Pre-trained models have achieved excellent performance on the dialogue task. However, for the continual increase of online chit-chat scenarios, directly fine-tuning these models for each of the new tasks not only explodes the capacity of the dialogue system on the embedded devices but also causes knowledge forgetting on pre-trained models and knowledge interference between diverse dialogue tasks. In this work, we propose a hierarchical inductive transfer framework to learn and deploy the dialogue skills continually and efficiently. First, we introduce the adapter module into pre-trained models for learning new dialogue tasks. As the only trainable module, it is beneficial for the dialogue system on the embedded devices to acquire new dialogue skills with negligible additional parameters. Then, for alleviating knowledge interference between tasks yet benefiting the regularization between them, we further design hierarchical inductive transfer that enables new tasks to use general knowledge in the base adapter without being misled by diverse knowledge in task-specific adapters. Empirical evaluation and analysis indicate that our framework obtains comparable performance under deployment-friendly model capacity.
Code (0)
등록된 구현이 없습니다.
Tasks
General KnowledgeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hierarchical Inductive Transfer for Continual Dialogue Learning
Pre-trained models have achieved excellent performance on the dialogue task. However, for the continual increase of online chit-chat scenarios, directly fine-tuning these models for each of the new tasks not only explode…
General KnowledgeReconciling meta-learning and continual learning with online mixtures of tasks
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when ta…
Continual LearningFew-Shot LearningInductive BiasMeta-LearningMulti-aspect Depression Severity Assessment via Inductive Dialogue System
With the advancement of chatbots and the growing demand for automatic depression detection, identifying depression in patient conversations has gained more attention. However, prior methods often assess depression in a b…
Depression DetectionEmotion ClassificationResponse GenerationTaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation
A practical dialogue system requires the capacity for ongoing skill acquisition and adaptability to new tasks while preserving prior knowledge. However, current methods for Continual Dialogue State Tracking (DST), a cruc…
dialog state trackingDialogue State TrackingTransfer LearningIs forgetting less a good inductive bias for forward transfer?
One of the main motivations of studying continual learning is that the problem setting allows a model to accrue knowledge from past tasks to learn new tasks more efficiently. However, recent studies suggest that the key …
Continual Learningimage-classificationImage ClassificationInductive Bias