Agent-Aware Dropout DQN for Safe and Efficient On-line Dialogue Policy Learning
Hand-crafted rules and reinforcement learning (RL) are two popular choices to obtain dialogue policy. The rule-based policy is often reliable within predefined scope but not self-adaptable, whereas RL is evolvable with data but often suffers from a bad initial performance. We employ a \textit{companion learning} framework to integrate the two approaches for \textit{on-line} dialogue policy learning, in which a pre-defined rule-based policy acts as a {``}teacher{''} and guides a data-driven RL system by giving example actions as well as additional rewards. A novel \textit{agent-aware dropout} Deep Q-Network (AAD-DQN) is proposed to address the problem of when to consult the teacher and how to learn from the teacher{'}s experiences. AAD-DQN, as a data-driven student policy, provides (1) two separate experience memories for student and teacher, (2) an uncertainty estimated by dropout to control the timing of consultation and learning. Simulation experiments showed that the proposed approach can significantly improve both \textit{safety}and \textit{efficiency} of on-line policy optimization compared to other companion learning approaches as well as supervised pre-training using static dialogue corpus.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech Recognition (ASR)Dialogue ManagementReinforcement LearningReinforcement Learning (RL)Speech RecognitionSpoken Dialogue SystemsSpoken Language UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis
Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems, demand reliable hazard identification. While large language models (LLMs) have shown promise in autom…
A Safety-Aware Role-Orchestrated Multi-Agent LLM Framework for Behavioral Health Communication Simulation
Single-agent large language model (LLM) systems struggle to simultaneously support diverse conversational functions and maintain safety in behavioral health communication. We propose a safety-aware, role-orchestrated mul…
Using In-Context Learning to Improve Dialogue Safety
While large neural-based conversational models have become increasingly proficient dialogue agents, recent work has highlighted safety issues with these systems. For example, these systems can be goaded into generating t…
In-Context LearningRe-RankingRetrievalDisruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network
Accurate detection and appropriate handling of disruptive talk in multi-party dialogue is essential for users to achieve shared goals. In collaborative game-based learning environments, detecting and attending to disrupt…
Agent-to-Agent Theory of Mind: Testing Interlocutor Awareness among Large Language Models
As large language models (LLMs) are increasingly integrated into multi-agent and human-AI systems, understanding their awareness of both self-context and conversational partners is essential for ensuring reliable perform…