paper-with-me

홈 › Papers

A Context-Aware Hierarchical BERT Fusion Network for Multi-turn Dialog Act Detection

2021-09-03 · Ting-Wei Wu, Ruolin Su, Biing-Hwang Juang

The success of interactive dialog systems is usually associated with the quality of the spoken language understanding (SLU) task, which mainly identifies the corresponding dialog acts and slot values in each turn. By treating utterances in isolation, most SLU systems often overlook the semantic context in which a dialog act is expected. The act dependency between turns is non-trivial and yet critical to the identification of the correct semantic representations. Previous works with limited context awareness have exposed the inadequacy of dealing with complexity in multiproned user intents, which are subject to spontaneous change during turn transitions. In this work, we propose to enhance SLU in multi-turn dialogs, employing a context-aware hierarchical BERT fusion Network (CaBERT-SLU) to not only discern context information within a dialog but also jointly identify multiple dialog acts and slots in each utterance. Experimental results show that our approach reaches new state-of-the-art (SOTA) performances in two complicated multi-turn dialogue datasets with considerable improvements compared with previous methods, which only consider single utterances for multiple intents and slot filling.

📄 PDF Abstract BibTeX arXiv:2109.01267

Code (1)

waynewu6250/CaBERT-SLU 공식 구현 pytorch

Tasks

slot-fillingSlot FillingSpoken Language Understanding

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Multi-Head Attention 설명 없음
WordPiece 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Similar Papers 제목 키워드 기반

Emotion-Aware Embedding Fusion in LLMs (Flan-T5, LLAMA 2, DeepSeek-R1, and ChatGPT 4) for Intelligent Response Generation

2024-10-02 · Abdur Rasool, Muhammad Irfan Shahzad, Hafsa Aslam, Vincent Chan 외

Empathetic and coherent responses are critical in auto-mated chatbot-facilitated psychotherapy. This study addresses the challenge of enhancing the emotional and contextual understanding of large language models (LLMs) i…

ChatbotInformativenessResponse Generation

Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation

2024-03-06 · CVPR 2024 1 · Xiao Ma, Sumit Patidar, Iain Haughton, Stephen James

This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation. HDP factorises a manipulation policy into a hierarchical structure: a high-level task-planning agent wh…

PositionTask Planning

Hierarchical Classification of Financial Transactions Through Context-Fusion of Transformer-based Embeddings and Taxonomy-aware Attention Layer

2023-12-12 · Antonio J. G. Busson, Rafael Rocha, Rennan Gaio, Rafael Miceli 외

This work proposes the Two-headed DragoNet, a Transformer-based model for hierarchical multi-label classification of financial transactions. Our model is based on a stack of Transformers encoder layers that generate cont…

Hierarchical Multi-label ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATION

Partner Matters! An Empirical Study on Fusing Personas for Personalized Response Selection in Retrieval-Based Chatbots

2021-05-19 · Jia-Chen Gu, Hui Liu, Zhen-Hua Ling, Quan Liu 외

Persona can function as the prior knowledge for maintaining the consistency of dialogue systems. Most of previous studies adopted the self persona in dialogue whose response was about to be selected from a set of candida…

Retrieval

Hierarchical Multi-Stage BERT Fusion Framework with Dual Attention for Enhanced Cyberbullying Detection in Social Media

2025-03-01 · Jiani Wang, Xiaochuan Xu, Peiyang Yu, Zeqiu Xu

Detecting and classifying cyberbullying in social media is hard because of the complex nature of online language and the changing nature of content. This study presents a multi-stage BERT fusion framework. It uses hierar…