Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding
To capture salient contextual information for spoken language understanding (SLU) of a dialogue, we propose time-aware models that automatically learn the latent time-decay function of the history without a manual time-decay function. We also propose a method to identify and label the current speaker to improve the SLU accuracy. In experiments on the benchmark dataset used in Dialog State Tracking Challenge 4, the proposed models achieved significantly higher F1 scores than the state-of-the-art contextual models. Finally, we analyze the effectiveness of the introduced models in detail. The analysis demonstrates that the proposed methods were effective to improve SLU accuracy individually.
Code (1)
Tasks
dialog state trackingSpoken Language UnderstandingSimilar Papers 제목 키워드 기반
Dynamically Context-Sensitive Time-Decay Attention for Dialogue Modeling
Spoken language understanding (SLU) is an essential component in conversational systems. Considering that contexts provide informative cues for better understanding, history can be leveraged for contextual SLU. However, …
Dialogue State TrackingSpoken Language UnderstandingMemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
Large language model (LLM) agents accumulate heterogeneous context, including system instructions, plans, user turns, retrieved documents, tool outputs, and intermediate reasoning, whose key-value (KV) cache can become a…
Position-Aware Depth Decay Decoding ($D^3$): Boosting Large Language Model Inference Efficiency
Due to the large number of parameters, the inference phase of Large Language Models (LLMs) is resource-intensive. Unlike traditional model compression, which needs retraining, recent dynamic computation methods show that…
GSM8KLanguage ModelingLanguage ModellingLarge Language Model+2How Time Matters: Learning Time-Decay Attention for Contextual Spoken Language Understanding in Dialogues
Spoken language understanding (SLU) is an essential component in conversational systems. Most SLU components treats each utterance independently, and then the following components aggregate the multi-turn information in …
Dialogue State TrackingImage CaptioningMachine TranslationSentence+3Remember Me: Bridging the Long-Range Gap in LVLMs with Three-Step Inference-Only Decay Resilience Strategies
Large Vision-Language Models (LVLMs) have achieved impressive performance across a wide range of multimodal tasks. However, they still face critical challenges in modeling long-range dependencies under the usage of Rotar…
Question Answering