From Awareness to Adherence: Bridging the Context Gap in Spoken Dialogue Systems via Context-Aware Decoding
Despite the success of end-to-end (E2E) spoken dialogue systems, maintaining strict context adherence in multi-round conversations remains a challenge. While prior works attribute these failures to models forgetting dialogue history, we highlight an equally critical but overlooked bottleneck: a gap between latent context awareness and active adherence. Although models internally recognize relevant past utterances, strong parametric priors often overshadow these signals during decoding. To bridge this gap, we propose an audio-adapted Context-Aware Decoding (CAD) approach. By leveraging internal attention mechanisms to isolate key historical rounds, our approach contrasts output distributions with and without this key context during inference, directly amplifying multimodal contextual signals. Evaluations on the Audio MultiChallenge benchmark demonstrate significant improvements in Semantic Memory and Self Coherence subtasks, successfully enforcing strict, context-faithful adherence.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Enhancing Dialogue Speech Recognition with Robust Contextual Awareness via Noise Representation Learning
Recent dialogue systems rely on turn-based spoken interactions, requiring accurate Automatic Speech Recognition (ASR). Errors in ASR can significantly impact downstream dialogue tasks. To address this, using dialogue con…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DecoderRepresentation Learning+2TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations
Task-oriented dialogue systems have been plagued by the difficulties of obtaining large-scale and high-quality annotated conversations. Furthermore, most of the publicly available datasets only include written conversati…
Data Augmentationspeech-recognitionSpeech RecognitionSpoken Dialogue Systems+1Approaching Dialogue State Tracking via Aligning Speech Encoders and LLMs
In this work, we approach spoken Dialogue State Tracking (DST) by bridging the representation spaces of speech encoders and LLMs via a small connector module, with a focus on fully open-sourced and open-data components (…
Dialogue State TrackingRoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue
Speech-to-speech dialogue models increasingly support persona control, yet existing spoken role-playing benchmarks remain largely character-centric and short-horizon. This leaves open whether spoken dialogue models can s…
Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models
Singlish, a Creole language rooted in English, is a key focus in linguistic research within multilingual and multicultural contexts. However, its spoken form remains underexplored, limiting insights into its linguistic s…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Question Answeringspeech-recognition+1