Papers Spoken Dialogue Systems
“Spoken Dialogue Systems” 태그가 달린 논문 254편 · 필터 해제
Prompt-Guided Turn-Taking Prediction
Turn-taking prediction models are essential components in spoken dialogue systems and conversational robots. Recent approaches leverage transformer-based architectures to predict speech activity continuously and in real-…
Language ModelingLanguage ModellingLarge Language ModelPrediction+1Towards Efficient Speech-Text Jointly Decoding within One Speech Language Model
Speech language models (Speech LMs) enable end-to-end speech-text modelling within a single model, offering a promising direction for spoken dialogue systems. The choice of speech-text jointly decoding paradigm plays a c…
Language ModelingLanguage ModellingQuestion AnsweringSpoken Dialogue SystemsTowards a Japanese Full-duplex Spoken Dialogue System
Full-duplex spoken dialogue systems, which can model simultaneous bidirectional features of human conversations such as speech overlaps and backchannels, have attracted significant attention recently. However, the study …
Spoken Dialogue Systemstext-to-speechText to SpeechChain-of-Thought Training for Open E2E Spoken Dialogue Systems
Unlike traditional cascaded pipelines, end-to-end (E2E) spoken dialogue systems preserve full differentiability and capture non-phonemic information, making them well-suited for modeling spoken interactions. However, exi…
Language ModelingLanguage Modellingspeech-recognitionSpeech Recognition+5WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
End-to-end spoken dialogue models such as GPT-4o-audio have recently garnered significant attention in the speech domain. However, the evaluation of spoken dialogue models' conversational performance has largely been ove…
Spoken Dialogue SystemsSpeculative End-Turn Detector for Efficient Speech Chatbot Assistant
Spoken dialogue systems powered by large language models have demonstrated remarkable abilities in understanding human speech and generating appropriate spoken responses. However, these systems struggle with end-turn det…
ChatbotCollaborative InferenceSpoken Dialogue Systemstext-to-speech+1ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems
Advancements in audio foundation models (FMs) have fueled interest in end-to-end (E2E) spoken dialogue systems, but different web interfaces for each system makes it challenging to compare and contrast them effectively. …
DiversitySpoken Dialogue SystemsTalking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics
The recent wave of audio foundation models (FMs) could provide new capabilities for conversational modeling. However, there have been limited efforts to evaluate these audio FMs comprehensively on their ability to have n…
BenchmarkingSpoken Dialogue SystemsFlexDuo: A Pluggable System for Enabling Full-Duplex Capabilities in Speech Dialogue Systems
Full-Duplex Speech Dialogue Systems (Full-Duplex SDS) have significantly enhanced the naturalness of human-machine interaction by enabling real-time bidirectional communication. However, existing approaches face challeng…
Action DetectionActivity DetectionSpoken Dialogue SystemsLLM-Enhanced Dialogue Management for Full-Duplex Spoken Dialogue Systems
Achieving full-duplex communication in spoken dialogue systems (SDS) requires real-time coordination between listening, speaking, and thinking. This paper proposes a semantic voice activity detection (VAD) module as a di…
Action DetectionActivity DetectionDecision MakingDialogue Management+3Multimodal Transformer Models for Turn-taking Prediction: Effects on Conversational Dynamics of Human-Agent Interaction during Cooperative Gameplay
This study investigates multimodal turn-taking prediction within human-agent interactions (HAI), particularly focusing on cooperative gaming environments. It comprises both model development and subsequent user study, ai…
Spoken Dialogue SystemsAn LLM Benchmark for Addressee Recognition in Multi-modal Multi-party Dialogue
Handling multi-party dialogues represents a significant step for advancing spoken dialogue systems, necessitating the development of tasks specific to multi-party interactions. To address this challenge, we are construct…
Language ModelingLanguage ModellingLarge Language ModelSpoken Dialogue SystemsReal-Time Textless Dialogue Generation
Recent advancements in large language models (LLMs) have led to significant progress in text-based dialogue systems. These systems can now generate high-quality responses that are accurate and coherent across a wide rang…
Dialogue GenerationRhythmSpoken Dialogue SystemsOmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios
With the rapid development of large language models, researchers have created increasingly advanced spoken dialogue systems that can naturally converse with humans. However, these systems still struggle to handle the ful…
feature selectionSpoken Dialogue SystemsSLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training
Recent advancements highlight the potential of end-to-end real-time spoken dialogue systems, showcasing their low latency and high quality. In this paper, we introduce SLAM-Omni, a timbre-controllable, end-to-end voice i…
Spoken Dialogue SystemsWavChat: A Survey of Spoken Dialogue Models
Recent advancements in spoken dialogue models, exemplified by systems like GPT-4o, have captured significant attention in the speech domain. Compared to traditional three-tier cascaded spoken dialogue models that compris…
speech-recognitionSpeech RecognitionSpoken Dialogue SystemsSurvey+2OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
Full-duplex spoken dialogue systems significantly surpass traditional turn-based dialogue systems, as they allow simultaneous bidirectional communication, closely mirroring human-human interactions. However, achieving lo…
Large Language ModelSpoken Dialogue SystemsLarge Language Models Know What To Say But Not When To Speak
Turn-taking is a fundamental mechanism in human communication that ensures smooth and coherent verbal interactions. Recent advances in Large Language Models (LLMs) have motivated their use in improving the turn-taking ca…
Spoken Dialogue SystemsData Augmentation Integrating Dialogue Flow and Style to Adapt Spoken Dialogue Systems to Low-Resource User Groups
This study addresses the interaction challenges encountered by spoken dialogue systems (SDSs) when engaging with users who exhibit distinct conversational behaviors, particularly minors, in scenarios where data are scarc…
Data AugmentationLanguage ModelingLanguage ModellingLarge Language Model+1SaSLaW: Dialogue Speech Corpus with Audio-visual Egocentric Information Toward Environment-adaptive Dialogue Speech Synthesis
This paper presents SaSLaW, a spontaneous dialogue speech corpus containing synchronous recordings of what speakers speak, listen to, and watch. Humans consider the diverse environmental factors and then control the feat…
Speech SynthesisSpoken Dialogue Systemstext-to-speechText to Speech