Unit-Based Agent for Semi-Cascaded Full-Duplex Dialogue Systems

2026-01-28 · Haoyuan Yu, Yuxuan Chen, Minjie Cai arxiv

Full-duplex voice interaction is crucial for natural human computer interaction. We present a framework that decomposes complex dialogue into minimal conversational units, enabling the system to process each unit independently and predict when to transit to the next. This framework is instantiated as a semi-cascaded full-duplex dialogue system built around a multimodal large language model, supported by auxiliary modules such as voice activity detection (VAD) and text-to-speech (TTS) synthesis. The resulting system operates in a train-free, plug-and-play manner. Experiments on the HumDial dataset demonstrate the effectiveness of our framework, which ranks second among all teams on the test set of the Human-like Spoken Dialogue Systems Challenge (Track 2: Full-Duplex Interaction). Code is available at the GitHub repository https://github.com/yu-haoyuan/fd-badcat.

📄 PDF Abstract BibTeX arXiv:2601.20230

Code (0)

등록된 구현이 없습니다.

Tasks

Activity Detection

DuplexCascade: Full-Duplex Speech-to-Speech Dialogue with VAD-Free Cascaded ASR-LLM-TTS Pipeline and Micro-Turn Optimization

2026-03-10 · Jianing Yang, Yusuke Fujita, Yui Sudo arxiv

Spoken dialog systems with cascaded ASR-LLM-TTS modules retain strong LLM intelligence, but VAD segmentation often forces half-duplex turns and brittle control. On the other hand, VAD-free end-to-end model support full-d…

VideoFDB: Evaluating Full-Duplex Vision-Speech Capabilities in Conversational Agents

2026-05-28 · Amrita Mazumdar, Seonwook Park, Rajarshi Roy, Nikhil Srihari 외 arxiv

Natural human conversation is full-duplex and audio-visual: people simultaneously speak and listen while continuously interpreting and producing nonverbal cues, such as nods, smiles, and gestures. To support successful h…

Visual Question Answering

Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

2026-04-06 · Guan-Ting Lin, Chen Chen, Zhehuai Chen, Hung-yi Lee arxiv

We introduce Full-Duplex-Bench-v3 (FDB-v3), a benchmark for evaluating spoken language models under naturalistic speech conditions and multi-step tool use. Unlike prior work, our dataset consists entirely of real human a…

Channel Estimation for Full-Duplex RIS-assisted HAPS Backhauling with Graph Attention Networks

2020-10-22 · Kürşat Tekbıyık, Güneş Karabulut Kurt, Chongwen Huang, Ali Rıza Ekti 외

In this paper, graph attention network (GAT) is firstly utilized for the channel estimation. In accordance with the 6G expectations, we consider a high-altitude platform station (HAPS) mounted reconfigurable intelligent …

Graph Attention

Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems

2022-05-30 · Ting-En Lin, Yuchuan Wu, Fei Huang, Luo Si 외

In this paper, we present Duplex Conversation, a multi-turn, multimodal spoken dialogue system that enables telephone-based agents to interact with customers like a human. We use the concept of full-duplex in telecommuni…

Data Augmentation Spoken Dialogue Systems