Triadic Multi-party Voice Activity Projection for Turn-taking in Spoken Dialogue Systems
Turn-taking is a fundamental component of spoken dialogue, however conventional studies mostly involve dyadic settings. This work focuses on applying voice activity projection (VAP) to predict upcoming turn-taking in triadic multi-party scenarios. The goal of VAP models is to predict the future voice activity for each speaker utilizing only acoustic data. This is the first study to extend VAP into triadic conversation. We trained multiple models on a Japanese triadic dataset where participants discussed a variety of topics. We found that the VAP trained on triadic conversation outperformed the baseline for all models but that the type of conversation affected the accuracy. This study establishes that VAP can be used for turn-taking in triadic dialogue scenarios. Future work will incorporate this triadic VAP turn-taking model into spoken dialogue systems.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
MuVAP: Multimodal Multiparty Voice Activity Projection for Turn-taking Prediction in the Wild
Current multiparty turn-taking models often rely on complex microphone arrays or multi-camera setups, limiting their applicability in human-robot interaction scenarios. We introduce MuVAP, a causal multimodal framework t…
Voice Activity Projection: Self-supervised Learning of Turn-taking Events
The modeling of turn-taking in dialog can be viewed as the modeling of the dynamics of voice activity of the interlocutors. We extend prior work and define the predictive task of Voice Activity Projection, a general, sel…
Self-Supervised LearningMultimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders
Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pa…
Spatio-temporal activity patterns induced by triadic interactions in an in silico neural medium
Triadic interactions are general mechanisms by which a node or neuron can regulate directly the link or synapse between other two neurons. The regulation takes place in a familiar way by either depressing or facilitating…
Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection
A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialogue stereo audio to future voice activiti…
CPU