Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion
Voice Activity Detection (VAD) plays a key role in speech processing, often utilizing hand-crafted or neural features. This study examines the effectiveness of Mel-Frequency Cepstral Coefficients (MFCCs) and pre-trained model (PTM) features, including wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, and Whisper. We propose FusionVAD, a unified framework that combines both feature types using three fusion strategies: concatenation, addition, and cross-attention (CA). Experimental results reveal that simple fusion techniques, particularly addition, outperform CA in both accuracy and efficiency. Fusion-based models consistently surpass single-feature models, highlighting the complementary nature of MFCCs and PTM features. Notably, our best-performing fusion model exceeds the state-of-the-art Pyannote across multiple datasets, achieving an absolute average improvement of 2.04%. These results confirm that simple feature fusion enhances VAD robustness while maintaining computational efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionActivity DetectionComputational EfficiencySimilar Papers 제목 키워드 기반
MLNET: An Adaptive Multiple Receptive-field Attention Neural Network for Voice Activity Detection
Voice activity detection (VAD) makes a distinction between speech and non-speech and its performance is of crucial importance for speech based services. Recently, deep neural network (DNN)-based VADs have achieved better…
Action DetectionActivity DetectionVSANet: Real-time Speech Enhancement Based on Voice Activity Detection and Causal Spatial Attention
The deep learning-based speech enhancement (SE) methods always take the clean speech's waveform or time-frequency spectrum feature as the learning target, and train the deep neural network (DNN) by reducing the error los…
Action DetectionActivity DetectionMulti-Task LearningSpeech EnhancementWindow Size Versus Accuracy Experiments in Voice Activity Detectors
Voice activity detection (VAD) plays a vital role in enabling applications such as speech recognition. We analyze the impact of window size on the accuracy of three VAD algorithms: Silero, WebRTC, and Root Mean Square (R…
Speech RecognitionActivity DetectionReal-time and Continuous Turn-taking Prediction Using Voice Activity Projection
A demonstration of a real-time and continuous turn-taking prediction system is presented. The system is based on a voice activity projection (VAP) model, which directly maps dialogue stereo audio to future voice activiti…
CPUReal-Time Voicemail Detection in Telephony Audio Using Temporal Speech Activity Features
Outbound AI calling systems must distinguish voicemail greetings from live human answers in real time to avoid wasted agent interactions and dropped calls. We present a lightweight approach that extracts 15 temporal feat…
Feature Importance