Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation
The audio segmentation mismatch between training data and those seen at run-time is a major problem in direct speech translation. Indeed, while systems are usually trained on manually segmented corpora, in real use cases they are often presented with continuous audio requiring automatic (and sub-optimal) segmentation. After comparing existing techniques (VAD-based, fixed-length and hybrid segmentation methods), in this paper we propose enhanced hybrid solutions to produce better results without sacrificing latency. Through experiments on different domains and language pairs, we show that our methods outperform all the other techniques, reducing by at least 30% the gap between the traditional VAD-based approach and optimal manual segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionActivity DetectionSegmentationTranslationSimilar Papers 제목 키워드 기반
Kernel-based Sensor Fusion with Application to Audio-Visual Voice Activity Detection
In this paper, we address the problem of multiple view data fusion in the presence of noise and interferences. Recent studies have approached this problem using kernel methods, by relying particularly on a product of ker…
Action DetectionActivity DetectionSensor FusionIntel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarization
This report describes our approach for the Audio-Visual Diarization (AVD) task of the Ego4D Challenge 2022. Specifically, we present multiple technical improvements over the official baselines. First, we improve the dete…
Action DetectionActive Speaker DetectionActivity DetectionSpeaker Embeddings as Individuality Proxy for Voice Stress Detection
Since the mental states of the speaker modulate speech, stress introduced by cognitive or physical loads could be detected in the voice. The existing voice stress detection benchmark has shown that the audio embeddings e…
Multi-Input Multi-Output Target-Speaker Voice Activity Detection For Unified, Flexible, and Robust Audio-Visual Speaker Diarization
Audio-visual learning has demonstrated promising results in many classical speech tasks (e.g., speech separation, automatic speech recognition, wake-word spotting). We believe that introducing visual modality will also b…
Action DetectionActivity Detectionaudio-visual learningAutomatic Speech Recognition+5Multi-Stage Speaker Diarization for Noisy Classrooms
Speaker diarization, the process of identifying "who spoke when" in audio recordings, is essential for understanding classroom dynamics. However, classroom settings present distinct challenges, including poor recording q…
Action DetectionActivity DetectionAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)+5