Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization
In spite of the popularity of end-to-end diarization systems nowadays, modular systems comprised of voice activity detection (VAD), speaker embedding extraction plus clustering, and overlapped speech detection (OSD) plus handling still attain competitive performance in many conditions. However, one of the main drawbacks of modular systems is the need to run (and train) different modules independently. In this work, we propose an approach to jointly train a model to produce speaker embeddings, VAD and OSD simultaneously and reach competitive performance at a fraction of the inference time of a standard approach. Furthermore, the joint inference leads to a simplified overall pipeline which brings us one step closer to a unified clustering-based method that can be trained end-to-end towards a diarization-specific objective.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionActivity DetectionClusteringSimilar Papers 제목 키워드 기반
A Teacher-Student approach for extracting informative speaker embeddings from speech mixtures
We introduce a monaural neural speaker embeddings extractor that computes an embedding for each speaker present in a speech mixture. To allow for supervised training, a teacher-student approach is employed: the teacher c…
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
This paper presents a streaming speaker-attributed automatic speech recognition (SA-ASR) model that can recognize ``who spoke what'' with low latency even when multiple people are speaking simultaneously. Our model is ba…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Decoderspeaker-diarization+4High-resolution embedding extractor for speaker diarisation
Speaker embedding extractors significantly influence the performance of clustering-based speaker diarisation systems. Conventionally, only one embedding is extracted from each speech segment. However, because of the slid…
Vocal Bursts Intensity PredictionIn search of strong embedding extractors for speaker diarisation
Speaker embedding extractors (EEs), which map input audio to a speaker discriminant latent space, are of paramount importance in speaker diarisation. However, there are several challenges when adopting EEs for diarisatio…
Data AugmentationSpeaker VerificationSelf-supervised speaker embeddings
Contrary to i-vectors, speaker embeddings such as x-vectors are incapable of leveraging unlabelled utterances, due to the classification loss over training speakers. In this paper, we explore an alternative training stra…
DecoderGeneral Classification