Papers Speech Representation Learning
“Speech Representation Learning” 태그가 달린 논문 131편 · 필터 해제
HYFuse: Aligning Heterogeneous Speech Pre-Trained Representations in Hyperbolic Space for Speech Emotion Recognition
Compression-based representations (CBRs) from neural audio codecs such as EnCodec capture intricate acoustic features like pitch and timbre, while representation-learning-based representations (RLRs) from pre-trained mod…
Emotion RecognitionRepresentation LearningSpeech Emotion RecognitionSpeech Representation LearningDuRep: Dual-Mode Speech Representation Learning via ASR-Aware Distillation
Recent advancements in speech encoders have drawn attention due to their integration with Large Language Models for various speech tasks. While most research has focused on either causal or full-context speech encoders, …
Representation LearningSpeech Representation LearningUniversal Semantic Disentangled Privacy-preserving Speech Representation Learning
The use of audio recordings of human speech to train LLMs poses privacy concerns due to these models' potential to generate outputs that closely resemble artifacts in the training data. In this study, we propose a speake…
DecoderPrivacy PreservingRepresentation LearningSpeaker anonymization+1UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation
Pre-training and representation learning have been playing an increasingly important role in modern speech processing. Nevertheless, different applications have been relying on different foundation models, since predomin…
DecoderRepresentation Learningspeech-recognitionSpeech Recognition+4Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation
Audio-visual speech recognition (AVSR) incorporates auditory and visual modalities to improve recognition accuracy, particularly in noisy environments where audio-only speech systems are insufficient. While previous rese…
Audio-Visual Speech RecognitionMulti-Task LearningRepresentation Learningspeech-recognition+3k2SSL: A Faster and Better Framework for Self-Supervised Speech Representation Learning
Self-supervised learning (SSL) has achieved great success in speech-related tasks. While Transformer and Conformer architectures have dominated SSL backbones, encoders like Zipformer, which excel in automatic speech reco…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)GPURepresentation Learning+4EH-MAM: Easy-to-Hard Masked Acoustic Modeling for Self-Supervised Speech Representation Learning
In this paper, we present EH-MAM (Easy-to-Hard adaptive Masked Acoustic Modeling), a novel self-supervised learning approach for speech representation learning. In contrast to the prior methods that use random masking sc…
Representation LearningSelf-Supervised Learningspeech-recognitionSpeech Recognition+1JOOCI: a Framework for Learning Comprehensive Speech Representations
Information in speech can be divided into two categories: what is being said (content) and how it is expressed (other). Current state-of-the-art (SOTA) techniques model speech at fixed segments, usually 10-25 ms, using a…
Representation LearningSpeech Representation LearningAre Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models
In this study, for the first time, we extensively investigate whether music foundation models (MFMs) or speech foundation models (SFMs) work better for singing voice deepfake detection (SVDD), which has recently attracte…
DeepFake DetectionFace SwappingRepresentation LearningSpeaker Recognition+1Self-Supervised Syllable Discovery Based on Speaker-Disentangled HuBERT
Self-supervised speech representation learning has become essential for extracting meaningful features from untranscribed audio. Recent advances highlight the potential of deriving discrete symbols from the features corr…
Acoustic Unit DiscoveryClusteringData AugmentationRepresentation Learning+3Progressive Residual Extraction based Pre-training for Speech Representation Learning
Self-supervised learning (SSL) has garnered significant attention in speech processing, excelling in linguistic tasks such as speech recognition. However, jointly improving the performance of pre-trained models on variou…
Emotion RecognitionRepresentation LearningSelf-Supervised LearningSpeaker Identification+5Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data Augmentation
Speech modeling methods learn one embedding for a fixed segment of speech, typically in between 10-25 ms. The information present in speech can be divided into two categories: "what is being said" (content) and "how it i…
Data AugmentationRepresentation LearningSelf-Supervised LearningSpeech Representation LearningTowards the Next Frontier in Speech Representation Learning Using Disentanglement
The popular frameworks for self-supervised learning of speech representations have largely focused on frame-level masked prediction of speech regions. While this has shown promising downstream task performance for speech…
DisentanglementRepresentation LearningSelf-Supervised Learningspeech-recognition+2Towards Robust Speech Representation Learning for Thousands of Languages
Self-supervised learning (SSL) has helped extend speech technologies to more languages by reducing the need for labeled data. However, models are still far from supporting the world's 7000+ languages. We propose XEUS, a …
Representation LearningSelf-Supervised LearningSpeech Representation LearningmHuBERT-147: A Compact Multilingual HuBERT Model
We present mHuBERT-147, the first general-purpose massively multilingual HuBERT speech representation model trained on 90K hours of clean, open-license data. To scale up the multi-iteration HuBERT approach, we use faiss-…
Automatic Speech Recognition (ASR)DiversitymodelSpeech Language Identification+1Emotion-Aware Speech Self-Supervised Representation Learning with Intensity Knowledge
Speech Self-Supervised Learning (SSL) has demonstrated considerable efficacy in various downstream tasks. Nevertheless, prevailing self-supervised models often overlook the incorporation of emotion-related prior informat…
Representation LearningSelf-Supervised LearningSpeech Representation LearningXLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception
Speech recognition and translation systems perform poorly on noisy inputs, which are frequent in realistic environments. Augmenting these systems with visual signals has the potential to improve robustness to noise. Howe…
Audio-Visual Speech RecognitionRepresentation LearningRobust Speech Recognitionspeech-recognition+4An Efficient End-to-End Approach to Noise Invariant Speech Features via Multi-Task Learning
Self-supervised speech representation learning enables the extraction of meaningful features from raw waveforms. These features can then be efficiently used across multiple downstream tasks. However, two significant issu…
DenoisingKnowledge DistillationMulti-Task LearningRepresentation Learning+1The Effect of Batch Size on Contrastive Self-Supervised Speech Representation Learning
Foundation models in speech are often trained using many GPUs, which implicitly leads to large effective batch sizes. In this paper we study the effect of batch size on pre-training, both in terms of statistics that can …
BenchmarkingRepresentation LearningSelf-Supervised LearningSpeech Representation LearningUNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization
Dysarthric speech reconstruction (DSR) systems aim to automatically convert dysarthric speech into normal-sounding speech. The technology eases communication with speakers affected by the neuromotor disorder and enhances…
DecoderDomain AdaptationGenerative Adversarial NetworkRepresentation Learning+1