AutoCycle-VC: Towards Bottleneck-Independent Zero-Shot Cross-Lingual Voice Conversion
This paper proposes a simple and robust zero-shot voice conversion system with a cycle structure and mel-spectrogram pre-processing. Previous works suffer from information loss and poor synthesis quality due to their reliance on a carefully designed bottleneck structure. Moreover, models relying solely on self-reconstruction loss struggled with reproducing different speakers' voices. To address these issues, we suggested a cycle-consistency loss that considers conversion back and forth between target and source speakers. Additionally, stacked random-shuffled mel-spectrograms and a label smoothing method are utilized during speaker encoder training to extract a time-independent global speaker representation from speech, which is the key to a zero-shot conversion. Our model outperforms existing state-of-the-art results in both subjective and objective evaluations. Furthermore, it facilitates cross-lingual voice conversions and enhances the quality of synthesized speech.
Code (0)
등록된 구현이 없습니다.
Tasks
Voice ConversionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Low-Burden Data Augmentation for Dysarthric ASR via Zero-Shot Voice Cloning
Automatic speech recognition remains unreliable for dysarthric speech due to data scarcity and high inter-speaker variability. While synthetic data can address these gaps, traditional methods often require extensive spea…
Speech RecognitionData AugmentationZeroSim: Zero-Shot Analog Circuit Evaluation with Unified Transformer Embeddings
Although recent advancements in learning-based analog circuit design automation have tackled tasks such as topology generation, device sizing, and layout synthesis, efficient performance evaluation remains a major bottle…
Zero-shot GeneralizationReinforcement LearningZero-shot Concept Bottleneck Models
Concept bottleneck models (CBMs) are inherently interpretable and intervenable neural network models, which explain their final label prediction by the intermediate prediction of high-level semantic concepts. However, th…
regressionTransductive Zero-Shot and Few-Shot CLIP
Transductive inference has been widely investigated in few-shot image classification but completely overlooked in the recent fast growing literature on adapting vision-langage models like CLIP. This paper addresses t…
ClassificationFew-Shot Image Classificationimage-classificationImage ClassificationCIBR: Cross-modal Information Bottleneck Regularization for Robust CLIP Generalization
Contrastive Language-Image Pretraining (CLIP) has achieved remarkable success in cross-modal tasks such as zero-shot image classification and text-image retrieval by effectively aligning visual and textual representation…
Contrastive Learningimage-classificationImage ClassificationImage Retrieval+5