PS4: Proxy-Supervised Joint Training for Real Target Speaker Extraction
Training target speaker extraction (TSE) models for real conversational mixtures remains challenging because large-scale training corpora and clean target speech for supervision are unavailable. We present PS4, a proxy-supervised training framework for TSE in real conversational mixtures, with two main contributions. First, we construct a large-scale corpus of 71,771 training samples derived from four public datasets, covering both Chinese and English scenarios. Each sample contains an overlapping speech mixture, per-speaker enrollment audio, a ground-truth transcript, and frame-level voice activity labels. Second, we propose a proxy-supervised joint training strategy that fine-tunes a BSRNN-based TSE model using four complementary differentiable objectives: ASR cross-entropy, speaker similarity, frame-level voice activity detection, and perceptual audio quality. Starting from a publicly available pre-trained checkpoint, only the BSRNN separator is updated during fine-tuning. On the REAL-T challenge leaderboard, PS4 ranks 2nd overall, achieving the best speaker similarity and timing F1 among all submitted systems.
Code (4)
Tasks
Activity DetectionSimilar Papers 제목 키워드 기반
HIR-ALIGN: Enhancing Hyperspectral Image Restoration via Diffusion-Based Data Generation
Hyperspectral image (HSI) restoration is crucial for reliable analysis, as real-world HSIs suffer from noise, blur, and resolution loss. However, existing models trained on source data often fail on target domains lackin…
Image RestorationJoint Supervised and Self-supervised Learning for MRI Reconstruction
Magnetic Resonance Imaging (MRI) represents an important diagnostic modality; however, its inherently slow acquisition process poses challenges in obtaining fully-sampled $k$-space data under motion. In the absence of fu…
DiagnosticMRI ReconstructionSelf-Supervised LearningRegularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation
Training a semantic segmentation model requires a large amount of pixel-level annotation, hampering its application at scale. With computer graphics, we can generate almost unlimited training data with precise annotation…
Domain AdaptationSemantic SegmentationUnsupervised Domain AdaptationAdaptive Semantic Segmentation with a Strategic Curriculum of Proxy Labels
Training deep networks for semantic segmentation requires annotation of large amounts of data, which can be time-consuming and expensive. Unfortunately, these trained networks still generalize poorly when tested in domai…
Domain AdaptationSemantic SegmentationUnsupervised Domain AdaptationLearning to Compose: Revisiting Proxy Task Design for Zero-Shot Composed Image Retrieval
Composed Image Retrieval (CIR) retrieves a target image from a reference image and a textual modification. While supervised CIR relies on costly triplets, Zero-Shot CIR (ZS-CIR) alleviates this reliance through proxy tas…
Image Retrieval