Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation

2022-05-18 · Qianqian Dong, Fengpeng Yue, Tom Ko, Mingxuan Wang, Qibing Bai, Yu Zhang

Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently. The task is very challenging due to data scarcity and complex speech-to-speech mapping. In this paper, we report our recent achievements in S2ST. Firstly, we build a S2ST Transformer baseline which outperforms the original Translatotron. Secondly, we utilize the external data by pseudo-labeling and obtain a new state-of-the-art result on the Fisher English-to-Spanish test set. Indeed, we exploit the pseudo data with a combination of popular techniques which are not trivial when applied to S2ST. Moreover, we evaluate our approach on both syntactically similar (Spanish-English) and distant (English-Chinese) language pairs. Our implementation is available at https://github.com/fengpeng-yue/speech-to-speech-translation.

📄 PDF Abstract BibTeX arXiv:2205.08993

Code (1)

fengpeng-yue/speech-to-speech-translation 공식 구현 pytorch

Tasks

Speech-to-Speech Translation Translation

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Position-Wise Feed-Forward Layer 설명 없음

Adam 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Residual Connection 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

PseudoProp: Robust Pseudo-Label Generation for Semi-Supervised Object Detection in Autonomous Driving Systems

2022-03-11 · Shu Hu, Chun-Hao Liu, Jayanta Dutta, Ming-Ching Chang 외

Semi-supervised object detection methods are widely used in autonomous driving systems, where only a fraction of objects are labeled. To propagate information from the labeled objects to the unlabeled ones, pseudo-labels…

Autonomous Driving Object object-detection Object Detection+2

TrajSSL: Trajectory-Enhanced Semi-Supervised 3D Object Detection

2024-09-17 · Philip Jacobson, Yichen Xie, Mingyu Ding, Chenfeng Xu 외

Semi-supervised 3D object detection is a common strategy employed to circumvent the challenge of manually labeling large-scale autonomous driving perception datasets. Pseudo-labeling approaches to semi-supervised learnin…

3D Object Detection Autonomous Driving Motion Forecasting Object+3

Image to Pseudo-Episode: Boosting Few-Shot Segmentation by Unlabeled Data

2024-05-14 · Jie Zhang, Yuhan Li, Yude Wang, Stephen Lin 외

Few-shot segmentation (FSS) aims to train a model which can segment the object from novel classes with a few labeled samples. The insufficient generalization ability of models leads to unsatisfactory performance when the…

Data Augmentation Pseudo Label

Leveraging Temporal Cues for Semi-Supervised Multi-View 3D Object Detection

2025-01-01 · CVPR 2025 1 · Jinhyung Park, Navyata Sanghvi, Hiroki Adachi, Yoshihisa Shibata 외

While recent advancements in camera-based 3D object detection demonstrate remarkable performance, they require thousands or even millions of human-annotated frames. This requirement significantly inhibits their deplo…

3D Object Detection 3D Object Tracking Autonomous Driving object-detection+3

Context Matters: Leveraging Spatiotemporal Metadata for Semi-Supervised Learning on Remote Sensing Images

2024-04-29 · Maximilian Bernhard, Tanveer Hannan, Niklas Strauß, Matthias Schubert

Remote sensing projects typically generate large amounts of imagery that can be used to train powerful deep neural networks. However, the amount of labeled images is often small, as remote sensing applications generally …

Pseudo Label Transfer Learning