Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection
Most existing approaches to disfluency detection heavily rely on human-annotated corpora, which is expensive to obtain in practice. There have been several proposals to alleviate this issue with, for instance, self-supervised learning techniques, but they still require human-annotated corpora. In this work, we explore the unsupervised learning paradigm which can potentially work with unlabeled text corpora that are cheaper and easier to obtain. Our model builds upon the recent work on Noisy Student Training, a semi-supervised learning approach that extends the idea of self-training. Experimental results on the commonly used English Switchboard test set show that our approach achieves competitive performance compared to the previous state-of-the-art supervised systems using contextualized word embeddings (e.g. BERT and ELECTRA).
Code (1)
Tasks
Self-Supervised LearningWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MSDA: Combining Pseudo-labeling and Self-Supervision for Unsupervised Domain Adaptation in ASR
In this work, we investigate the Meta PL unsupervised domain adaptation framework for Automatic Speech Recognition (ASR). We introduce a Multi-Stage Domain Adaptation pipeline (MSDA), a sample-efficient, two-stage adapta…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Domain AdaptationSelf-Supervised Learning+3Geometric Unsupervised Domain Adaptation for Semantic Segmentation
Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation. However, they introduce a domain gap that severely hurts real-world…
Depth EstimationDepth PredictionDomain AdaptationMonocular Depth Estimation+3ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation
Understanding the 3D world without supervision is currently a major challenge in computer vision as the annotations required to supervise deep networks for tasks in this domain are expensive to obtain on a large scale. I…
Image ReconstructionSelf-Supervised LearningViewpoint EstimationSelf supervised contrastive learning for digital histopathology
Unsupervised learning has been a long-standing goal of machine learning and is especially important for medical image analysis, where the learning can compensate for the scarcity of labeled datasets. A promising subclass…
Contrastive LearningMedical Image AnalysisSelf-Supervised LearningSelfAct: Personalized Activity Recognition based on Self-Supervised and Active Learning
Supervised Deep Learning (DL) models are currently the leading approach for sensor-based Human Activity Recognition (HAR) on wearable and mobile devices. However, training them requires large amounts of labeled data whos…
Active LearningActivity RecognitionHuman Activity Recognition