Adapting the NICT-JLE Corpus for Disfluency Detection Models
The detection of disfluencies such as hesitations, repetitions and false starts commonly found in speech is a widely studied area of research. With a standardised process for evaluation using the Switchboard Corpus, model performance can be easily compared across approaches. This is not the case for disfluency detection research on learner speech, however, where such datasets have restricted access policies, making comparison and subsequent development of improved models more challenging. To address this issue, this paper describes the adaptation of the NICT-JLE corpus, containing approximately 300 hours of English learners' oral proficiency tests, to a format that is suitable for disfluency detection model training and evaluation. Points of difference between the NICT-JLE and Switchboard corpora are explored, followed by a detailed overview of adaptations to the tag set and meta-features of the NICT-JLE corpus. The result of this work provides a standardised train, heldout and test set for use in future research on disfluency detection for learner speech.
Code (0)
등록된 구현이 없습니다.
Tasks
TAGSimilar Papers 제목 키워드 기반
Planning and Generating Natural and Diverse Disfluent Texts as Augmentation for Disfluency Detection
Existing approaches to disfluency detection heavily depend on human-annotated data. Numbers of data augmentation methods have been proposed to alleviate the dependence on labeled data. However, current augmentation appro…
Data AugmentationDisfluency Detection using a Bidirectional LSTM
We introduce a new approach for disfluency detection using a Bidirectional Long-Short Term Memory neural network (BLSTM). In addition to the word sequence, the model takes as input pattern match features that were develo…
Streaming Joint Speech Recognition and Disfluency Detection
Disfluency detection has mainly been solved in a pipeline approach, as post-processing of speech recognition. In this study, we propose Transformer-based encoder-decoder models that jointly solve speech recognition and d…
DecoderLanguage Modellingspeech-recognitionSpeech RecognitionDisfluencies and Human Speech Transcription Errors
This paper explores contexts associated with errors in transcrip-tion of spontaneous speech, shedding light on human perceptionof disfluencies and other conversational speech phenomena. Anew version of the Switchboard co…
NICT's Corpus Filtering Systems for the WMT18 Parallel Corpus Filtering Task
This paper presents the NICT's participation in the WMT18 shared parallel corpus filtering task. The organizers provided 1 billion words German-English corpus crawled from the web as part of the Paracrawl project. This c…
Machine TranslationNMTSentenceTranslation