Segmental Recurrent Neural Networks
We introduce segmental recurrent neural networks (SRNNs) which define, given an input sequence, a joint probability distribution over segmentations of the input and labelings of the segments. Representations of the input segments (i.e., contiguous subsequences of the input) are computed by encoding their constituent tokens using bidirectional recurrent neural nets, and these "segment embeddings" are used to define compatibility scores with output labels. These local compatibility scores are integrated using a global semi-Markov conditional random field. Both fully supervised training -- in which segment boundaries and labels are observed -- as well as partially supervised training -- in which segment boundaries are latent -- are straightforward. Experiments on handwriting recognition and joint Chinese word segmentation/POS tagging show that, compared to models that do not explicitly represent segments such as BIO tagging schemes and connectionist temporal classification (CTC), SRNNs obtain substantially higher accuracies.
Code (2)
Tasks
Chinese Word SegmentationHandwriting RecognitionPOSPOS TaggingSimilar Papers 제목 키워드 기반
Segmental Recurrent Neural Networks for End-to-end Speech Recognition
We study the segmental recurrent neural network for end-to-end acoustic modelling. This model connects the segmental conditional random field (CRF) with a recurrent neural network (RNN) used for feature extraction. Compa…
Acoustic ModellingLanguage ModelingLanguage Modellingspeech-recognition+1End-to-End Neural Segmental Models for Speech Recognition
Segmental models are an alternative to frame-based models for sequence prediction, where hypothesized path weights are based on entire segment scores rather than a single frame at a time. Neural segmental models are segm…
Decoderspeech-recognitionSpeech RecognitionImproving Coverage of an Inuktitut Morphological Analyzer Using a Segmental Recurrent Neural Network
A Masked Segmental Language Model for Unsupervised Natural Language Segmentation
We introduce a Masked Segmental Language Model (MSLM) for joint language modeling and unsupervised segmentation. While near-perfect supervised methods have been developed for segmenting human-like linguistic units in res…
Language ModelingLanguage ModellingSegmentationA Masked Segmental Language Model for Unsupervised Natural Language Segmentation
Segmentation remains an important preprocessing step both in languages where "words" or other important syntactic/semantic units (like morphemes) are not clearly delineated by white space, as well as when dealing with co…
Language ModelingLanguage ModellingSegmentation