Incremental Learning for Fully Unsupervised Word Segmentation Using Penalized Likelihood and Model Selection
We present a novel incremental learning approach for unsupervised word segmentation that combines features from probabilistic modeling and model selection. This includes super-additive penalties for addressing the cognitive burden imposed by long word formation, and new model selection criteria based on higher-order generative assumptions. Our approach is fully unsupervised; it relies on a small number of parameters that permits flexible modeling and a mechanism that automatically learns parameters from the data. Through experimentation, we show that this intricate design has led to top-tier performance in both phonemic and orthographic word segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Incremental LearningModel SelectionSegmentationSimilar Papers 제목 키워드 기반
Unsupervised Neural Word Segmentation for Chinese via Segmental Language Modeling
Previous traditional approaches to unsupervised Chinese word segmentation (CWS) can be roughly classified into discriminative and generative models. The former uses the carefully designed goodness measures for candidate …
Chinese Word SegmentationDecoderLanguage ModelingLanguage Modelling+1Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networks
We investigate segmenting and clustering speech into low-bitrate phone-like sequences without supervision. We specifically constrain pretrained self-supervised vector-quantized (VQ) neural networks so that blocks of cont…
ClusteringSegmentationWord Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring
Recent work on unsupervised speech segmentation has used self-supervised models with phone and word segmentation modules that are trained jointly. This paper instead revisits an older approach to word segmentation: botto…
Acoustic Unit DiscoverySegmentationLearning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs
Recent research has discovered that a shared bilingual word embedding space can be induced by projecting monolingual word embedding spaces from two languages using a self-learning paradigm without any bilingual supervisi…
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsDependency ParsingDocument Classification+3Optimizing Segmentation Granularity for Neural Machine Translation
In neural machine translation (NMT), it is has become standard to translate using subword units to allow for an open vocabulary and improve accuracy on infrequent words. Byte-pair encoding (BPE) and its variants are the …
Machine TranslationNMTTranslation