Unsupervised Text Segmentation Based on Native Language Characteristics
Most work on segmenting text does so on the basis of topic changes, but it can be of interest to segment by other, stylistically expressed characteristics such as change of authorship or native language. We propose a Bayesian unsupervised text segmentation approach to the latter. While baseline models achieve essentially random segmentation on our task, indicating its difficulty, a Bayesian model that incorporates appropriately compact language models and alternating asymmetric priors can achieve scores on the standard metrics around halfway to perfect segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
SegmentationText SegmentationSimilar Papers 제목 키워드 기반
Unsupervised Neural Word Segmentation for Chinese via Segmental Language Modeling
Previous traditional approaches to unsupervised Chinese word segmentation (CWS) can be roughly classified into discriminative and generative models. The former uses the carefully designed goodness measures for candidate …
Chinese Word SegmentationDecoderLanguage ModelingLanguage Modelling+1Building Morphological Chains for Agglutinative Languages
In this paper, we build morphological chains for agglutinative languages by using a log-linear model for the morphological segmentation task. The model is based on the unsupervised morphological segmentation system calle…
SegmentationSubword Segmental Language Modelling for Nguni Languages
Subwords have become the standard units of text in NLP, enabling efficient open-vocabulary models. With algorithms like byte-pair encoding (BPE), subword segmentation is viewed as a preprocessing step applied to the corp…
Language ModelingLanguage ModellingSegmentationUnsupervised anomaly segmentation via deep feature reconstruction
Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect vi…
Anomaly DetectionAnomaly SegmentationDescriptiveDFR: Deep Feature Reconstruction for Unsupervised Anomaly Segmentation
Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect vi…
Anomaly DetectionAnomaly SegmentationDescriptive