Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages
We show that unsupervised sequence-segmentation performance can be transferred to extremely low-resource languages by pre-training a Masked Segmental Language Model (Downey et al., 2021) multilingually. Further, we show that this transfer can be achieved by training over a collection of low-resource languages that are typologically similar (but phylogenetically unrelated) to the target language. In our experiments, we transfer from a collection of 10 Indigenous American languages (AmericasNLP, Mager et al., 2021) to K'iche', a Mayan language. We compare our multilingual model to a monolingual (from-scratch) baseline, as well as a model pre-trained on Quechua only. We show that the multilingual pre-trained approach yields consistent segmentation quality across target dataset sizes, exceeding the monolingual baseline in 6/10 experimental settings. Our model yields especially strong results at small target sizes, including a zero-shot performance of 20.6 F1. These results have promising implications for low-resource NLP pipelines involving human-like linguistic units, such as the sparse transcription framework proposed by Bird (2020).
Code (1)
Tasks
Language ModelingLanguage ModellingSegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages
We show that unsupervised sequence-segmentation performance can be transferred to extremely low-resource languages by pre-training a Masked Segmental Language Model (Downey et al., 2021) multilingually. Further, we show …
Language ModelingLanguage ModellingMorfessor-enriched features and multilingual training for canonical morphological segmentation
In our submission to the SIGMORPHON 2022 Shared Task on Morpheme Segmentation, we study whether an unsupervised morphological segmentation method, Morfessor, can help in a supervised setting. Previous research has shown …
Morpheme SegmentaitonSentenceBilingual alignment transfers to multilingual alignment for unsupervised parallel text mining
This work presents methods for learning cross-lingual sentence representations using paired or unpaired bilingual texts. We hypothesize that the cross-lingual alignment strategy is transferable, and therefore a model tra…
Language ModelingLanguage ModellingRetrievalSentence+1TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clustering
Temporal action segmentation in untrimmed videos has gained increased attention recently. However, annotating action classes and frame-wise boundaries is extremely time consuming and cost intensive, especially on large-s…
Action SegmentationClusteringSegmentationTemporal Action Segmentation+1MultiLinguahah : A New Unsupervised Multilingual Acoustic Laughter Segmentation Method
Laughter is a social non-vocalization that is universal across cultures and languages, and is crucial for human communication, including social bonding and communication signaling. However, detecting laughter in audio is…
Anomaly Detection