On Unsupervised Training of Link Grammar Based Language Models
In this short note we explore what is needed for the unsupervised training of graph language models based on link grammars. First, we introduce the ter-mination tags formalism required to build a language model based on a link grammar formalism of Sleator and Temperley [21] and discuss the influence of context on the unsupervised learning of link grammars. Second, we pro-pose a statistical link grammar formalism, allowing for statistical language generation. Third, based on the above formalism, we show that the classical dissertation of Yuret [25] on discovery of linguistic relations using lexical at-traction ignores contextual properties of the language, and thus the approach to unsupervised language learning relying just on bigrams is flawed. This correlates well with the unimpressive results in unsupervised training of graph language models based on bigram approach of Yuret.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingText GenerationSimilar Papers 제목 키워드 기반
Learning Language from a Large (Unannotated) Corpus
A novel approach to the fully automated, unsupervised extraction of dependency grammars and associated syntax-to-semantic-relationship mappings from large text corpora is described. The suggested approach builds on the a…
Minimally-Supervised Morphological Segmentation using Adaptor Grammars
This paper explores the use of Adaptor Grammars, a nonparametric Bayesian modelling framework, for minimally supervised morphological segmentation. We compare three training methods: unsupervised training, semi-supervise…
Machine TranslationModel SelectionMORPHSpeech Recognition+1Unsupervised Neural Machine Translation with Universal Grammar
Machine translation usually relies on parallel corpora to provide parallel signals for training. The advent of unsupervised machine translation has brought machine translation away from this reliance, though performance …
Machine TranslationTranslationUnsupervised Machine TranslationGuiding Symbolic Natural Language Grammar Induction via Transformer-Based Sequence Probabilities
A novel approach to automated learning of syntactic rules governing natural languages is proposed, based on using probabilities assigned to sentences (and potentially longer word sequences) by transformer neural network …
ClusteringArithmetics-Based Decomposition of Numeral Words -- Arithmetic Conditions give the Unpacking Strategy
This paper presents a novel numeral decomposer based on arithmetic criteria. The criteria are not dependent on a base-10 assumption but only on Hurford's Packing Strategy. Hurford's Packing Strategy constitutes numerals …