Surface Statistics of an Unknown Language Indicate How to Parse It
We introduce a novel framework for delexicalized dependency parsing in a new language. We show that useful features of the target language can be extracted automatically from an unparsed corpus, which consists only of gold part-of-speech (POS) sequences. Providing these features to our neural parser enables it to parse sequences like those in the corpus. Strikingly, our system has no supervision in the target language. Rather, it is a multilingual system that is trained end-to-end on a variety of other languages, so it learns a feature extractor that works well. We show experimentally across multiple languages: (1) Features computed from the unparsed corpus improve parsing accuracy. (2) Including thousands of synthetic languages in the training yields further improvement. (3) Despite being computed from unparsed corpora, our learned task-specific features beat previous work{'}s interpretable typological features that require parsed corpora or expert categorization of the language. Our best method improved attachment scores on held-out test languages by an average of 5.6 percentage points over past work that does not inspect the unparsed data (McDonald et al., 2011), and by 20.7 points over past {``}grammar induction{''} work that does not use training languages (Naseem et al., 2010).
Code (0)
등록된 구현이 없습니다.
Tasks
Dependency ParsingPOSSimilar Papers 제목 키워드 기반
Deep networks learn to parse uniform-depth context-free languages from local statistics
Understanding how the structure of language can be learned from sentences alone is a central question in both cognitive science and machine learning. Studies of the internal representations of Large Language Models (LLMs…
Efficient Truncated Statistics with Unknown Truncation
We study the problem of estimating the parameters of a Gaussian distribution when samples are only shown if they fall in some (unknown) subset $S \subseteq \R^d$. This core problem in truncated statistics has long histor…
Synthetic Data Made to Order: The Case of Parsing
To approximately parse an unfamiliar language, it helps to have a treebank of a similar language. But what if the closest available treebank still has the wrong word order? We show how to (stochastically) permute the con…
Cross-Lingual TransferDependency ParsingLanguage ModelingLanguage Modelling+1Signal and Noise Statistics Oblivious Sparse Reconstruction using OMP/OLS
Orthogonal matching pursuit (OMP) and orthogonal least squares (OLS) are widely used for sparse signal reconstruction in under-determined linear regression problems. The performance of these compressed sensing (CS) algor…
compressed sensingNoise Statistics Oblivious GARD For Robust Regression With Sparse Outliers
Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are…
regression