Memory limitations are hidden in grammar
The ability to produce and understand an unlimited number of different sentences is a hallmark of human language. Linguists have sought to define the essence of this generative capacity using formal grammars that describe the syntactic dependencies between constituents, independent of the computational limitations of the human brain. Here, we evaluate this independence assumption by sampling sentences uniformly from the space of possible syntactic structures. We find that the average dependency distance between syntactically related words, a proxy for memory limitations, is less than expected by chance in a collection of state-of-the-art classes of dependency grammars. Our findings indicate that memory limitations have permeated grammatical descriptions, suggesting that it may be impossible to build a parsimonious theory of human linguistic productivity independent of non-linguistic cognitive constraints.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecture
We introduce a general method to extract knowledge from a recurrent neural network (Long Short Term Memory) that has learnt to detect if a given input sequence is valid or not, according to an unknown generative automato…
ClusteringvalidMemory-Bounded Left-Corner Unsupervised Grammar Induction on Child-Directed Input
This paper presents a new memory-bounded left-corner parsing model for unsupervised raw-text syntax induction, using unsupervised hierarchical hidden Markov models (UHHMM). We deploy this algorithm to shed light on the e…
Language AcquisitionSentenceLow-Rank Constraints for Fast Inference in Structured Models
Structured distributions, i.e. distributions over combinatorial spaces, are commonly used to learn latent probabilistic representations from observed data. However, scaling these models is bottlenecked by the high comput…
Language ModelingLanguage ModellingMusic ModelingA weakly supervised sequence tagging and grammar induction approach to semantic frame slot filling
This paper describes continuing work on semantic frame slot filling for a command and control task using a weakly-supervised approach. We investigate the advantages of using retraining techniques that take the output of …
slot-fillingSlot FillingEffects of limited and heterogeneous memory in hidden-action situations
Limited memory of decision-makers is often neglected in economic models, although it is reasonable to assume that it significantly influences the models' outcomes. The hidden-action model introduced by Holmstr\"om also i…