Order matters: Distributional properties of speech to young children bootstraps learning of semantic representations
Some researchers claim that language acquisition is critically dependent on experiencing linguistic input in order of increasing complexity. We set out to test this hypothesis using a simple recurrent neural network (SRN) trained to predict word sequences in CHILDES, a 5-million-word corpus of speech directed to children. First, we demonstrated that age-ordered CHILDES exhibits a gradual increase in linguistic complexity. Next, we compared the performance of two groups of SRNs trained on CHILDES which had either been age-ordered or not. Specifically, we assessed learning of grammatical and semantic structure and showed that training on age-ordered input facilitates learning of semantic, but not of sequential structure. We found that this advantage is eliminated when the models were trained on input with utterance boundary information removed.
Code (0)
등록된 구현이 없습니다.
Tasks
Language AcquisitionSimilar Papers 제목 키워드 기반
Decoding Order Matters in Autoregressive Speech Synthesis
Autoregressive speech synthesis often adopts a left-to-right order, yet generation order is a modelling choice. We investigate decoding order through masked diffusion framework, which progressively unmasks positions and …
Speech SynthesisThe Random Conditional Distribution for Higher-Order Probabilistic Inference
The need to condition distributional properties such as expectation, variance, and entropy arises in algorithmic fairness, model simplification, robustness and many other areas. At face value however, distributional prop…
FairnessProbabilistic ProgrammingMasked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little
A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines. In this paper…
Language ModelingLanguage ModellingMasked Language ModelingAnalysis of Self-Supervised Speech Models on Children's Speech and Infant Vocalizations
To understand why self-supervised learning (SSL) models have empirically achieved strong performances on several speech-processing downstream tasks, numerous studies have focused on analyzing the encoded information of t…
Phoneme RecognitionSelf-Supervised LearningDetection of Consonant Errors in Disordered Speech Based on Consonant-vowel Segment Embedding
Speech sound disorder (SSD) refers to a type of developmental disorder in young children who encounter persistent difficulties in producing certain speech sounds at the expected age. Consonant errors are the major indica…