Do RNN States Encode Abstract Phonological Processes?
Sequence-to-sequence models have delivered impressive results in word formation tasks such as morphological inflection, often learning to model subtle morphophonological details with limited training data. Despite the performance, the opacity of neural models makes it difficult to determine whether complex generalizations are learned, or whether a kind of separate rote memorization of each morphophonological process takes place. To investigate whether complex alternations are simply memorized or whether there is some level of generalization across related sound changes in a sequence-to-sequence model, we perform several experiments on Finnish consonant gradation -- a complex set of sound changes triggered in some words by certain suffixes. We find that our models often -- though not always -- encode 17 different consonant gradation processes in a handful of dimensions in the RNN. We also show that by scaling the activations in these dimensions we can control whether consonant gradation occurs and the direction of the gradation.
Code (0)
등록된 구현이 없습니다.
Tasks
MemorizationMorphological InflectionSimilar Papers 제목 키워드 기반
Do RNN States Encode Abstract Phonological Alternations?
Sequence-to-sequence models have delivered impressive results in word formation tasks such as morphological inflection, often learning to model subtle morphophonological details with limited training data. Despite the pe…
MemorizationMorphological InflectionMarkov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes
Our work aims at developing reinforcement learning algorithms that do not rely on the Markov assumption. We consider the class of Non-Markov Decision Processes where histories can be abstracted into a finite set of state…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English
Self-supervised and supervised speech models are increasingly used to investigate which linguistic information their internal representations encode, and at what level of abstraction they encode it. One underexplored phe…
Speech RecognitionConstraint-based Learning of Phonological Processes
Phonological processes are context-dependent sound changes in natural languages. We present an unsupervised approach to learning human-readable descriptions of phonological processes from collections of related utterance…
Program SynthesisEncoder-decoder models for latent phonological representations of words
We use sequence-to-sequence networks trained on sequential phonetic encoding tasks to construct compositional phonological representations of words. We show that the output of an encoder network can predict the phonetic …
Decoder