Sample-efficient Linguistic Generalizations through Program Synthesis: Experiments with Phonology Problems
Neural models excel at extracting statistical patterns from large amounts of data, but struggle to learn patterns or reason about language from only a few examples. In this paper, we ask: Can we learn explicit rules that generalize well from only a few examples? We explore this question using program synthesis. We develop a synthesis model to learn phonology rules as programs in a domain-specific language. We test the ability of our models to generalize from few training examples using our new dataset of problems from the Linguistics Olympiad, a challenging set of tasks that require strong linguistic reasoning ability. In addition to being highly sample-efficient, our approach generates human-readable programs, and allows control over the generalizability of the learnt programs.
Code (1)
Tasks
Program SynthesisSimilar Papers 제목 키워드 기반
Stress Rules from Surface Forms: Experiments with Program Synthesis
Learning linguistic generalizations from only a few examples is a challenging task. Recent work has shown that program synthesis – a method to learn rules from data in the form of programs in a domain-specific language –…
Program SynthesisHeteroscedasticity-aware residuals-based contextual stochastic optimization
We explore generalizations of some integrated learning and optimization frameworks for data-driven contextual stochastic optimization that can adapt to heteroscedasticity. We identify conditions on the stochastic program…
regressionStochastic OptimizationLanguage Embeddings Sometimes Contain Typological Generalizations
To what extent can neural network models learn generalizations about language structure, and how do we find out what they have learned? We explore these questions by training neural models for a range of natural language…
Multilingual Word EmbeddingsWord EmbeddingsLearning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)
One reason pretraining on self-supervised linguistic tasks is effective is that it teaches models features that are helpful for language understanding. However, we want pretrained models to learn not only to represent li…
Binary ClassificationDiagnosticAdvanced Large Language Model (LLM)-Driven Verilog Development: Enhancing Power, Performance, and Area Optimization in Code Synthesis
The increasing use of Advanced Language Models (ALMs) in diverse sectors, particularly due to their impressive capability to generate top-tier content following linguistic instructions, forms the core of this investigati…
Language ModelingLanguage ModellingLarge Language Model