Transformer-based Program Synthesis for Low-Data Environments
Recent advancements in large pre-trained transformer models (GPT2/3, T5) have found use in program synthesis to generate programs that satisfy a set of input/output examples. However, these models perform poorly on long-horizon and low-data tasks, and often don't seem to understand the semantics of the languages they generate. We investigate an approach that tackles both of these issues, by using attributed context-free-grammars of programming languages to generate programs, and then analyzing generated programs so that they can be annotated with compile and runtime attributes, such as types, so that information about the program can be remembered during long-horizon generation. We firstly find that synthesized datasets can be made efficiently and can provide transformer models with enough data in order to perform well on some synthesis tasks. We also find that giving models access to program attributes is especially effective in low-data environments, and tends improve the quality and reduce errors of transformer-generated programs.
Code (1)
Tasks
Program SynthesisSimilar Papers 제목 키워드 기반
AutoCoder: Leveraging Transformers for Automatic Code Synthesis
Program synthesis from natural language descriptions is a challenging task. This paper explores two variants of transformer models for the task of program synthesis and showcase higher performance than the existing SOTA …
Program SynthesisAdversarial Robustness of Program Synthesis Models
The resurgence of automatic program synthesis has been observed with the rise of deep learning. In this paper, we study the behaviour of the program synthesis model under adversarial settings. Our experiments suggest tha…
Adversarial RobustnessProgram SynthesisAre Transformers All That Karel Needs?
Recent works have shown the incredible promise of using neural networks for the task of program synthesis from input-output examples. In this paper, we propose using Transformer-based architectures as a baseline for the …
AllDecoderProgram SynthesisLTLf Synthesis on First-Order Agent Programs in Nondeterministic Environments
We investigate the synthesis of policies for high-level agent programs expressed in Golog, a language based on situation calculus that incorporates nondeterministic programming constructs. Unlike traditional approaches f…
Program Synthesis Guided Reinforcement Learning for Partially Observed Environments
A key challenge for reinforcement learning is solving long-horizon planning problems. Recent work has leveraged programs to guide reinforcement learning in these settings. However, these approaches impose a high manual b…
MinecraftProgram Synthesisreinforcement-learningReinforcement Learning+1