Towards Neural Functional Program Evaluation
This paper explores the capabilities of current transformer-based language models for program evaluation of simple functional programming languages. We introduce a new program generation mechanism that allows control over syntactic sugar for semantically equivalent programs. T5 experiments reveal that neural functional program evaluation performs surprisingly well, achieving high 90% exact program match scores for most in-distribution and out-of-distribution tests. Using pretrained T5 weights has significant advantages over random initialization. We present and evaluate on three datasets to study generalization abilities that are specific to functional programs based on: type, function composition, and reduction steps. Code and data are publicly available at https://github.com/ElementAI/neural-interpreters.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Perish or Flourish? A Holistic Evaluation of Large Language Models for Code Generation in Functional Programming
Functional programming provides strong foundations for developing reliable and secure software systems, yet its adoption remains not widespread due to the steep learning curve. Recent advances in Large Language Models (L…
Code GenerationA Survey on Natural Language Processing for Programming
Natural language processing for programming aims to use NLP techniques to assist programming. It is increasingly prevalent for its effectiveness in improving productivity. Distinct from natural language, a programming la…
SurveyInvestigating the Performance of Language Models for Completing Code in Functional Programming Languages: a Haskell Case Study
Language model-based code completion models have quickly grown in use, helping thousands of developers write code in many different programming languages. However, research on code completion models typically focuses on …
Code CompletionHumanEvalLanguage ModellingIncorporating Social Welfare in Program-Evaluation and Treatment Choice
The econometric literature on treatment-effects typically takes functionals of outcome-distributions as `social welfare' and ignores program-impacts on unobserved utilities. We show how to incorporate aggregate utility w…
AVATAR: A Parallel Corpus for Java-Python Program Translation
Program translation refers to migrating source code from one programming language to another. It has tremendous practical value in software development, as porting software across languages is time-consuming and costly. …
Translation