paper-with-me

Papers

Towards Neural Functional Program Evaluation

2021-12-09 · NeurIPS Workshop AIPLANS 2021 12 · Torsten Scholak, Jonathan Pilault, Joey Velez-Ginorio

This paper explores the capabilities of current transformer-based language models for program evaluation of simple functional programming languages. We introduce a new program generation mechanism that allows control over syntactic sugar for semantically equivalent programs. T5 experiments reveal that neural functional program evaluation performs surprisingly well, achieving high 90% exact program match scores for most in-distribution and out-of-distribution tests. Using pretrained T5 weights has significant advantages over random initialization. We present and evaluate on three datasets to study generalization abilities that are specific to functional programs based on: type, function composition, and reduction steps. Code and data are publicly available at https://github.com/ElementAI/neural-interpreters.

📄 PDF Abstract BibTeX arXiv:2112.04630

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Gated Linear Unit A Gated Linear Unit, or GLU computes: $$ \mathrm{GLU}(a, b) = a \otimes \sigma(b) $$ It is used in natural language processing architectures, for example the Gated CNN,…
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Inverse Square Root Schedule Inverse Square Root is a learning rate schedule 1 / $\sqrt{\max\left(n, k\right)}$ where $n$ is the current training iteration and $k$ is the number of warm-up steps. This…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Similar Papers 제목 키워드 기반

Perish or Flourish? A Holistic Evaluation of Large Language Models for Code Generation in Functional Programming

2026-01-05 · Nguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le 외 arxiv

Functional programming provides strong foundations for developing reliable and secure software systems, yet its adoption remains not widespread due to the steep learning curve. Recent advances in Large Language Models (L…

Code Generation

A Survey on Natural Language Processing for Programming

2022-12-12 · Qingfu Zhu, Xianzhen Luo, Fang Liu, Cuiyun Gao 외

Natural language processing for programming aims to use NLP techniques to assist programming. It is increasingly prevalent for its effectiveness in improving productivity. Distinct from natural language, a programming la…

Survey

Investigating the Performance of Language Models for Completing Code in Functional Programming Languages: a Haskell Case Study

2024-03-22 · Tim van Dam, Frank van der Heijden, Philippe de Bekker, Berend Nieuwschepen 외

Language model-based code completion models have quickly grown in use, helping thousands of developers write code in many different programming languages. However, research on code completion models typically focuses on …

Code CompletionHumanEvalLanguage Modelling

Incorporating Social Welfare in Program-Evaluation and Treatment Choice

2021-05-18 · Debopam Bhattacharya, Tatiana Komarova

The econometric literature on treatment-effects typically takes functionals of outcome-distributions as `social welfare' and ignores program-impacts on unobserved utilities. We show how to incorporate aggregate utility w…

AVATAR: A Parallel Corpus for Java-Python Program Translation

2021-08-26 · Wasi Uddin Ahmad, Md Golam Rahman Tushar, Saikat Chakraborty, Kai-Wei Chang

Program translation refers to migrating source code from one programming language to another. It has tremendous practical value in software development, as porting software across languages is time-consuming and costly. …

Translation