Structural Language Models of Code

2019-09-30 · ICML 2020 1 · Uri Alon, Roy Sadaka, Omer Levy, Eran Yahav

We address the problem of any-code completion - generating a missing piece of source code in a given program without any restriction on the vocabulary or structure. We introduce a new approach to any-code completion that leverages the strict syntax of programming languages to model a code snippet as a tree - structural language modeling (SLM). SLM estimates the probability of the program's abstract syntax tree (AST) by decomposing it into a product of conditional probabilities over its nodes. We present a neural model that computes these conditional probabilities by considering all AST paths leading to a target node. Unlike previous techniques that have severely restricted the kinds of expressions that can be generated in this task, our approach can generate arbitrary code in any programming language. Our model significantly outperforms both seq2seq and a variety of structured approaches in generating Java and C# code. Our code, data, and trained models are available at http://github.com/tech-srl/slm-code-generation/ . An online demo is available at http://AnyCodeGen.org .

📄 PDF Abstract BibTeX arXiv:1910.00577

Code (2)

tech-srl/slm-code-generation 공식 구현 tf

tihonovcore/model tf

Tasks

C++ codeCode Completion Code GenerationLanguage ModelingLanguage Modelling

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…

Discrete Structural Planning for Generating Diverse Translations

2018-09-27 · Raphael Shu, Hideki Nakayama

Planning is important for humans when producing complex languages, which is a missing part in current language generation models. In this work, we add a planning phase in neural machine translation to control the global …

Diversity Machine Translation Sentence Text Generation+1

Structural Language Models for Any-Code Generation

2019-09-25 · Uri Alon, Roy Sadaka, Omer Levy, Eran Yahav

We address the problem of Any-Code Generation (AnyGen) - generating code without any restriction on the vocabulary or structure. The state-of-the-art in this problem is the sequence-to-sequence (seq2seq) approach, which …

C++ code Code Generation Language Modeling Language Modelling

GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding

2024-09-06 · Ziyin Zhang, Hang Yu, Shijie Li, Peng Di 외

Programming languages possess rich semantic information such as data flow that is represented by graphs and not available from the surface form of source code. Recent code language models have scaled to billions of param…

cross-modal alignment Language Modelling Large Language Model

Source Code Summarization with Structural Relative Position Guided Transformer

2022-02-14 · Zi Gong, Cuiyun Gao, Yasheng Wang, Wenchao Gu 외

Source code summarization aims at generating concise and clear natural language descriptions for programming languages. Well-written code summaries are beneficial for programmers to participate in the software developmen…

Code Summarization Position Source Code Summarization

MLCPD: A Unified Multi-Language Code Parsing Dataset with Universal AST Schema

2025-10-18 · Jugal Gajjar, Kamalasankari Subramaniakuppusamy arxiv

We introduce the MultiLang Code Parser Dataset (MLCPD), a large-scale, language-agnostic dataset unifying syntactic and structural representations of code across ten major programming languages. MLCPD contains over seven…

Representation Learning