Structural Language Models for Any-Code Generation
We address the problem of Any-Code Generation (AnyGen) - generating code without any restriction on the vocabulary or structure. The state-of-the-art in this problem is the sequence-to-sequence (seq2seq) approach, which treats code as a sequence and does not leverage any structural information. We introduce a new approach to AnyGen that leverages the strict syntax of programming languages to model a code snippet as tree structural language modeling (SLM). SLM estimates the probability of the program's abstract syntax tree (AST) by decomposing it into a product of conditional probabilities over its nodes. We present a neural model that computes these conditional probabilities by considering all AST paths leading to a target node. Unlike previous structural techniques that have severely restricted the kinds of expressions that can be generated, our approach can generate arbitrary expressions in any programming language. Our model significantly outperforms both seq2seq and a variety of existing structured approaches in generating Java and C# code. We make our code, datasets, and models available online.
Code (0)
등록된 구현이 없습니다.
Tasks
C++ codeCode GenerationLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Repo0: Design-Driven Zero-to-All Code Generation
Large language model agents have made substantial progress in code generation, yet most existing systems assume a predefined repository architecture. This assumption does not hold in zero-to-all code generation, where an…
Code GenerationMeasuring LLM Code Generation Stability via Structural Entropy
Assessing the stability of code generation from large language models (LLMs) is essential for judging their reliability in real-world development. We extend prior "structural-entropy concepts" to the program domain by pa…
Code GenerationDiscrete Structural Planning for Generating Diverse Translations
Planning is important for humans when producing complex languages, which is a missing part in current language generation models. In this work, we add a planning phase in neural machine translation to control the global …
DiversityMachine TranslationSentenceText Generation+1VeriGRAG: Enhancing LLM-Based Verilog Code Generation with Structure-Aware Soft Prompts
Large language models (LLMs) have demonstrated strong capabilities in generating Verilog code from natural language descriptions. However, Verilog code inherently encodes structural information of hardware circuits. Effe…
Code GenerationTokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models
Understanding how Large Language Models (LLMs) make token-level decisions during code generation remains a major challenge for both researchers and practitioners. While recent tools provide insights into model internals …
Code Generation