Code Completion using Neural Attention and Byte Pair Encoding

2020-04-14 · Youri Arkesteijn, Nikhil Saldanha, Bastijn Kostense

In this paper, we aim to do code completion based on implementing a Neural Network from Li et. al.. Our contribution is that we use an encoding that is in-between character and word encoding called Byte Pair Encoding (BPE). We use this on the source code files treating them as natural text without first going through the abstract syntax tree (AST). We have implemented two models: an attention-enhanced LSTM and a pointer network, where the pointer network was originally introduced to solve out of vocabulary problems. We are interested to see if BPE can replace the need for the pointer network for code completion.

📄 PDF Abstract BibTeX arXiv:2004.06343

Code (0)

등록된 구현이 없습니다.

Tasks

Code Completion

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

[LivE@PeRson]How do I talk to a real person at Expedia? How do I talk to a real person at Expedia? To speak with a live Expedia agent, call 1 (888) 829-0881 anytime. The customer support team is available 24/7 to help with…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Pointer Network Pointer Networks tackle problems where input and output data are sequential data, but can't be solved by seq2seq type models because discrete categories of output elements…

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

TINY_SCHILLER: A Drop-In German Drama Corpus for Small Language Models

2026-07-22 · Mark Schutera arxiv

tiny_schiller closes the small-language-model prototyping, fine-tuning, education, and research gap for German literary text, providing a single-file, drop-in counterpart to Karpathy's tiny_shakespeare. The available Ger…

Fast and Memory-Efficient Neural Code Completion

2020-04-28 · Alexey Svyatkovskiy, Sebastian Lee, Anna Hadjitofi, Maik Riechert 외

Code completion is one of the most widely used features of modern integrated development environments (IDEs). While deep learning has made significant progress in the statistical prediction of source code, state-of-the-a…

Code Completion Language Modelling Learning-To-Rank Reranking

Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding

2019-11-01 · CONLL 2019 11 · Chaodong Tong, Huailiang Peng, Qiong Dai, Lei Jiang 외

We propose a method called reverse mapping bytepair encoding, which maps named-entity information and other word-level linguistic features back to subwords during the encoding procedure of bytepair encoding (BPE). We emp…

Natural Language Understanding RTE SST-2

Composing Byte-Pair Encodings for Morphological Sequence Classification

2020-12-01 · UDW (COLING) 2020 12 · Adam Ek, Jean-Philippe Bernardy

Byte-pair encodings is a method for splitting a word into sub-word tokens, a language model then assigns contextual representations separately to each of these tokens. In this paper, we evaluate four different methods of…

Classification Language Modeling Language Modelling

HoloByte: Continuous Hyperspherical Distillation for Tokenizer-Free Modeling

2026-03-10 · Vladimer Khasia arxiv

Sequence modeling universally relies on discrete subword tokenization to circumvent the $\mathcal{O}(N^2)$ computational intractability of native byte-level attention. However, this heuristic quantization imposes artific…