On the Embeddings of Variables in Recurrent Neural Networks for Source Code
Source code processing heavily relies on the methods widely used in natural language processing (NLP), but involves specifics that need to be taken into account to achieve higher quality. An example of this specificity is that the semantics of a variable is defined not only by its name but also by the contexts in which the variable occurs. In this work, we develop dynamic embeddings, a recurrent mechanism that adjusts the learned semantics of the variable when it obtains more information about the variable's role in the program. We show that using the proposed dynamic embeddings significantly improves the performance of the recurrent neural network, in code completion and bug fixing tasks.
Code (1)
Tasks
Bug fixingCode CompletionSpecificitySimilar Papers 제목 키워드 기반
Z-Forcing: Training Stochastic Recurrent Networks
Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variab…
Language ModelingLanguage ModellingVariational InferenceLearning Semantic Vector Representations of Source Code via a Siamese Neural Network
The abundance of open-source code, coupled with the success of recent advances in deep learning for natural language processing, has given rise to a promising new application of machine learning to source code. In this w…
BIG-bench Machine LearningUnderstanding Neural Machine Translation by Simplification: The Case of Encoder-free Models
In this paper, we try to understand neural machine translation (NMT) via simplifying NMT architectures and training encoder-free NMT models. In an encoder-free model, the sums of word embeddings and positional embeddings…
DecoderMachine TranslationNMTTranslation+1Deep Recurrent Generative Decoder for Abstractive Text Summarization
We propose a new framework for abstractive text summarization based on a sequence-to-sequence oriented encoder-decoder model equipped with a deep recurrent generative decoder (DRGN). Latent structure information implie…
Abstractive Text SummarizationDecoderText SummarizationVariational InferenceVariational Recurrent Neural Machine Translation
Partially inspired by successful applications of variational recurrent neural networks, we propose a novel variational recurrent neural machine translation (VRNMT) model in this paper. Different from the variational NMT,…
DecoderMachine TranslationNMTSentence+1