paper-with-me

홈 › Papers

A Transformer-based Approach for Source Code Summarization

2020-05-01 · ACL 2020 6 · Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang

Generating a readable summary that describes the functionality of a program is known as source code summarization. In this task, learning code representation by modeling the pairwise relationship between code tokens to capture their long-range dependencies is crucial. To learn code representation for summarization, we explore the Transformer model that uses a self-attention mechanism and has shown to be effective in capturing long-range dependencies. In this work, we show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin. We perform extensive analysis and ablation studies that reveal several important findings, e.g., the absolute encoding of source code tokens' position hinders, while relative encoding significantly improves the summarization performance. We have made our code publicly available to facilitate future research.

📄 PDF Abstract BibTeX arXiv:2005.00653

Code (9)

wasiahmad/NeuralCodeSum 공식 구현 pytorch
CGCL-codes/naturalcc pytorch
MaratSaidov/source-code-summarization
Zhangxq-1/coderepository pytorch
code-desc/codesc pytorch
csebuetnlp/CoDesc pytorch
jadecxliu/codeqa pytorch
sjj0403/GSCS pytorch
tmuttaqueen/NeuralCodeSum-smalltest pytorch

Tasks

Code SummarizationPositionSource Code Summarization

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Position-Wise Feed-Forward Layer 설명 없음
Residual Connection 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Similar Papers 제목 키워드 기반

Retrieval-based Layer-wise Adaptive Transformer for Source Code Summarization

2021-11-16 · ACL ARR November 2021 11 · Anonymous

We propose a model that learns both the sequential and the structural features of code for source code summarization. We adopt the Abstract Syntax Tree (AST) and graph convolution to model the structural information and …

Code SummarizationRetrievalSource Code Summarization

AST-Transformer: Encoding Abstract Syntax Trees Efficiently for Code Summarization

2021-12-02 · Ze Tang, Chuanyi Li, Jidong Ge, Xiaoyu Shen 외

Code summarization aims to generate brief natural language descriptions for source code. As source code is highly structured and follows strict programming language grammars, its Abstract Syntax Tree (AST) is often lever…

Code Summarization

Efficient Extractive Summarization with MAMBA-Transformer Hybrids for Low-Resource Scenarios

2026-03-01 · Nisrine Ait Khayi arxiv

Extractive summarization of long documents is bottlenecked by quadratic complexity, often forcing truncation and limiting deployment in resource-constrained settings. We introduce the first Mamba-Transformer hybrid for e…

ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation

2022-05-13 · NAACL (ACL) 2022 7 · Long Phan, Hieu Tran, Hieu Nguyen, Trieu H. Trinh

We present ViT5, a pretrained Transformer-based encoder-decoder model for the Vietnamese language. With T5-style self-supervised pretraining, ViT5 is trained on a large corpus of high-quality and diverse Vietnamese texts…

Abstractive Text SummarizationDecodernamed-entity-recognitionNamed Entity Recognition+4

GN-Transformer: Fusing Sequence and Graph Representation for Improved Code Summarization

2021-11-17 · Junyan Cheng, Iordanis Fostiropoulos, Barry Boehm

As opposed to natural languages, source code understanding is influenced by grammatical relationships between tokens regardless of their identifier name. Graph representations of source code such as Abstract Syntax Tree …

Code Summarization