Retrieval-based Layer-wise Adaptive Transformer for Source Code Summarization

2021-11-16 · ACL ARR November 2021 11 · Anonymous

We propose a model that learns both the sequential and the structural features of code for source code summarization. We adopt the Abstract Syntax Tree (AST) and graph convolution to model the structural information and the Transformer to model the sequential information. We convert code snippets into ASTs and apply graph convolution to obtain structurally-encoded node representations. Then, the sequences of the graph-convolutioned AST nodes are processed by the Transformer layers. Since structurally-neighboring nodes will have similar representations in graph-convolutioned trees, the Transformer layers can effectively capture not only the sequential information but also the structural information such as sentences or blocks of source code. We show that our model outperforms the state-of-the-art for source code summarization by experiments and human evaluations.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Code SummarizationRetrieval Source Code Summarization

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

MINER: Mining Multimodal Internal Representation for Efficient Retrieval

2026-05-07 · Weien Li, Rui Song, Zeyu Li, Haochen Liu 외 arxiv

Visual document retrieval has become essential for accessing information in visually rich documents. Existing approaches fall into two camps. Late-interaction retrievers achieve strong quality through fine-grained token-…

The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP

2026-05-26 · Kahyeon Nam, Hyesong Choi arxiv

Deploying Vision-Language Models on resource-constrained hardware typically requires INT8 quantization, but in joint-embedding architectures such as CLIP this introduces a failure mode distinct from quantized CNN classif…

Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking

2026-04-17 · Jun Feng, Jiahui Tang, Zhicheng He, Hang Lv 외 arxiv

Adaptive Retrieval-Augmented Generation aims to mitigate the interference of extraneous noise by dynamically determining the necessity of retrieving supplementary passages. However, as Large Language Models evolve with i…

ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models

2025-08-10 · Archchana Sindhujan, Shenbin Qian, Chan Chi Chun Matthew, Constantin Orasan 외 arxiv

Large Language Models (LLMs) have shown remarkable performance across a wide range of natural language processing tasks. Quality Estimation (QE) for Machine Translation (MT), which assesses the quality of a source-target…

Machine Translation Language Modelling

Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers

2026-05-08 · Mana Sakai, Masaaki Imaizumi arxiv

Understanding why trained Transformers generalize well is a fundamental problem in modern machine learning theory, and complexity-based generalization bounds provide a principled way to study this question. While existin…