Enhancing Neural Code Representation with Additional Context

2025-10-14 · Huy Nguyen, Christoph Treude, Patanamon Thongtanunam arxiv

Automated program comprehension underpins many software engineering tasks, from code summarisation to clone detection. Recent deep learning models achieve strong results but typically rely on source code alone, overlooking contextual information such as version history or structural relationships. This limits their ability to capture how code evolves and operates. We conduct an empirical study on how enriching code representations with such contextual signals affects neural model performance on key comprehension tasks. Two downstream tasks, code clone detection and code summarisation, are evaluated using SeSaMe (1,679 Java methods) and CodeSearchNet (63,259 methods). Five representative models (CodeBERT, GraphCodeBERT, CodeT5, PLBART, ASTNN) are fine-tuned under code-only and context-augmented settings. Results show that context generally improves performance: version history consistently boosts clone detection (e.g., CodeT5 +15.92% F1) and summarisation (e.g., GraphCodeBERT +5.56% METEOR), while call-graph effects vary by model and task. Combining multiple contexts yields further gains (up to +21.48% macro-F1). Human evaluation on 100 Java snippets confirms that context-augmented summaries are significantly preferred for Accuracy and Content Adequacy (p <= 0.026; |delta| up to 0.55). These findings highlight the potential of contextual signals to enhance code comprehension and open new directions for optimising contextual encoding in neural SE models.

📄 PDF Abstract BibTeX arXiv:2510.12082

Code (0)

등록된 구현이 없습니다.

Leveraging Summary Guidance on Medical Report Summarization

2023-02-08 · Yunqi Zhu, Xuebing Yang, Yuanyuan Wu, Wensheng Zhang

This study presents three deidentified large medical text datasets, named DISCHARGE, ECHO and RADIOLOGY, which contain 50K, 16K and 378K pairs of report and summary that are derived from MIMIC-III, respectively. We imple…

16k Abstractive Text Summarization Decoder

Drop your Decoder: Pre-training with Bag-of-Word Prediction for Dense Passage Retrieval

2024-01-20 · Guangyuan Ma, Xing Wu, Zijia Lin, Songlin Hu

Masked auto-encoder pre-training has emerged as a prevalent technique for initializing and enhancing dense retrieval systems. It generally utilizes additional Transformer decoder blocks to provide sustainable supervision…

Decoder Passage Retrieval Retrieval Unsupervised Pre-training

FusionMind -- Improving question and answering with external context fusion

2023-12-31 · Shreyas Verma, Manoj Parmar, Palash Choudhary, Sanchita Porwal

Answering questions using pre-trained language models (LMs) and knowledge graphs (KGs) presents challenges in identifying relevant knowledge and performing joint reasoning.We compared LMs (fine-tuned for the task) with t…

Knowledge Graphs Multiple-choice Question Answering

ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs

2026-06-04 · Ailiang Lin, Zhuoyun Li, Yusong Wang, Keyu Mao 외 arxiv

Recent advances in Large Language Models (LLMs) have opened new avenues for generating training-free text embeddings. However, the causal attention in decoder-only LLMs prevents earlier tokens from attending to future co…

DECAMP: Towards Scene-Consistent Multi-Agent Motion Prediction with Disentangled Context-Aware Pre-Training

2025-09-12 · Jianxin Shi, Zengqi Peng, Xiaolong Chen, Tianyu Wo 외 arxiv

Trajectory prediction is a critical component of autonomous driving, essential for ensuring both safety and efficiency on the road. However, traditional approaches often struggle with the scarcity of labeled data and exh…

Representation Learning Trajectory Prediction Autonomous Driving Motion Forecasting