cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree

2025-06-18 · Yilin Zhang, Xinran Zhao, Zora Zhiruo Wang, Chenyang Yang, Jiayi Wei, Tongshuang Wu

Retrieval-Augmented Generation (RAG) has become essential for large-scale code generation, grounding predictions in external code corpora to improve actuality. However, a critical yet underexplored aspect of RAG pipelines is chunking -- the process of dividing documents into retrievable units. Existing line-based chunking heuristics often break semantic structures, splitting functions or merging unrelated code, which can degrade generation quality. We propose chunking via Abstract Syntax Trees (\ourwork), a structure-aware method that recursively breaks large AST nodes into smaller chunks and merges sibling nodes while respecting size limits. This approach generates self-contained, semantically coherent units across programming languages and tasks, improving performance on diverse code generation tasks, e.g., boosting Recall@5 by 4.3 points on RepoEval retrieval and Pass@1 by 2.67 points on SWE-bench generation. Our work highlights the importance of structure-aware chunking for scaling retrieval-enhanced code intelligence.

📄 PDF Abstract BibTeX arXiv:2506.15655

Code (1)

yilinjz/astchunk 공식 구현

Tasks

Chunking Code Generation RAG RetrievalRetrieval-augmented Generation

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

TS-RAG: Retrieval Augmented Generation for Time Series Forecasting

2026-08-06 · Yixiong Xiao, Congxi Xiao, Jingbo Zhou arxiv

While deep learning models, particularly transformer-based architectures, have shown impressive performance in time series forecasting, the application of retrieval-augmented generation (RAG) in this domain remains limit…

Time Series Forecasting

Removal of Hallucination on Hallucination: Debate-Augmented RAG

2025-05-24 · Wentao Hu, WengYu Zhang, Yiyang Jiang, Chen Jason Zhang 외

Retrieval-Augmented Generation (RAG) enhances factual accuracy by integrating external knowledge, yet it introduces a critical issue: erroneous or biased retrieval can mislead generation, compounding hallucinations, a ph…

Hallucination RAG Retrieval Retrieval-augmented Generation

Enhancing Financial Time-Series Forecasting with Retrieval-Augmented Large Language Models

2025-02-09 · Mengxi Xiao, Zihao Jiang, Lingfei Qian, Zhengyu Chen 외

Stock movement prediction, a critical task in financial time-series forecasting, relies on identifying and retrieving key influencing factors from vast and complex datasets. However, traditional text-trained or numeric s…

Large Language Model RAG Retrieval Retrieval-augmented Generation+2

Stationarity-Aware Retrieval-Augmented Time Series Forecasting

2026-06-02 · Shiqiao Zhou, Holger Schöner, Zipeng Wu, Edouard Fouché 외 arxiv

Time series forecasting relies on historical patterns, but real-world series often exhibit non-stationarity and regime shifts that challenge fully parametric forecasters. Inspired by Retrieval-Augmented Generation (RAG),…

Time Series Forecasting

ARCS: Agentic Retrieval-Augmented Code Synthesis with Iterative Refinement

2025-04-29 · Manish Bhattarai, Miguel Cordova, Javier Santos, Dan O'Malley

In supercomputing, efficient and optimized code generation is essential to leverage high-performance systems effectively. We propose Agentic Retrieval-Augmented Code Synthesis (ARCS), an advanced framework for accurate, …

Code Generation HumanEval RAG Retrieval+2