Redundancy and Concept Analysis for Code-trained Language Models
Code-trained language models have proven to be highly effective for various code intelligence tasks. However, they can be challenging to train and deploy for many software engineering applications due to computational bottlenecks and memory constraints. Implementing effective strategies to address these issues requires a better understanding of these 'black box' models. In this paper, we perform the first neuron-level analysis for source code models to identify \textit{important} neurons within latent representations. We achieve this by eliminating neurons that are highly similar or irrelevant to the given task. This approach helps us understand which neurons and layers can be eliminated (redundancy analysis) and where important code properties are located within the network (concept analysis). Using redundancy analysis, we make observations relevant to knowledge transfer and model optimization applications. We find that over 95\% of the neurons are redundant with respect to our code intelligence tasks and can be eliminated without significant loss in accuracy. We also discover several subsets of neurons that can make predictions with baseline accuracy. Through concept analysis, we explore the traceability and distribution of human-recognizable concepts within latent code representations which could be used to influence model predictions. We trace individual and subsets of important neurons to specific code properties and identify 'number' neurons, 'string' neurons, and higher-level 'text' neurons for token-level tasks and higher-level concepts important for sentence-level downstream tasks. This also helps us understand how decomposable and transferable task-related features are and can help devise better techniques for transfer learning, model compression, and the decomposition of deep neural networks into modules.
Code (0)
등록된 구현이 없습니다.
Tasks
MemorizationModel CompressionModel OptimizationSentenceTransfer LearningVulnerability DetectionSimilar Papers 제목 키워드 기반
DNA and Human Language: Epigenetic Memory and Redundancy in Linear Sequence
DNA is often described as the 'language of life', but whether it possesses formal linguistic properties remains unresolved. Here, we present the first empirical evidence that DNA sequences exhibit core linguistic feature…
Delving Deeper: Hierarchical Visual Perception for Robust Video-Text Retrieval
Video-text retrieval (VTR) aims to locate relevant videos using natural language queries. Current methods, often based on pre-trained models like CLIP, are hindered by video's inherent redundancy and their reliance on co…
Natural Language QueriesVideo-Text RetrievalR^2MoE: Redundancy-Removal Mixture of Experts for Lifelong Concept Learning
Enabling large-scale generative models to continuously learn new visual concepts is essential for personalizing pre-trained models to meet individual user preferences. Existing approaches for continual visual concept lea…
Mixture-of-ExpertsConceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
Vision-language models learn powerful multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract interpretable features, they rely on expanding the representation dime…
Measuring the Redundancy of Decoder Layers in SpeechLLMs
Speech Large Language Models route speech encoder representations into an LLM decoder that typically accounts for over 90% of total parameters. We study how much of this decoder capacity is actually needed for speech tas…