Topical: Learning Repository Embeddings from Source Code using Attention
This paper presents Topical, a novel deep neural network for repository level embeddings. Existing methods, reliant on natural language documentation or naive aggregation techniques, are outperformed by Topical's utilization of an attention mechanism. This mechanism generates repository-level representations from source code, full dependency graphs, and script level textual data. Trained on publicly accessible GitHub repositories, Topical surpasses multiple baselines in tasks such as repository auto-tagging, highlighting the attention mechanism's efficacy over traditional aggregation methods. Topical also demonstrates scalability and efficiency, making it a valuable contribution to repository-level representation computation. For further research, the accompanying tools, code, and training dataset are provided at: https://github.com/jpmorganchase/topical.
Code (1)
Similar Papers 제목 키워드 기반
Neural Attention-Aware Hierarchical Topic Model
Neural topic models (NTMs) apply deep neural networks to topic modelling. Despite their success, NTMs generally ignore two important aspects: (1) only document-level word count information is utilized for the training, w…
modelSentenceTopic ModelsGrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge
Contextualized embeddings based on large language models (LLMs) are available for various languages, but their coverage is often limited for lower resourced languages. Using LLMs for such languages is often difficult due…
Natural Language InferenceSentiment AnalysisTopic ClassificationWord Embeddings+1JOINTLY LEARNING TOPIC SPECIFIC WORD AND DOCUMENT EMBEDDING
Document embedding generally ignores underlying topics, which fails to capture polysemous terms that can mislead to improper thematic representation. Moreover, embedding a new document during the test process needs a com…
Document ClassificationDocument EmbeddingWord EmbeddingsGrammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings
Recent research has demonstrated that vector space models of semantics can reflect undesirable biases in human culture. Our investigation of crosslinguistic word embeddings reveals that topical gender bias interacts with…
Cultural Vocal Bursts Intensity PredictionLemmatizationMachine TranslationTranslation+1Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks
Recent advances in Large Language Models (LLMs) have shown promise in function-level code generation, yet repository-level software engineering tasks remain challenging. Current solutions predominantly rely on proprietar…
Code GenerationLanguage ModelingLanguage ModellingLarge Language Model+2