Addressing Leakage in Self-Supervised Contextualized Code Retrieval
We address contextualized code retrieval, the search for code snippets helpful to fill gaps in a partial input program. Our approach facilitates a large-scale self-supervised contrastive training by splitting source code randomly into contexts and targets. To combat leakage between the two, we suggest a novel approach based on mutual identifier masking, dedentation, and the selection of syntax-aligned targets. Our second contribution is a new dataset for direct evaluation of contextualized code retrieval, based on a dataset of manually aligned subpassages of code clones. Our experiments demonstrate that our approach improves retrieval substantially, and yields new state-of-the-art results for code clone and defect detection.
Code (0)
등록된 구현이 없습니다.
Tasks
Defect DetectionRetrievalSimilar Papers 제목 키워드 기반
Self-supervised Learning of Contextualized Local Visual Embeddings
We present Contextualized Local Visual Embeddings (CLoVE), a self-supervised convolutional-based method that learns representations suited for dense prediction tasks. CLoVE deviates from current methods and optimizes a s…
Instance SegmentationKeypoint Detectionobject-detectionObject Detection+3Masked Autoencoders for Point Cloud Self-supervised Learning
As a promising scheme of self-supervised learning, masked autoencoding has significantly advanced natural language processing and computer vision. Inspired by this, we propose a neat scheme of masked autoencoders for poi…
3D Part Segmentation3D Point Cloud ClassificationFew-Shot 3D Point Cloud ClassificationFew-Shot Learning+1Distill-SODA: Distilling Self-Supervised Vision Transformer for Source-Free Open-Set Domain Adaptation in Computational Pathology
Developing computational pathology models is essential for reducing manual tissue typing from whole slide images, transferring knowledge from the source domain to an unlabeled, shifted target domain, and identifying unse…
Data AugmentationDomain AdaptationOpen Set Learningwhole slide imagesEfficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language
Current self-supervised learning algorithms are often modality-specific and require large amounts of computational resources. To address these issues, we increase the training efficiency of data2vec, a learning objective…
Decoderimage-classificationImage ClassificationNatural Language Understanding+3SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction
Open relation extraction is the task of extracting open-domain relation facts from natural language sentences. Existing works either utilize heuristics or distant-supervised annotations to train a supervised classifier o…
ClusteringLanguage ModelingLanguage ModellingRelation+2