An Initial Exploration of Fine-tuning Small Language Models for Smart Contract Reentrancy Vulnerability Detection
Large Language Models (LLMs) are being used more and more for various coding tasks, including to help coders identify bugs and are a promising avenue to support coders in various tasks including vulnerability detection -- particularly given the flexibility of such generative AI models and tools. Yet for many tasks it may not be suitable to use LLMs, for which it may be more suitable to use smaller language models that can fit and easily execute and train on a developer's computer. In this paper we explore and evaluate whether smaller language models can be fine-tuned to achieve reasonable results for a niche area: vulnerability detection -- specifically focusing on detecting the reentrancy bug in Solidity smart contracts.
Code (0)
등록된 구현이 없습니다.
Tasks
Vulnerability DetectionSimilar Papers 제목 키워드 기반
Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning
Through encouraging self-exploration, reinforcement learning from verifiable rewards (RLVR) has significantly advanced the mathematical reasoning capabilities of large language models. As the starting point for RLVR, the…
Reinforcement LearningMathematical ReasoningEntropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models
Supervised fine-tuning (SFT) followed by reinforcement learning (RL) has become a standard post-training paradigm for large language models. This paradigm provides a cold-start for RL exploration, avoiding the inefficien…
Reinforcement LearningMathematical ReasoningEnhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
Large language models (LLMs) primarily rely on supervised fine-tuning (SFT) as a key method to adapt pre-trained models to domain-specific tasks such as mathematical reasoning. However, standard SFT uniformly penalizes a…
Reinforcement LearningMathematical ReasoningStep-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models
Fine-tuning large language models (LLMs) on downstream tasks requires substantial computational resources. A class of parameter-efficient fine-tuning (PEFT) aims to mitigate these computational challenges by selectively …
Computational EfficiencyNatural Language Understandingparameter-efficient fine-tuningFine-Tuning Large Neural Language Models for Biomedical Natural Language Processing
Motivation: A perennial challenge for biomedical researchers and clinical practitioners is to stay abreast with the rapid growth of publications and medical notes. Natural language processing (NLP) has emerged as a promi…
text similarityTransfer Learning