Software Vulnerability and Functionality Assessment using LLMs
While code review is central to the software development process, it can be tedious and expensive to carry out. In this paper, we investigate whether and how Large Language Models (LLMs) can aid with code reviews. Our investigation focuses on two tasks that we argue are fundamental to good reviews: (i) flagging code with security vulnerabilities and (ii) performing software functionality validation, i.e., ensuring that code meets its intended functionality. To test performance on both tasks, we use zero-shot and chain-of-thought prompting to obtain final ``approve or reject'' recommendations. As data, we employ seminal code generation datasets (HumanEval and MBPP) along with expert-written code snippets with security vulnerabilities from the Common Weakness Enumeration (CWE). Our experiments consider a mixture of three proprietary models from OpenAI and smaller open-source LLMs. We find that the former outperforms the latter by a large margin. Motivated by promising results, we finally ask our models to provide detailed descriptions of security vulnerabilities. Results show that 36.7% of LLM-generated descriptions can be associated with true CWE vulnerabilities.
Code (0)
등록된 구현이 없습니다.
Tasks
Code GenerationHumanEvalmbppSimilar Papers 제목 키워드 기반
ChatNVD: Advancing Cybersecurity Vulnerability Assessment with Large Language Models
The increasing frequency and sophistication of cybersecurity vulnerabilities in software systems underscores the need for more robust and effective vulnerability assessment methods. However, existing approaches often rel…
Secure Code Generation via Online Reinforcement Learning with Vulnerability Reward Model
Large language models (LLMs) are increasingly used in software development, yet their tendency to generate insecure code remains a major barrier to real-world deployment. Existing secure code alignment methods often suff…
Vulnerability DetectionReinforcement LearningCode GenerationEvalSVA: Multi-Agent Evaluators for Next-Gen Software Vulnerability Assessment
Software Vulnerability (SV) assessment is a crucial process of determining different aspects of SVs (e.g., attack vectors and scope) for developers to effectively prioritize efforts in vulnerability mitigation. It presen…
Towards an Improved Understanding of Software Vulnerability Assessment Using Data-Driven Approaches
The thesis advances the field of software security by providing knowledge and automation support for software vulnerability assessment using data-driven approaches. Software vulnerability assessment provides important an…
Helpful or Harmful? Evaluating LLM-Assisted Vulnerability Patching via a Human Study
Software vulnerability remediation is a cognitively demanding task that requires specialized security expertise often lacking in general developers. In the meantime, Large Language Models (LLMs) assisted tools show poten…
Vulnerability Detection