Data Augmentation by Fuzzing for Neural Test Generation
Testing is essential to modern software engineering for building reliable software. Given the high costs of manually creating test cases, automated test case generation, particularly methods utilizing large language models, has become increasingly popular. These neural approaches generate semantically meaningful tests that are more maintainable compared with traditional automatic testing methods like fuzzing. However, the diversity and volume of unit tests in current datasets are limited. In this paper, we introduce a novel data augmentation technique, *FuzzAug*, that introduces the benefits of fuzzing to large language models to preserve valid program semantics and provide diverse inputs. This enhances the model's ability to embed correct inputs that can explore more branches of the function under test. Our evaluations show that models trained with dataset augmented by FuzzAug increase assertion accuracy by 5%, improve compilation rate by more than 10%, and generate unit test functions with 5% more branch coverage. This technique demonstrates the potential of using dynamic software testing to improve neural test generation, offering significant enhancements in neural test generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Code GenerationData AugmentationDiversitysoftware testingvalidSimilar Papers 제목 키워드 기반
DeepSmartFuzzer: Reward Guided Test Generation For Deep Learning
Testing Deep Neural Network (DNN) models has become more important than ever with the increasing usage of DNN models in safety-critical domains such as autonomous cars. The traditional approach of testing DNNs is to crea…
Data AugmentationDeep LearningLegion: Best-First Concolic Testing
Concolic execution and fuzzing are two complementary coverage-based testing techniques. How to achieve the best of both remains an open challenge. To address this research problem, we propose and evaluate Legion. Legion …
Decision MakingDecision Making Under UncertaintySequential Decision MakingGreybox fuzzing as a contextual bandits problem
Greybox fuzzing is one of the most useful and effective techniques for the bug detection in large scale application programs. It uses minimal amount of instrumentation. American Fuzzy Lop (AFL) is a popular coverage base…
Multi-Armed BanditsLearn&Fuzz: Machine Learning for Input Fuzzing
Fuzzing consists of repeatedly testing an application with modified, or fuzzed, inputs with the goal of finding security vulnerabilities in input-parsing code. In this paper, we show how to automate the generation of an …
BIG-bench Machine LearningOn the Challenges of Fuzzing Techniques via Large Language Models
In the modern era where software plays a pivotal role, software security and vulnerability analysis are essential for secure software development. Fuzzing test, as an efficient and traditional software testing method, ha…
software testingSurvey