Metamorphic Relation Prioritization for Effective Regression Testing
Metamorphic testing (MT) is widely used for testing programs that face the oracle problem. It uses a set of metamorphic relations (MRs), which are relations among multiple inputs and their corresponding outputs to determine whether the program under test is faulty. Typically, MRs vary in their ability to detect faults in the program under test, and some MRs tend to detect the same set of faults. In this paper, we propose approaches to prioritize MRs to improve the efficiency and effectiveness of MT for regression testing. We present two MR prioritization approaches: (1) fault-based and (2) coverage-based. To evaluate these MR prioritization approaches, we conduct experiments on three complex open-source software systems. Our results show that the MR prioritization approaches developed by us significantly outperform the current practice of executing the source and follow-up test cases of the MRs in an ad-hoc manner in terms of fault detection effectiveness. Further, fault-based MR prioritization leads to reducing the number of source and follow-up test cases that needs to be executed as well as reducing the average time taken to detect a fault, which would result in saving time and cost during the testing process.
Code (0)
등록된 구현이 없습니다.
Tasks
Fault DetectionregressionRelationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Efficient Fairness Testing in Large Language Models: Prioritizing Metamorphic Relations for Bias Detection
Large Language Models (LLMs) are increasingly deployed in various applications, raising critical concerns about fairness and potential biases in their outputs. This paper explores the prioritization of metamorphic relati…
Bias DetectionDiversityFairnessFault Detection+1Automated identification of metamorphic test scenarios for an ocean-modeling application
Metamorphic testing seeks to validate software in the absence of test oracles. Our application domain is ocean modeling, where test oracles often do not exist, but where symmetries of the simulated physical systems are k…
BIG-bench Machine LearningRelationObject-based Metamorphic Testing through Image Structuring
Testing software is often costly due to the need of mass-producing test cases and providing a test oracle for it. This is often referred to as the oracle problem. One method that has been proposed in order to alleviate t…
ObjectMetamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning
Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations. This is called the Rashomon effect of (explainable) machine learning…
Feature ImportanceSystematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing Perspective
Metamorphic testing has recently been used to check the safety of neural NLP models. Its main advantage is that it does not rely on a ground truth to generate test cases. However, existing studies are mostly concerned wi…