Specifying and Testing $k$-Safety Properties for Machine-Learning Models
Machine-learning models are becoming increasingly prevalent in our lives, for instance assisting in image-classification or decision-making tasks. Consequently, the reliability of these models is of critical importance and has resulted in the development of numerous approaches for validating and verifying their robustness and fairness. However, beyond such specific properties, it is challenging to specify, let alone check, general functional-correctness expectations from models. In this paper, we take inspiration from specifications used in formal methods, expressing functional-correctness properties by reasoning about $k$ different executions, so-called $k$-safety properties. Considering a credit-screening model of a bank, the expected property that "if a person is denied a loan and their income decreases, they should still be denied the loan" is a 2-safety property. Here, we show the wide applicability of $k$-safety properties for machine-learning models and present the first specification language for expressing them. We also operationalize the language in a framework for automatically validating such properties using metamorphic testing. Our experiments show that our framework is effective in identifying property violations, and that detected bugs could be used to train better models.
Code (1)
Tasks
BIG-bench Machine LearningDecision MakingFairnessimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Software Testing for Machine Learning
Machine learning has become prevalent across a wide variety of applications. Unfortunately, machine learning has also shown to be susceptible to deception, leading to errors, and even fatal failures. This circumstance ca…
BIG-bench Machine Learningsoftware testingVehicle: Bridging the Embedding Gap in the Verification of Neuro-Symbolic Programs
Neuro-symbolic programs -- programs containing both machine learning components and traditional symbolic code -- are becoming increasingly widespread. However, we believe that there is still a lack of a general methodolo…
Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification
LLM agents increasingly perform autonomous actions through external tools, leading to complex and evolving safety risks. However, existing safety testing targets expert-designed safety violations, and the corresponding o…
Towards Probability-based Safety Verification of Systems with Components from Machine Learning
Machine learning (ML) has recently created many new success stories. Hence, there is a strong motivation to use ML technology in software-intensive systems, including safety-critical systems. This raises the issue of saf…
BIG-bench Machine LearningTest Where Decisions Matter: Importance-driven Testing for Deep Reinforcement Learning
In many Deep Reinforcement Learning (RL) problems, decisions in a trained policy vary in significance for the expected safety and performance of the policy. Since RL policies are very complex, testing efforts should conc…
Deep Reinforcement LearningReinforcement Learning (RL)