Testing Framework for Black-box AI Models
With widespread adoption of AI models for important decision making, ensuring reliability of such models remains an important challenge. In this paper, we present an end-to-end generic framework for testing AI Models which performs automated test generation for different modalities such as text, tabular, and time-series data and across various properties such as accuracy, fairness, and robustness. Our tool has been used for testing industrial AI models and was very effective to uncover issues present in those models. Demo video link: https://youtu.be/984UCU17YZI
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingFairnessTime SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Data Synthesis for Testing Black-Box Machine Learning Models
The increasing usage of machine learning models raises the question of the reliability of these models. The current practice of testing with limited data is often insufficient. In this paper, we provide a framework for a…
BIG-bench Machine LearningEfficient Black-box Assessment of Autonomous Vehicle Safety
While autonomous vehicle (AV) technology has shown substantial progress, we still lack tools for rigorous and scalable testing. Real-world testing, the $\textit{de-facto}$ evaluation method, is dangerous to the public. M…
Interpreting Black Box Models via Hypothesis Testing
In science and medicine, model interpretations may be reported as discoveries of natural phenomena or used to guide patient treatments. In such high-stakes tasks, false discoveries may lead investigators astray. These ap…
Two-sample testingOntology Based Test Case Generation for Black Box Testing
Software systems are not considered complete unless properly tested and verified. In existing literature, a growing interest on establishment of automated testing techniques has been observed. However, tedious manual pro…
software testingAn Efficient Model Inference Algorithm for Learning-based Testing of Reactive Systems
Learning-based testing (LBT) is an emerging methodology to automate iterative black-box requirements testing of software systems. The methodology involves combining model inference with model checking techniques. However…
Incremental Learning