FANNet: Formal Analysis of Noise Tolerance, Training Bias and Input Sensitivity in Neural Networks
With a constant improvement in the network architectures and training methodologies, Neural Networks (NNs) are increasingly being deployed in real-world Machine Learning systems. However, despite their impressive performance on "known inputs", these NNs can fail absurdly on the "unseen inputs", especially if these real-time inputs deviate from the training dataset distributions, or contain certain types of input noise. This indicates the low noise tolerance of NNs, which is a major reason for the recent increase of adversarial attacks. This is a serious concern, particularly for safety-critical applications, where inaccurate results lead to dire consequences. We propose a novel methodology that leverages model checking for the Formal Analysis of Neural Network (FANNet) under different input noise ranges. Our methodology allows us to rigorously analyze the noise tolerance of NNs, their input node sensitivity, and the effects of training bias on their performance, e.g., in terms of classification accuracy. For evaluation, we use a feed-forward fully-connected NN architecture trained for the Leukemia classification. Our experimental results show $\pm 11\%$ noise tolerance for the given trained network, identify the most sensitive input nodes, and confirm the biasness of the available training dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationSensitivitySimilar Papers 제목 키워드 기반
Scaling Model Checking for DNN Analysis via State-Space Reduction and Input Segmentation (Extended Version)
Owing to their remarkable learning capabilities and performance in real-world applications, the use of machine learning systems based on Neural Networks (NNs) has been continuously increasing. However, various case studi…
Efficient PAC Learning of Halfspaces with Constant Malicious Noise Rate
Understanding noise tolerance of machine learning algorithms is a central quest in learning theory. In this work, we study the problem of computationally efficient PAC learning of halfspaces in the presence of malicious …
Learning TheoryPAC learningDenoised Conformal Alignment for Reliable Selection of Conditional Average Treatment Effect Predictions
In selective deployment, practitioners act only on a model-chosen subset of individuals based on predicted conditional average treatment effects, but marginal conformal guarantees need not control reliability on that sel…
Gap-Increasing Policy Evaluation for Efficient and Noise-Tolerant Reinforcement Learning
In real-world applications of reinforcement learning (RL), noise from inherent stochasticity of environments is inevitable. However, current policy evaluation algorithms, which plays a key role in many RL algorithms, are…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Fault Tolerance of Neural Networks in Adversarial Settings
Artificial Intelligence systems require a through assessment of different pillars of trust, namely, fairness, interpretability, data and model privacy, reliability (safety) and robustness against against adversarial atta…
Adversarial RobustnessFairness