Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection
Detecting Out-of-distribution (OOD) inputs have been a critical issue for neural networks in the open world. However, the unstable behavior of OOD detection along the optimization trajectory during training has not been explored clearly. In this paper, we first find the performance of OOD detection suffers from overfitting and instability during training: 1) the performance could decrease when the training error is near zero, and 2) the performance would vary sharply in the final stage of training. Based on our findings, we propose Average of Pruning (AoP), consisting of model averaging and pruning, to mitigate the unstable behaviors. Specifically, model averaging can help achieve a stable performance by smoothing the landscape, and pruning is certified to eliminate the overfitting by eliminating redundant features. Comprehensive experiments on various datasets and architectures are conducted to verify the effectiveness of our method.
Code (0)
등록된 구현이 없습니다.
Tasks
Out-of-Distribution DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Small Object Detection Based on Modified FSSD and Model Compression
Small objects have relatively low resolution, the unobvious visual features which are difficult to be extracted, so the existing object detection methods cannot effectively detect small objects, and the detection speed a…
Model Compressionobject-detectionObject DetectionSmall Object DetectionITP: Instance-Aware Test Pruning for Out-of-Distribution Detection
Out-of-distribution (OOD) detection is crucial for ensuring the reliable deployment of deep models in real-world scenarios. Recently, from the perspective of over-parameterization, a series of methods leveraging weight s…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionLayer-Adaptive State Pruning for Deep State Space Models
Due to the lack of state dimension optimization methods, deep state space models (SSMs) have sacrificed model capacity, training search space, or stability to alleviate computational costs caused by high state dimensions…
State Space ModelsDRoP: Distributionally Robust Pruning
In the era of exceptionally data-hungry models, careful selection of the training data is essential to mitigate the extensive costs of deep learning. Data pruning offers a solution by removing redundant or uninformative …
FairnessResource-Constrained Affect Modelling via Variance Regularisation Pruning
Affective computing systems are increasingly embedded in pervasive and interactive environments, such as adaptive games, assistive technologies, and resource-constrained platforms, where computational efficiency must be …
Computational Efficiency