Prior Activation Distribution (PAD): A Versatile Representation to Utilize DNN Hidden Units
In this paper, we introduce the concept of Prior Activation Distribution (PAD) as a versatile and general technique to capture the typical activation patterns of hidden layer units of a Deep Neural Network used for classification tasks. We show that the combined neural activations of such a hidden layer have class-specific distributional properties, and then define multiple statistical measures to compute how far a test sample's activations deviate from such distributions. Using a variety of benchmark datasets (including MNIST, CIFAR10, Fashion-MNIST & notMNIST), we show how such PAD-based measures can be used, independent of any training technique, to (a) derive fine-grained uncertainty estimates for inferences; (b) provide inferencing accuracy competitive with alternatives that require execution of the full pipeline, and (c) reliably isolate out-of-distribution test samples.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Versatile Physics-based Character Control with Hybrid Latent Representation
We present a versatile latent representation that enables physically simulated character to efficiently utilize motion priors. To build a powerful motion embedding that is shared across multiple tasks, the physics contro…
Motion Generationmotion in-betweeningQuantizationSelf-Distribution Binary Neural Networks
In this work, we study the binary neural networks (BNNs) of which both the weights and activations are binary (i.e., 1-bit representation). Feature representation is critical for deep neural networks, while in BNNs, the …
QuantizationThe Spotlight Resonance Method: Resolving the Alignment of Embedded Activations
Understanding how deep learning models represent data is currently difficult due to the limited number of methodologies available. This paper demonstrates a versatile and novel visualisation tool for determining the axis…
UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering
Activation-based control steers large language models (LLMs) by intervening on their internal representations during inference, and has emerged as an effective paradigm for controlling behaviors such as persona and style…
Instruction FollowingExploring Versatile Prior for Human Motion via Motion Frequency Guidance
Prior plays an important role in providing the plausible constraint on human motion. Previous works design motion priors following a variety of paradigms under different circumstances, leading to the lack of versatility.…
DenoisingRepresentation Learning