Autoencoding Under Normalization Constraints
Likelihood is a standard estimate for outlier detection. The specific role of the normalization constraint is to ensure that the out-of-distribution (OOD) regime has a small likelihood when samples are learned using maximum likelihood. Because autoencoders do not possess such a process of normalization, they often fail to recognize outliers even when they are obviously OOD. We propose the Normalized Autoencoder (NAE), a normalized probabilistic model constructed from an autoencoder. The probability density of NAE is defined using the reconstruction error of an autoencoder, which is differently defined in the conventional energy-based model. In our model, normalization is enforced by suppressing the reconstruction of negative samples, significantly improving the outlier detection performance. Our experimental results confirm the efficacy of NAE, both in detecting outliers and in generating in-distribution samples.
Code (2)
Tasks
Outlier DetectionSimilar Papers 제목 키워드 기반
Few-Shot and Zero-Shot Learning for Historical Text Normalization
Historical text normalization often relies on small training datasets. Recent work has shown that multi-task learning can lead to significant improvements by exploiting synergies with related datasets, but there has been…
LemmatizationMulti-Task LearningText NormalizationZero-Shot LearningGeneral Point Model with Autoencoding and Autoregressive
The pre-training architectures of large language models encompass various types, including autoencoding models, autoregressive models, and encoder-decoder models. We posit that any modality can potentially benefit from a…
DecoderLanguage ModelingLanguage ModellingLarge Language Model+3General Point Model Pretraining with Autoencoding and Autoregressive
The pre-training architectures of large language models encompass various types including autoencoding models autoregressive models and encoder-decoder models. We posit that any modality can potentially benefit from …
DecoderLanguage ModelingLanguage ModellingLarge Language Model+3Evolution is Driven by Natural Autoencoding: Reframing Species, Interaction Codes, Cooperation, and Sexual Reproduction
The continuity of life and its evolution, we proposed, emerge from an interactive group process manifested in networks of interaction. We term this process \textit{survival-of-the-fitted}. Here, we reason that survival o…
Dimensionality ReductionVAMAE: Vessel-Aware Masked Autoencoders for OCT Angiography
Optical coherence tomography angiography (OCTA) provides non-invasive visualization of retinal microvasculature, but learning robust representations remains challenging due to sparse vessel structures and strong topologi…
Self-Supervised Learning