Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples

2020-10-20 · NeurIPS 2020 12 · Jay Nandy, Wynne Hsu, Mong Li Lee

Among existing uncertainty estimation approaches, Dirichlet Prior Network (DPN) distinctly models different predictive uncertainty types. However, for in-domain examples with high data uncertainties among multiple classes, even a DPN model often produces indistinguishable representations from the out-of-distribution (OOD) examples, compromising their OOD detection performance. We address this shortcoming by proposing a novel loss function for DPN to maximize the \textit{representation gap} between in-domain and OOD examples. Experimental results demonstrate that our proposed approach consistently improves OOD detection performance.

📄 PDF Abstract BibTeX arXiv:2010.10474

Code (1)

jayjaynandy/maximize-representation-gap 공식 구현 tf

Tasks

Out of Distribution (OOD) Detection

Methods 이 논문이 사용한 방법론

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Average Pooling 설명 없음

Concatenated Skip Connection A Concatenated Skip Connection is a type of skip connection that seeks to reuse features by concatenating them to new layers, allowing more information to be retained from…

Batch Normalization 설명 없음

Residual Connection 설명 없음

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Grouped Convolution A Grouped Convolution uses a group of convolutions - multiple kernels per layer - resulting in multiple channel outputs per layer. This leads to wider networks helping a…

Domain Adaptation via Maximizing Surrogate Mutual Information

2021-10-23 · Haiteng Zhao, Chang Ma, Qinyu Chen, Zhi-Hong Deng

Unsupervised domain adaptation (UDA) aims to predict unlabeled data from target domain with access to labeled data from the source domain. In this work, we propose a novel framework called SIDA (Surrogate Mutual Informat…

Domain Adaptation Transfer Learning Unsupervised Domain Adaptation

Domain Adaptation by Maximizing Population Correlation with Neural Architecture Search

2021-09-12 · Zhixiong Yue, Pengxin Guo, Yu Zhang

In Domain Adaptation (DA), where the feature distributions of the source and target domains are different, various distance-based methods have been proposed to minimize the discrepancy between the source and target domai…

Domain Adaptation Neural Architecture Search

Transfer Learning with Label Noise

2017-07-31 · Xiyu Yu, Tongliang Liu, Mingming Gong, Kun Zhang 외

Transfer learning aims to improve learning in target domain by borrowing knowledge from a related but different source domain. To reduce the distribution shift between source and target domains, recent methods have focus…

Denoising Transfer Learning

Label-Noise Robust Domain Adaptation

2020-01-01 · ICML 2020 1 · Xiyu Yu, Tongliang Liu, Mingming Gong, Kun Zhang 외

Domain adaptation aims to correct the classifiers when faced with distribution shift between source (training) and target (test) domains. State-of-the-art domain adaptation methods make use of deep networks to extract do…

Denoising Domain Adaptation

TTTFlow: Unsupervised Test-Time Training with Normalizing Flow

2022-10-20 · David Osowiechi, Gustavo A. Vargas Hakim, Mehrdad Noori, Milad Cheraghalikhani 외

A major problem of deep neural networks for image classification is their vulnerability to domain changes at test-time. Recent methods have proposed to address this problem with test-time training (TTT), where a two-bran…

image-classification Image Classification Source-Free Domain Adaptation Test-time Adaptation