paper-with-me

홈 › Papers

Measuring Invariances in Deep Networks

2009-12-01 · NeurIPS 2009 12 · Ian Goodfellow, Honglak Lee, Quoc V. Le, Andrew Saxe, Andrew Y. Ng

For many computer vision applications, the ideal image feature would be invariant to multiple confounding image properties, such as illumination and viewing angle. Recently, deep architectures trained in an unsupervised manner have been proposed as an automatic method for extracting useful features. However, outside of using these learning algorithms in a classifier, they can be sometimes difficult to evaluate. In this paper, we propose a number of empirical tests that directly measure the degree to which these learned features are invariant to different image transforms. We find that deep autoencoders become invariant to increasingly complex image transformations with depth. This further justifies the use of “deep” vs. “shallower” representations. Our performance metrics agree with existing measures of invariance. Our evaluation metrics can also be used to evaluate future work in unsupervised deep learning, and thus help the development of future algorithms.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Measuring Representational Robustness of Neural Networks Through Shared Invariances

2022-06-23 · Vedant Nanda, Till Speicher, Camila Kolling, John P. Dickerson 외

A major challenge in studying robustness in deep learning is defining the set of ``meaningless'' perturbations to which a given Neural Network (NN) should be invariant. Most work on robustness implicitly uses a human as …

Do Invariances in Deep Neural Networks Align with Human Perception?

2021-11-29 · Vedant Nanda, Ayan Majumdar, Camila Kolling, John P. Dickerson 외

An evaluation criterion for safe and trustworthy deep learning is how well the invariances captured by representations of deep neural networks (DNNs) are shared with humans. We identify challenges in measuring these inva…

Data AugmentationSelf-Supervised Learning

Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation

2023-02-19 · Lorenz Kuhn, Yarin Gal, Sebastian Farquhar

We introduce a method to measure uncertainty in large language models. For tasks like question answering, it is essential to know when we can trust the natural language outputs of foundation models. We show that measurin…

Question AnsweringText Generation

Does Invariant Risk Minimization Capture Invariance?

2021-01-04 · Pritish Kamath, Akilesh Tangella, Danica J. Sutherland, Nathan Srebro

We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al. (2019) can fail to capture "natural" invariances, at least when used in its practical "linear" form, and even on very simple problems whic…

Learning Invariances in Neural Networks from Training Data

2020-12-01 · NeurIPS 2020 12 · Gregory Benton, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson

Invariances to translations have imbued convolutional neural networks with powerful generalization properties. However, we often do not know a priori what invariances are present in the data, or to what extent a model sh…

image-classificationImage ClassificationMolecular Property PredictionProperty Prediction