Deep Isometric Learning for Visual Recognition

2020-06-30 · ICML 2020 1 · Haozhi Qi, Chong You, Xiaolong Wang, Yi Ma, Jitendra Malik

Initialization, normalization, and skip connections are believed to be three indispensable techniques for training very deep convolutional neural networks and obtaining state-of-the-art performance. This paper shows that deep vanilla ConvNets without normalization nor skip connections can also be trained to achieve surprisingly good performance on standard image recognition benchmarks. This is achieved by enforcing the convolution kernels to be near isometric during initialization and training, as well as by using a variant of ReLU that is shifted towards being isometric. Further experiments show that if combined with skip connections, such near isometric networks can achieve performances on par with (for ImageNet) and better than (for COCO) the standard ResNet, even without normalization at all. Our code is available at https://github.com/HaozhiQi/ISONet.

📄 PDF Abstract BibTeX arXiv:2006.16992

Code (1)

HaozhiQi/ISONet 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Bottleneck Residual Block A Bottleneck Residual Block is a variant of the residual block that utilises 1x1 convolutions to create a bottleneck. The…

Batch Normalization 설명 없음

Average Pooling 설명 없음

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Residual Connection 설명 없음

Kaiming Initialization 설명 없음

Mathematical Justification of Hard Negative Mining via Isometric Approximation Theorem

2022-10-20 · Albert Xu, Jhih-Yi Hsieh, Bhaskar Vundurthy, Eliana Cohen 외

In deep metric learning, the Triplet Loss has emerged as a popular method to learn many computer vision and natural language processing tasks such as facial recognition, object detection, and visual-semantic embeddings. …

Contrastive Learning Metric Learning object-detection Object Detection+1

Multi-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulation

2025-08-15 · Hongbin Lin, Juan Rojas, Kwok Wai Samuel Au arxiv

Sampling efficiency is critical for deploying visuomotor learning in real-world robotic manipulation. While task symmetry has emerged as a promising inductive bias to improve efficiency, most prior work is limited to iso…

Reinforcement Learning Robot Manipulation Data Augmentation

Isometric 3D Adversarial Examples in the Physical World

2022-10-27 · Yibo Miao, Yinpeng Dong, Jun Zhu, Xiao-Shan Gao

3D deep learning models are shown to be as vulnerable to adversarial examples as 2D models. However, existing attack methods are still far from stealthy and suffer from severe performance degradation in the physical worl…

Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning

2024-03-20 · Shivam Ratnakant Mhaskar, Nirmesh J. Shah, Mohammadi Zaki, Ashishkumar P. Gudmalwar 외

Traditional Automatic Video Dubbing (AVD) pipeline consists of three key modules, namely, Automatic Speech Recognition (ASR), Neural Machine Translation (NMT), and Text-to-Speech (TTS). Within AVD pipelines, isometric-NM…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Machine Translation NMT+7

Isometric Propagation Network for Generalized Zero-shot Learning

2021-02-03 · ICLR 2021 1 · Lu Liu, Tianyi Zhou, Guodong Long, Jing Jiang 외

Zero-shot learning (ZSL) aims to classify images of an unseen class only based on a few attributes describing that class but no access to any training sample. A popular strategy is to learn a mapping between the semantic…

Generalized Zero-Shot Learning Zero-Shot Learning