Hypernetwork-Based Augmentation

2020-06-11 · Chih-Yang Chen, Che-Han Chang

Data augmentation is an effective technique to improve the generalization of deep neural networks. Recently, AutoAugment proposed a well-designed search space and a search algorithm that automatically finds augmentation policies in a data-driven manner. However, AutoAugment is computationally intensive. In this paper, we propose an efficient gradient-based search algorithm, called Hypernetwork-Based Augmentation (HBA), which simultaneously learns model parameters and augmentation hyperparameters in a single training. Our HBA uses a hypernetwork to approximate a population-based training algorithm, which enables us to tune augmentation hyperparameters by gradient descent. Besides, we introduce a weight sharing strategy that simplifies our hypernetwork architecture and speeds up our search algorithm. We conduct experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet. Our results show that HBA is competitive to the state-of-the-art methods in terms of both search speed and accuracy.

📄 PDF Abstract BibTeX arXiv:2006.06320

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

HyperNetwork A HyperNetwork is a network that generates weights for a main network. The behavior of the main network is the same with any usual neural network: it learns to map some raw…

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

AutoAugment 설명 없음

Language Modeling with Recurrent Highway Hypernetworks

2017-12-01 · NeurIPS 2017 12 · Joseph Suarez

We present extensive experimental and theoretical support for the efficacy of recurrent highway networks (RHNs) and recurrent hypernetworks complimentary to the original works. Where the original RHN work primarily provi…

Language Modeling Language Modelling

Neural Augmentation of Kalman Filter with Hypernetwork for Channel Tracking

2021-09-26 · Kumar Pratik, Rana Ali Amjad, Arash Behboodi, Joseph B. Soriaga 외

We propose Hypernetwork Kalman Filter (HKF) for tracking applications with multiple different dynamics. The HKF combines generalization power of Kalman filters with expressive power of neural networks. Instead of keeping…

Hypernetwork Dismantling via Deep Reinforcement Learning

2021-04-29 · Dengcheng Yan, Wenxin Xie, Yiwen Zhang, Qiang He 외

Network dismantling aims to degrade the connectivity of a network by removing an optimal set of nodes. It has been widely adopted in many real-world applications such as epidemic control and rumor containment. However, c…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

HyperNetworks

2016-09-27 · David Ha, Andrew Dai, Quoc V. Le

This work explores hypernetworks: an approach of using a one network, also known as a hypernetwork, to generate the weights for another network. Hypernetworks provide an abstraction that is similar to what is found in na…

Handwriting generation Language Modelling Machine Translation Translation

Scaling Laws for Hypernetwork-Based Knowledge Injection in Large Language Models

2026-07-21 · Nischay Dhankhar, Dos Baha, Abulhair Saparov arxiv

Injecting factual knowledge into large language models (LLMs) reliably and at scale remains an open challenge. Hypernetworks provide a promising solution to large-scale knowledge injection. Although hypernetworks are typ…

Test-time Adaptation