AdaResNet: Enhancing Residual Networks with Dynamic Weight Adjustment for Improved Feature Integration
In very deep neural networks, gradients can become extremely small during backpropagation, making it challenging to train the early layers. ResNet (Residual Network) addresses this issue by enabling gradients to flow directly through the network via skip connections, facilitating the training of much deeper networks. However, in these skip connections, the input ipd is directly added to the transformed data tfd, treating ipd and tfd equally, without adapting to different scenarios. In this paper, we propose AdaResNet (Auto-Adapting Residual Network), which automatically adjusts the ratio between ipd and tfd based on the training data. We introduce a variable, weight}_{tfd}^{ipd, to represent this ratio. This variable is dynamically adjusted during backpropagation, allowing it to adapt to the training data rather than remaining fixed. Experimental results demonstrate that AdaResNet achieves a maximum accuracy improvement of over 50\% compared to traditional ResNet.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting
We investigate recently introduced domain-class incremental learning scenarios for vision-language models (VLMs). Recent works address this challenge using parameter-efficient methods, such as prefix-tuning or adapters, …
Class Incremental LearningContinual LearningResidual-Quantile Adjustment for Adaptive Training of Physics-informed Neural Network
Adaptive training methods for Physics-informed neural network (PINN) require dedicated constructions of the distribution of weights assigned at each training sample. To efficiently seek such an optimal weight distributio…
Adaptive boosting with dynamic weight adjustment
Adaptive Boosting with Dynamic Weight Adjustment is an enhancement of the traditional Adaptive boosting commonly known as AdaBoost, a powerful ensemble learning technique. Adaptive Boosting with Dynamic Weight Adjustment…
Ensemble LearningLightweight Improved Residual Network for Efficient Inverse Tone Mapping
The display devices like HDR10 televisions are increasingly prevalent in our daily life for visualizing high dynamic range (HDR) images. But the majority of media images on the internet remain in 8-bit standard dynamic r…
Image Reconstructioninverse tone mappingInverse-Tone-MappingTone MappingRAMBA: 4D Radar Mapping by Bundle Adjustment
4D radar is increasingly attractive for robotic mapping because it provides range, azimuth, elevation, and Doppler measurements while remaining robust in adverse visibility conditions. Although recent radar and radar--in…