Adapting Resilient Propagation for Deep Learning
The Resilient Propagation (Rprop) algorithm has been very popular for backpropagation training of multilayer feed-forward neural networks in various applications. The standard Rprop however encounters difficulties in the context of deep neural networks as typically happens with gradient-based learning algorithms. In this paper, we propose a modification of the Rprop that combines standard Rprop steps with a special drop out technique. We apply the method for training Deep Neural Networks as standalone components and in ensemble formulations. Results on the MNIST dataset show that the proposed modification alleviates standard Rprop's problems demonstrating improved learning speed and accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Formally Guaranteed Control Adaptation for ODD-Resilient Autonomous Systems
Ensuring reliable performance in situations outside the Operational Design Domain (ODD) remains a primary challenge in devising resilient autonomous systems. We explore this challenge by introducing an approach for adapt…
Artificial Neural Network Approach for the Identification of Clove Buds Origin Based on Metabolites Composition
This paper examines the use of artificial neural network approach in identifying the origin of clove buds based on metabolites composition. Generally, large data sets are critical for accurate identification. Machine lea…
Cascade-Forward Neural Network Based on Resilient Backpropagation for Simultaneous Parameters and State Space Estimations of Brushed DC Machines
A sensorless speed, average temperature and resistance estimation technique based on Neural Network (NN) for brushed DC machines is proposed in this paper. The literature on parameters and state spaces estimations of the…
Bach in 2014: Music Composition with Recurrent Neural Network
We propose a framework for computer music composition that uses resilient propagation (RProp) and long short term memory (LSTM) recurrent neural network. In this paper, we show that LSTM network learns the structure and …
MemSAM: Taming Segment Anything Model for Echocardiography Video Segmentation
We propose a novel echocardiographical video segmentation model by adapting SAM to medical videos to address some long-standing challenges in ultrasound video segmentation including (1) massive speckle noise and arti…
SegmentationVideo SegmentationVideo Semantic Segmentation