Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback
Recent years have witnessed the great success of convolutional neural network (CNN) based models in the field of computer vision. CNN is able to learn hierarchically abstracted features from images in an end-to-end training manner. However, most of the existing CNN models only learn features through a feedforward structure and no feedback information from top to bottom layers is exploited to enable the networks to refine themselves. In this paper, we propose a "Learning with Rethinking" algorithm. By adding a feedback layer and producing the emphasis vector, the model is able to recurrently boost the performance based on previous prediction. Particularly, it can be employed to boost any pre-trained models. This algorithm is tested on four object classification benchmark datasets: CIFAR-100, CIFAR-10, MNIST-background-image and ILSVRC-2012 dataset. These results have demonstrated the advantage of training CNN models with the proposed feedback mechanism.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Recurrent Stacking of Layers in Neural Networks: An Application to Neural Machine Translation
In deep neural network modeling, the most common practice is to stack a number of recurrent, convolutional, or feed-forward layers in order to obtain high-quality continuous space representations which in turn improves t…
Knowledge DistillationMachine TranslationNMTTransfer Learning+1Recurrent Soft Attention Model for Common Object Recognition
We propose the Recurrent Soft Attention Model, which integrates the visual attention from the original image to a LSTM memory cell through a down-sample network. The model recurrently transmits visual attention to the me…
modelObjectobject-detectionObject Detection+1Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition
Inspired by "predictive coding" - a theory in neuroscience, we develop a bi-directional and dynamic neural network with local recurrent processing, namely predictive coding network (PCN). Unlike feedforward-only convolut…
image-classificationImage ClassificationObject RecognitionPredictionFBNet: Feedback Network for Point Cloud Completion
The rapid development of point cloud learning has driven point cloud completion into a new era. However, the information flows of most existing completion methods are solely feedforward, and high-level information is rar…
Point Cloud CompletionFAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning
Our proposed system FAMULUS helps students learn to diagnose based on automatic feedback in virtual patient simulations, and it supports instructors in labeling training data. Diagnosing is an exceptionally difficult ski…
DiagnosticMultiple-choice