Self-Contrastive Forward-Forward Algorithm
Agents that operate autonomously benefit from lifelong learning capabilities. However, compatible training algorithms must comply with the decentralized nature of these systems, which imposes constraints on both the parameter counts and the computational resources. The Forward-Forward (FF) algorithm is one of these. FF relies only on feedforward operations, the same used for inference, for optimizing layer-wise objectives. This purely forward approach eliminates the need for transpose operations required in traditional backpropagation. Despite its potential, FF has failed to reach state-of-the-art performance on most standard benchmark tasks, in part due to unreliable negative data generation methods for unsupervised learning. In this work, we propose the Self-Contrastive Forward-Forward (SCFF) algorithm, a competitive training method aimed at closing this performance gap. Inspired by standard self-supervised contrastive learning for vision tasks, SCFF generates positive and negative inputs applicable across various datasets. The method demonstrates superior performance compared to existing unsupervised local learning algorithms on several benchmark datasets, including MNIST, CIFAR-10, STL-10, and Tiny ImageNet. We extend FF's application to training recurrent neural networks, expanding its utility to sequential data tasks. These findings pave the way for high-accuracy, real-time learning on resource-constrained edge devices.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningLifelong learningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Contrastive Forward-Forward: A Training Algorithm of Vision Transformer
Although backpropagation is widely accepted as a training algorithm for artificial neural networks, researchers are always looking for inspiration from the brain to find ways with potentially better performance. Forward-…
Contrastive Learningimage-classificationImage ClassificationForward-Forward Contrastive Learning
Medical image classification is one of the most important tasks for computer-aided diagnosis. Deep learning models, particularly convolutional neural networks, have been successfully used for disease classification from …
ClassificationContrastive Learningimage-classificationImage Classification+1A Study of Forward-Forward Algorithm for Self-Supervised Learning
Self-supervised representation learning has seen remarkable progress in the last few years, with some of the recent methods being able to learn useful image representations without labels. These methods are trained using…
Representation LearningSelf-Supervised LearningA Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Tasks
The so-called Forward-Forward Algorithm (FFA) has recently gained momentum as an alternative to the conventional back-propagation algorithm for neural network learning, yielding competitive performance across various mod…
Continual Learningimage-classificationImage ClassificationHCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm
Deep neural networks trained with backpropagation have achieved outstanding performance in vision tasks but remain biologically implausible, computationally demanding, and difficult to interpret. The Forward-Forward (FF)…
Contrastive Learning