Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment
The Forward-Forward (FF) algorithm offers a biologically inspired alternative to backpropagation by replacing gradient-based credit assignment with local, forward-only objectives. While recent extensions have adapted FF to convolutional neural networks (CNNs), existing formulations rely on static channel-class partitions and struggle to perform effectively in complex tasks. In this work, we introduce a learnable channel-class assignment mechanism that enables adaptive, data-driven specialization of convolutional channels, supported by entropy and orthogonality regularization to promote learning performance. We further propose a loss-aware layer contribution strategy that adaptively weights intermediate-layer predictions based on their validation performance, enhancing the effectiveness of forward-only inference. Integrated into residual CNNs, the proposed method achieves consistently superior performance across CIFAR-10, CIFAR-100, and Tiny-ImageNet compared to existing similar forward-only methods. Notably, it establishes new state-of-the-art performance among FF-based models, substantially narrowing the gap with backpropagation. These findings demonstrate that introducing learnable channel specialization and layer contribution weighting significantly enhances the representational capacity of forward-only learning in deep CNNs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm
The Forward-Forward (FF) Algorithm has been recently proposed to alleviate the issues of backpropagation (BP) commonly used to train deep neural networks. However, its current formulation exhibits limitations such as the…
image-classificationImage ClassificationChannel Compression: Rethinking Information Redundancy among Channels in CNN Architecture
Model compression and acceleration are attracting increasing attentions due to the demand for embedded devices and mobile applications. Research on efficient convolutional neural networks (CNNs) aims at removing feature …
Acoustic Scene ClassificationEvent Detectionimage-classificationImage Classification+3LightViT: Towards Light-Weight Convolution-Free Vision Transformers
Vision transformers (ViTs) are usually considered to be less light-weight than convolutional neural networks (CNNs) due to the lack of inductive bias. Recent works thus resort to convolutions as a plug-and-play module an…
GPUimage-classificationImage ClassificationInductive Bias+3Dynamic Shuffle: An Efficient Channel Mixture Method
The redundancy of Convolutional neural networks not only depends on weights but also depends on inputs. Shuffling is an efficient operation for mixing channel information but the shuffle order is usually pre-defined. To …
Binarizationimage-classificationImage ClassificationCompressing Quaternion Convolutional Neural Networks for Audio Classification
Conventional Convolutional Neural Networks (CNNs) in the real domain have been widely used for audio classification. However, their convolution operations process multi-channel inputs independently, limiting the ability …
Environmental Sound ClassificationSpeech Emotion RecognitionMusic Genre RecognitionKnowledge Distillation