Improving Neural Networks by Adopting Amplifying and Attenuating Neurons
In the present study, an amplifying neuron and attenuating neuron, which can be easily implemented into neural networks without any significant additional computational effort, are proposed. The activated output value is squared for the amplifying neuron, while the value becomes its reciprocal for the attenuating one. Theoretically, the order of neural networks increases when the amplifying neuron is placed in the hidden layer. The performance assessments of neural networks were conducted to verify that the amplifying and attenuating neurons enhance the performance of neural networks. From the numerical experiments, it was revealed that the neural networks that contain the amplifying and attenuating neurons yield more accurate results, compared to those without them.
Code (1)
Similar Papers 제목 키워드 기반
Attenuating Bias in Word Vectors
Word vector representations are well developed tools for various NLP and Machine Learning tasks and are known to retain significant semantic and syntactic structure of languages. But they are prone to carrying and amplif…
Unveiling the Influence of Amplifying Language-Specific Neurons
Language-specific neurons in LLMs that strongly correlate with individual languages have been shown to influence model behavior by deactivating them. However, their role in amplification remains underexplored. This work …
Cross-Lingual TransferActive Speech Enhancement: Active Speech Denoising Decliping and Deveraberation
We introduce a new paradigm for active sound modification: Active Speech Enhancement (ASE). While Active Noise Cancellation (ANC) algorithms focus on suppressing external interference, ASE goes further by actively shapin…
DenoisingMambaSpeech DenoisingSpeech EnhancementRealistic Saliency Guided Image Enhancement
Common editing operations performed by professional photographers include the cleanup operations: de-emphasizing distracting elements and enhancing subjects. These edits are challenging, requiring a delicate balance betw…
Image EnhancementAdversarial Robustness in Deep Learning: Attacks on Fragile Neurons
We identify fragile and robust neurons of deep learning architectures using nodal dropouts of the first convolutional layer. Using an adversarial targeting algorithm, we correlate these neurons with the distribution of a…
Adversarial AttackAdversarial RobustnessDeep Learningimage-classification+1