Compete to Compute
Local competition among neighboring neurons is common in biological neural networks (NNs). We apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Who Reviews The Reviewers? A Multi-Level Jury Problem
We consider the problem of determining a binary ground truth using advice from a group of independent reviewers (experts) who express their guess about a ground truth correctly with some independent probability (competen…
From Learning Resources to Competencies: LLM-Based Tagging with Evidence and Graph Constraints
Linking learning resources to a structured competency framework is key to enabling competency-based search and curriculum analytics in Learning Management Systems (LMS). However, manual tagging is labor-intensive, and fu…
META-DES.H: a dynamic ensemble selection technique using meta-learning and a dynamic weighting approach
In Dynamic Ensemble Selection (DES) techniques, only the most competent classifiers are selected to classify a given query sample. Hence, the key issue in DES is how to estimate the competence of each classifier in a poo…
Meta-LearningBlack-box Testing of First-Order Logic Ontologies Using WordNet
Artificial Intelligence aims to provide computer programs with commonsense knowledge to reason about our world. This paper offers a new practical approach towards automated commonsense reasoning with first-order logic (F…
Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion
Adapting large language models (LLMs) to low-resource languages (LRLs) is constrained by the scarcity of task data and computational resources. Although Proxy Tuning offers a logit-level strategy for introducing scaling …
Continual Pretraining