Papers Permuted-MNIST
“Permuted-MNIST” 태그가 달린 논문 37편 · 필터 해제
Preserving Plasticity in Continual Learning with Adaptive Linearity Injection
Loss of plasticity in deep neural networks is the gradual reduction in a model's capacity to incrementally learn and has been identified as a key obstacle to learning in non-stationary problem settings. Recent work has s…
class-incremental learningClass Incremental LearningContinual LearningIncremental Learning+1Bayesian continual learning and forgetting in neural networks
Biological synapses effortlessly balance memory retention and flexibility, yet artificial neural networks still struggle with the extremes of catastrophic forgetting and catastrophic remembering. Here, we introduce Metap…
Bayesian InferenceContinual Learningimage-classificationImage Classification+2Exploiting Task Relationships for Continual Learning Using Transferability-Aware Task Embeddings
Continual learning (CL) has been an essential topic in the contemporary application of deep neural networks, where catastrophic forgetting (CF) can impede a model's ability to acquire knowledge progressively. Existing CL…
Continual LearningPermuted-MNISTDisentangling and Mitigating the Impact of Task Similarity for Continual Learning
Continual learning of partially similar tasks poses a challenge for artificial neural networks, as task similarity presents both an opportunity for knowledge transfer and a risk of interference and catastrophic forgettin…
Continual LearningPermuted-MNISTTransfer LearningBayesian Metaplasticity from Synaptic Uncertainty
Catastrophic forgetting remains a challenge for neural networks, especially in lifelong learning scenarios. In this study, we introduce MEtaplasticity from Synaptic Uncertainty (MESU), inspired by metaplasticity and Baye…
Bayesian InferenceContinual LearningLifelong learningPermuted-MNISTArtificial Neuronal Ensembles with Learned Context Dependent Gating
Biological neural networks are capable of recruiting different sets of neurons to encode different memories. However, when training artificial neural networks on a set of tasks, typically, no mechanism is employed for se…
Continual LearningPermuted-MNISTRotated MNISTAnalysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regime
Overparameterization is known to permit strong generalization performance in neural networks. In this work, we provide an initial theoretical analysis of its effect on catastrophic forgetting in a continual learning setu…
Continual Learningimage-classificationImage ClassificationPermuted-MNISTIncreasing Depth of Neural Networks for Life-long Learning
Purpose: We propose a novel method for continual learning based on the increasing depth of neural networks. This work explores whether extending neural network depth may be beneficial in a life-long learning setting. Met…
Continual LearningPermuted-MNISTThe CLEAR Benchmark: Continual LEArning on Real-World Imagery
Continual learning (CL) is widely regarded as crucial challenge for lifelong AI. However, existing CL benchmarks, e.g. Permuted-MNIST and Split-CIFAR, make use of artificial temporal variation and do not align with or ge…
Continual Learningimage-classificationImage ClassificationPermuted-MNIST+1Mixture-of-Variational-Experts for Continual Learning
One weakness of machine learning algorithms is the poor ability of models to solve new problems without forgetting previously acquired knowledge. The Continual Learning (CL) paradigm has emerged as a protocol to systemat…
Continual LearningDomain-IL Continual LearningPermuted-MNISTreinforcement-learning+4Continual Learning with Memory Cascades
Continual learning poses an important challenge to machine learning models. Kirkpatrick et al. introduced a model that combats forgetting during continual learning by using a Bayesian prior to transfer knowledge between …
Continual LearningPermuted-MNISTShared and Private VAEs with Generative Replay for Continual Learning
Continual learning tries to learn new tasks without forgetting previously learned ones. In reality, most of the existing artificial neural network(ANN) models fail, while humans do the same by remembering previous works …
Continual LearningPermuted-MNISTLifelong Learning with Sketched Structural Regularization
Preventing catastrophic forgetting while continually learning new tasks is an essential problem in lifelong learning. Structural regularization (SR) refers to a family of algorithms that mitigate catastrophic forgetting …
Continual LearningLifelong learningPermuted-MNISTCKConv: Continuous Kernel Convolution For Sequential Data
Conventional neural architectures for sequential data present important limitations. Recurrent networks suffer from exploding and vanishing gradients, small effective memory horizons, and must be trained sequentially. Co…
Permuted-MNISTSequential Image ClassificationTime Series AnalysisShort-Term Memory Optimization in Recurrent Neural Networks by Autoencoder-based Initialization
Training RNNs to learn long-term dependencies is difficult due to vanishing gradients. We explore an alternative solution based on explicit memorization using linear autoencoders for sequences, which allows to maximize t…
MemorizationPermuted-MNISTHiPPO: Recurrent Memory with Optimal Polynomial Projections
A central problem in learning from sequential data is representing cumulative history in an incremental fashion as more data is processed. We introduce a general framework (HiPPO) for the online compression of continuous…
Permuted-MNISTSequential Image ClassificationTime SeriesTime Series AnalysisEnabling Continual Learning with Differentiable Hebbian Plasticity
Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge. However, catastrophic forgetting poses a grand challenge for neural networks performing su…
Continual LearningPermuted-MNISTSplit-MNISTMultilayer Neuromodulated Architectures for Memory-Constrained Online Continual Learning
We focus on the problem of how to achieve online continual learning under memory-constrained conditions where the input data may not be known \emph{a priori}. These constraints are relevant in edge computing scenarios. W…
Bayesian Optimizationclass-incremental learningClass Incremental LearningContinual Learning+4Continual Learning with Extended Kronecker-factored Approximate Curvature
We propose a quadratic penalty method for continual learning of neural networks that contain batch normalization (BN) layers. The Hessian of a loss function represents the curvature of the quadratic penalty function, and…
Continual LearningGeneral ClassificationPermuted-MNISTvalidAutoencoder-based Initialization for Recurrent Neural Networks with a Linear Memory
Orthogonal recurrent neural networks address the vanishing gradient problem by parameterizing the recurrent connections using an orthogonal matrix. This class of models is particularly effective to solve tasks that requi…
MemorizationPermuted-MNIST