Empirical investigations on WVA structural issues
In this paper we want to present the results of empirical verification of some issues concerning the methods for overcoming catastrophic forgetting in neural networks. First, in the introduction, we will try to describe in detail the problem of catastrophic forgetting and methods for overcoming it for those who are not yet familiar with this topic. Then we will discuss the essence and limitations of the WVA method which we presented in previous papers. Further, we will touch upon the issues of applying the WVA method to gradients or optimization steps of weights, choosing the optimal attenuation function in this method, as well as choosing the optimal hyper-parameters of the method depending on the number of tasks in sequential training of neural networks.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Do More Negative Samples Necessarily Hurt in Contrastive Learning?
Recent investigations in noise contrastive estimation suggest, both empirically as well as theoretically, that while having more "negative samples" in the contrastive loss improves downstream classification performance i…
Contrastive LearningInvestigation of condominium building collapse in Surfside, Florida: A video feature tracking approach
On June 24, 2021, a 12-story condominium building (Champlain Towers South) in Surfside, Florida partially collapsed, resulting in one of the deadliest building collapses in United States history with 98 people confirmed …
SoK: Cross-border Criminal Investigations and Digital Evidence
Digital evidence underpin the majority of crimes as their analysis is an integral part of almost every criminal investigation. Even if we temporarily disregard the numerous challenges in the collection and analysis of di…
Hyperparameter Optimization Can Even be Harmful in Off-Policy Learning and How to Deal with It
There has been a growing interest in off-policy evaluation in the literature such as recommender systems and personalized medicine. We have so far seen significant progress in developing estimators aimed at accurately es…
counterfactualDecision MakingHyperparameter OptimizationOff-policy evaluation+1Decoy Selection for Protein Structure Prediction Via Extreme Gradient Boosting and Ranking
Identifying one or more biologically-active/native decoys from millions of non-native decoys is one of the major challenges in computational structural biology. The extreme lack of balance in positive and negative sample…
BIG-bench Machine LearningClusteringProtein Structure Prediction