Model Generalization: A Sharpness Aware Optimization Perspective
Sharpness-Aware Minimization (SAM) and adaptive sharpness-aware minimization (ASAM) aim to improve the model generalization. And in this project, we proposed three experiments to valid their generalization from the sharpness aware perspective. And our experiments show that sharpness aware-based optimization techniques could help to provide models with strong generalization ability. Our experiments also show that ASAM could improve the generalization performance on un-normalized data, but further research is needed to confirm this.
Code (1)
Tasks
modelvalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Gradient constrained sharpness-aware prompt learning for vision-language models
This paper targets a novel trade-off problem in generalizable prompt learning for vision-language models (VLM), i.e., improving the performance on unseen classes while maintaining the performance on seen classes. Compari…
Prompt LearningRSAM: Learning on manifolds with Riemannian Sharpness-aware Minimization
Nowadays, understanding the geometry of the loss landscape shows promise in enhancing a model's generalization ability. In this work, we draw upon prior works that apply geometric principles to optimization and present a…
Contrastive Learningimage-classificationImage ClassificationX-SAM: Boosting Sharpness-Aware Minimization with Dominant-Eigenvector Gradient Correction
Sharpness-Aware Minimization (SAM) aims to improve generalization by minimizing a worst-case perturbed loss over a small neighborhood of model parameters. However, during training, its optimization behavior does not alwa…
1st-Order Magic: Analysis of Sharpness-Aware Minimization
Sharpness-Aware Minimization (SAM) is an optimization technique designed to improve generalization by favoring flatter loss minima. To achieve this, SAM optimizes a modified objective that penalizes sharpness, using comp…
COSCO: A Sharpness-Aware Training Framework for Few-shot Multivariate Time Series Classification
Multivariate time series classification is an important task with widespread domains of applications. Recently, deep neural networks (DNN) have achieved state-of-the-art performance in time series classification. However…
ClassificationTime SeriesTime Series Classification