Robustness and Regularization in Hierarchical Re-Basin
This paper takes a closer look at Git Re-Basin, an interesting new approach to merge trained models. We propose a hierarchical model merging scheme that significantly outperforms the standard MergeMany algorithm. With our new algorithm, we find that Re-Basin induces adversarial and perturbation robustness into the merged models, with the effect becoming stronger the more models participate in the hierarchical merging scheme. However, in our experiments Re-Basin induces a much bigger performance drop than reported by the original authors.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Image Segmentation by Size-Dependent Single Linkage Clustering of a Watershed Basin Graph
We present a method for hierarchical image segmentation that defines a disaffinity graph on the image, over-segments it into watershed basins, defines a new graph on the basins, and then merges basins with a modified, si…
ClusteringImage SegmentationSemantic SegmentationTo Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning
Transfer learning and ensembling are two popular techniques for improving the performance and robustness of neural networks. Due to the high cost of pre-training, ensembles of models fine-tuned from a single pre-trained …
DiversityTransfer LearningUpper bound for the stability of Boolean networks
Boolean networks, inspired by gene regulatory networks, were developed to understand the complex behaviors observed in biological systems, with network attractors corresponding to biological phenotypes or cell types. In …
The conjugated null space method of blind PSF estimation and deconvolution optimization
We have shown that the vector of the point spread function (PSF) lexicographical presentation belongs to the left side conjugated null space (NS) of the autoregression (AR) matrix operator on condition the AR parameters …
Denoisingimage smoothingProbabilistic Inverse Modeling: An Application in Hydrology
The astounding success of these methods has made it imperative to obtain more explainable and trustworthy estimates from these models. In hydrology, basin characteristics can be noisy or missing, impacting streamflow pre…