Post-Training BatchNorm Recalibration

2020-10-12 · Gil Shomron, Uri Weiser

We revisit non-blocking simultaneous multithreading (NB-SMT) introduced previously by Shomron and Weiser (2020). NB-SMT trades accuracy for performance by occasionally "squeezing" more than one thread into a shared multiply-and-accumulate (MAC) unit. However, the method of accommodating more than one thread in a shared MAC unit may contribute noise to the computations, thereby changing the internal statistics of the model. We show that substantial model performance can be recouped by post-training recalibration of the batch normalization layers' running mean and running variance statistics, given the presence of NB-SMT.

📄 PDF Abstract BibTeX arXiv:2010.05625

Code (0)

등록된 구현이 없습니다.

Tasks

Blocking

Methods 이 논문이 사용한 방법론

Batch Normalization 설명 없음

Adaptive Signal Resuscitation: Channel-wise Post-Pruning Repair for Sparse Vision Networks

2026-05-20 · Qishi Zhan, Ziheng Chen, Minxuan Hu arxiv

One-shot magnitude pruning can cause severe accuracy collapse in the high-sparsity regime, even when the pruning mask preserves the largest weights. We argue that this failure reflects a granularity mismatch in post-prun…

How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability

2026-05-21 · Qishi Zhan, Minxuan Hu, Liang He arxiv

Unstructured magnitude pruning at high sparsity can reduce neural network accuracy to near-random performance, while labeled retraining may be unavailable in practical deployment settings. Label-free post-pruning repair …

Test Time Adaptation Using Adaptive Quantile Recalibration

2025-11-05 · Paria Mehrbod, Pedro Vianna, Geraldin Nanfack, Guy Wolf 외 arxiv

Domain adaptation is a key strategy for enhancing the generalizability of deep learning models in real-world scenarios, where test distributions often diverge significantly from the training domain. However, conventional…

Test-time Adaptation Domain Adaptation

An Empirical Analysis of the Shift and Scale Parameters in BatchNorm

2023-03-22 · Yashna Peerthum, Mark Stamp

Batch Normalization (BatchNorm) is a technique that improves the training of deep neural networks, especially Convolutional Neural Networks (CNN). It has been empirically demonstrated that BatchNorm increases performance…

Training BatchNorm Only in Neural Architecture Search and Beyond

2021-12-01 · Yichen Zhu, Jie Du, Yuqin Zhu, Yi Wang 외

This work investigates the usage of batch normalization in neural architecture search (NAS). Specifically, Frankle et al. find that training BatchNorm only can achieve nontrivial performance. Furthermore, Chen et al. cla…

Fairness Neural Architecture Search