Local Stability and Gaussian Smoothing of Quantized Neural Networks
We study Gaussian averaging as a smooth surrogate for quantized neural models. Under bounded local oscillation, we derive a local dimension-dependent bound on |f-g|, linking Gaussian smoothing to the stability analysis of discontinuous networks. We compute closed-form Gaussian averages of the rectified linear unit (ReLU) and sign activation functions, and illustrate the mechanism on a high-dimensional binary perceptron, where layer-preactivation aggregation under an explicit quantization-noise surrogate yields the Gaussian envelope used in inference-side smoothing and training-side smooth surrogate gradients.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
LOTION: Smoothing the Optimization Landscape for Quantized Training
Optimizing neural networks for quantized objectives is fundamentally challenging because the quantizer is piece-wise constant, yielding zero gradients everywhere except at quantization thresholds where the derivative is …
Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations
Quantization of Convolutional Neural Networks (CNNs) is a common approach to ease the computational burden involved in the deployment of CNNs, especially on low-resource edge devices. However, fixed-point arithmetic is n…
Autonomous Drivingimage-classificationImage ClassificationNode Classification+1Taylor Moment Expansion for Continuous-Discrete Gaussian Filtering and Smoothing
The paper is concerned with non-linear Gaussian filtering and smoothing in continuous-discrete state-space models, where the dynamic model is formulated as an It\^{o} stochastic differential equation (SDE), and the measu…
State Space ModelsA Novel Evolution Strategy with Directional Gaussian Smoothing for Blackbox Optimization
We propose an improved evolution strategy (ES) using a novel nonlocal gradient operator for high-dimensional black-box optimization. Standard ES methods with $d$-dimensional Gaussian smoothing suffer from the curse of di…
global-optimizationTraining for Trustworthy Saliency Maps: Adversarial Training Meets Feature-Map Smoothing
Gradient-based saliency methods such as Vanilla Gradient (VG) and Integrated Gradients (IG) are widely used to explain image classifiers, yet the resulting maps are often noisy and unstable, limiting their usefulness in …