Performance Guaranteed Network Acceleration via High-Order Residual Quantization
Input binarization has shown to be an effective way for network acceleration. However, previous binarization scheme could be regarded as simple pixel-wise thresholding operations (i.e., order-one approximation) and suffers a big accuracy loss. In this paper, we propose a highorder binarization scheme, which achieves more accurate approximation while still possesses the advantage of binary operation. In particular, the proposed scheme recursively performs residual quantization and yields a series of binary input images with decreasing magnitude scales. Accordingly, we propose high-order binary filtering and gradient propagation operations for both forward and backward computations. Theoretical analysis shows approximation error guarantee property of proposed method. Extensive experimental results demonstrate that the proposed scheme yields great recognition accuracy while being accelerated.
Code (0)
등록된 구현이 없습니다.
Tasks
BinarizationQuantizationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
OrderDP: A Theoretically Guaranteed Lossless Dynamic Data Pruning Framework
Data pruning (DP), as an oft-stated strategy to alleviate heavy training burdens, reduces the volume of training samples according to a well-defined pruning method while striving for near-lossless performance. However, e…
Computational EfficiencyPhysics-Informed Neural Networks for Satellite State Estimation
The Space Domain Awareness (SDA) community routinely tracks satellites in orbit by fitting an orbital state to observations made by the Space Surveillance Network (SSN). In order to fit such orbits, an accurate model of …
State EstimationAnderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis
Anderson acceleration (AA) is widely used for accelerating the convergence of nonlinear fixed-point methods $x_{k+1}=q(x_{k})$, $x_k \in \mathbb{R}^n$, but little is known about how to quantify the convergence accelerati…
Multi-Weight Respecification of Scan-specific Learning for Parallel Imaging
Parallel imaging is widely used in magnetic resonance imaging as an acceleration technology. Traditional linear reconstruction methods in parallel imaging often suffer from noise amplification. Recently, a non-linear rob…
Momentum as Residual-Driven Multiplier Correction for Deep Learning Optimization
Momentum-based optimizers are widely used in modern deep learning, yet the relations among momentum recursion, update geometry, and acceleration remain only partially understood. We develop an $\textbf{A}$DMM-$\textbf{I}…
Reinforcement Learning