Analytical Guarantees on Numerical Precision of Deep Neural Networks
The acclaimed successes of neural networks often overshadow their tremendous complexity. We focus on numerical precision – a key parameter defining the complexity of neural networks. First, we present theoretical bounds on the accuracy in presence of limited precision. Interestingly, these bounds can be computed via the back-propagation algorithm. Hence, by combining our theoretical analysis and the back-propagation algorithm, we are able to readily determine the minimum precision needed to preserve accuracy without having to resort to time-consuming fixed-point simulations. We provide numerical evidence showing how our approach allows us to maintain high accuracy but with lower complexity than state-of-the-art binary networks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Petri Net Description of Biological Neural Circuits for Fast Hardware Prototyping
Current approaches to simulating biological neural circuits, whether on general-purpose hardware or dedicated neuromorphic platforms, remain constrained by fixed-timestep numerical integration, hardware-imposed precision…
SEADA: An efficient methodology for optimizing mixed-precision DNNs on multi-precision spatial architectures
Mixed-precision computation has been introduced in deep neural networks (DNNs) as an effective approach to reduce latency, energy consumption, and memory footprint. However, efficiently mapping mixed-precision networks o…
Comparison of Distal Teacher Learning with Numerical and Analytical Methods to Solve Inverse Kinematics for Rigid-Body Mechanisms
Several publications are concerned with learning inverse kinematics, however, their evaluation is often limited and none of the proposed methods is of practical relevance for rigid-body kinematics with a known forward mo…
Virtual Image Correlation uncertainty
The Virtual Image Correlation method applies for the measurement of silhouettes boundaries with sub-pixel precision. It consists in a correlation between the image of interest and a virtual image based on a parametrized …
Towards a Calculus of Echo State Networks
Reservoir computing is a recent trend in neural networks which uses the dynamical perturbations on the phase space of a system to compute a desired target function. We present how one can formulate an expectation of syst…