TOCO: A Framework for Compressing Neural Network Models Based on Tolerance Analysis
Neural network compression methods have enabled deploying large models on emerging edge devices with little cost, by adapting already-trained models to the constraints of these devices. The rapid development of AI-capable edge devices with limited computation and storage requires streamlined methodologies that can efficiently satisfy the constraints of different devices. In contrast, existing methods often rely on heuristic and manual adjustments to maintain accuracy, support only coarse compression policies, or target specific device constraints that limit their applicability. We address these limitations by proposing the TOlerance-based COmpression (TOCO) framework. TOCO uses an in-depth analysis of the model, to maintain the accuracy, in an active learning system. The results of the analysis are tolerances that can be used to perform compression in a fine-grained manner. Finally, by decoupling compression from the tolerance analysis, TOCO allows flexibility to changes in the hardware.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningNeural Network CompressionSimilar Papers 제목 키워드 기반
Once-for-All Sequence Compression for Self-Supervised Speech Models
The sequence length along the time axis is often the dominant factor of the computation in speech processing. Works have been proposed to reduce the sequence length for lowering the computational cost in self-supervised …
AllAI-enabled Blockchain: An Outlier-aware Consensus Protocol for Blockchain-based IoT Networks
A new framework for a secure and robust consensus in blockchain-based IoT networks is proposed using machine learning. Hyperledger fabric, which is a blockchain platform developed as part of the Hyperledger project, thou…
BIG-bench Machine LearningOutlier DetectionA Language for Modeling And Optimizing Experimental Biological Protocols
Automation is becoming ubiquitous in all laboratory activities, leading towards precisely defined and codified laboratory protocols. However, the integration between laboratory protocols and mathematical models is still …
FAIR-Pruner: A Flexible Framework for Automatic Layer-Wise Pruning via Tolerance of Difference
Structured pruning is a standard tool for compressing deep neural networks, but its practical performance depends on how sparsity is allocated across layers. We propose FAIR-Pruner, a search-free framework for adaptive l…
Measuring the Checker: Mutation Analysis for GPU-Kernel Benchmark Oracles
Benchmarks for LLM-generated GPU kernels decide correctness with a few random inputs and a loose floating-point tolerance, and their verdicts now feed leaderboards and reinforcement-learning rewards. Recent work agrees t…