홈 › Papers

TAS: Ternarized Neural Architecture Search for Resource-Constrained Edge Devices

2022-03-14 · Design, Automation and Test in Europe Conference (DATE) 2022 3 · Mohammad Loni, Hamid Mousavi, Mohammad Riazati, Masoud Daneshtalab, and Mikael Sjodin

Ternary Neural Networks (TNNs) compress network weights and activation functions into 2-bit representation resulting in remarkable network compression and energy efficiency. However, there remains a significant gap in accuracy between TNNs and full-precision counterparts. Recent advances in Neural Architectures Search (NAS) promise opportunities in automated optimization for various deep learning tasks. Unfortunately, this area is unexplored for optimizing TNNs. This paper proposes TAS, a framework that drastically reduces the accuracy gap between TNNs and their full-precision counterparts by integrating quantization into the network design. We experienced that directly applying NAS to the ternary domain provides accuracy degradation as the search settings are customized for full-precision networks. To address this problem, we propose (i) a new cell template for ternary networks with maximum gradient propagation; and (ii) a novel learnable quantizer that adaptively relaxes the ternarization mechanism from the distribution of the weights and activation functions. Experimental results reveal that TAS delivers 2.64% higher accuracy and ≈2.8× memory saving over competing methods with the same bit-width resolution on the CIFAR-10 dataset. These results suggest that TAS is an effective method that paves the way for the efficient design of the next generation of quantized neural networks.

📄 PDF Abstract BibTeX

Code (1)

HERO-MDH/TAS 공식 구현 pytorch

Tasks

Neural Architecture Search Quantization

FTerViT: Fully Ternary Vision Transformer

2026-05-20 · Szymon Ruciński, Pietro Bonazzi, Engin Türetken, Simon Narduzzi 외 arxiv

Ternary Vision Transformers offer substantial model compression, however state-of-the-art methods only ternarize the encoder layers, leaving patch embeddings, LayerNorm parameters, and classifier heads in full precision.…

Knowledge Distillation Model Compression

Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation

2018-10-02 · CVPR 2019 6 · Zhezhi He, Deliang Fan

In the past years, Deep convolution neural network has achieved great success in many artificial intelligence applications. However, its enormous model size and massive computation cost have become the main obstacle for …

Efficient Ternary Weight Embedding Model: Bridging Scalability and Performance

2024-11-23 · Jiayi Chen, Chen Wu, ShaoQun Zhang, Nan Li 외

Embedding models have become essential tools in both natural language processing and computer vision, enabling efficient semantic search, recommendation, clustering, and more. However, the high memory and computational d…

Computational Efficiency Knowledge Distillation Recommendation Systems

Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy

2018-07-20 · Zhezhi He, Boqing Gong, Deliang Fan

Deep convolution neural network has achieved great success in many artificial intelligence applications. However, its enormous model size and massive computation cost have become the main obstacle for deployment of such …

Model Compression Vocal Bursts Intensity Prediction

TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge

2025-10-23 · Shu-Hao Zhang, Wei-Cheng Tang, Chen Wu, Peng Hu 외 arxiv

Recent years have witnessed an increasing interest in image-text contrastive modeling, exemplified by models such as Contrastive Language-Image Pretraining (CLIP). In this paper, we propose the TernaryCLIP, a lightweight…

Zero-Shot Image Classification Text Retrieval