Full-Precision Free Binary Graph Neural Networks
Binary neural networks have become a promising research topic due to their fast inference speed and low energy consumption advantages. However, most existing works focus on binary convolutional neural networks, while less attention has been paid to binary graph neural networks. A common drawback of existing works on binary graph neural networks is that they still include lots of inefficient full-precision operations and hence are not efficient enough. In this paper, we propose a novel method, called full-precision free binary graph neural networks (FFBGN), to avoid full-precision operations for binarizing graph neural networks. To address the challenges introduced by re-quantization which is a necessary procedure for avoiding full-precision operations, in FFBGN we first study the impact of different computation orders to find an effective computation order and then introduce mixture of experts to increase the model capacity. Experiments on three large-scale datasets show that performing re-quantization in different computation orders significantly impacts the performance of binary graph neural network models, and FFBGN can outperform other baselines to achieve state-of-the-art performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph Neural NetworkMixture-of-ExpertsQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Training Multi-Layer Binary Neural Networks With Local Binary Error Signals
Binary Neural Networks (BNNs) significantly reduce computational complexity and memory usage in machine and deep learning by representing weights and activations with just one bit. However, most existing training algorit…
Multi-class ClassificationDefending against black-box adversarial attacks with gradient-free trained sign activation neural networks
Binary neural networks have been shown to be more adversarially robust than full-precision networks but their improvements are marginal. We propose an ensemble of sign activation neural networks trained with a novel grad…
Adversarial DefenseHigh-contrast, speckle-free, true 3D holography via binary CGH optimization
Holography is a promising approach to implement the three-dimensional (3D) projection beyond the present two-dimensional technology. True 3D holography requires abilities of arbitrary 3D volume projection with high-axial…
3D HolographyQuantizationVocal Bursts Intensity PredictionBinary and Ternary Natural Language Generation
Ternary and binary neural networks enable multiplication-free computation and promise multiple orders of magnitude efficiency gains over full-precision networks if implemented on specialized hardware. However, since both…
Machine TranslationQuantizationText GenerationTranslationRaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs
Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance. Residual binarization enables hardware-friendly, matmul-free in…