paper-with-me

홈 › Papers

Full-Precision Free Binary Graph Neural Networks

2021-09-29 · Kai-Lang Yao, Wu-Jun Li

Binary neural networks have become a promising research topic due to their fast inference speed and low energy consumption advantages. However, most existing works focus on binary convolutional neural networks, while less attention has been paid to binary graph neural networks. A common drawback of existing works on binary graph neural networks is that they still include lots of inefficient full-precision operations and hence are not efficient enough. In this paper, we propose a novel method, called full-precision free binary graph neural networks (FFBGN), to avoid full-precision operations for binarizing graph neural networks. To address the challenges introduced by re-quantization which is a necessary procedure for avoiding full-precision operations, in FFBGN we first study the impact of different computation orders to find an effective computation order and then introduce mixture of experts to increase the model capacity. Experiments on three large-scale datasets show that performing re-quantization in different computation orders significantly impacts the performance of binary graph neural network models, and FFBGN can outperform other baselines to achieve state-of-the-art performance.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Graph Neural NetworkMixture-of-ExpertsQuantization

Methods 이 논문이 사용한 방법론

Graph Neural Network 설명 없음

Similar Papers 제목 키워드 기반

Training Multi-Layer Binary Neural Networks With Local Binary Error Signals

2024-11-28 · Luca Colombo, Fabrizio Pittorino, Manuel Roveri

Binary Neural Networks (BNNs) significantly reduce computational complexity and memory usage in machine and deep learning by representing weights and activations with just one bit. However, most existing training algorit…

Multi-class Classification

Defending against black-box adversarial attacks with gradient-free trained sign activation neural networks

2021-01-01 · Yunzhe Xue, Meiyan Xie, Zhibo Yang, Usman Roshan

Binary neural networks have been shown to be more adversarially robust than full-precision networks but their improvements are marginal. We propose an ensemble of sign activation neural networks trained with a novel grad…

Adversarial Defense

High-contrast, speckle-free, true 3D holography via binary CGH optimization

2022-01-07 · Byounghyo Lee, Dongyeon Kim, Seungjae Lee, Chun Chen 외

Holography is a promising approach to implement the three-dimensional (3D) projection beyond the present two-dimensional technology. True 3D holography requires abilities of arbitrary 3D volume projection with high-axial…

3D HolographyQuantizationVocal Bursts Intensity Prediction

Binary and Ternary Natural Language Generation

2023-06-02 · Zechun Liu, Barlas Oguz, Aasish Pappu, Yangyang Shi 외

Ternary and binary neural networks enable multiplication-free computation and promise multiple orders of magnitude efficiency gains over full-precision networks if implemented on specialized hardware. However, since both…

Machine TranslationQuantizationText GenerationTranslation

RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs

2026-02-05 · Youngcheon You, Banseok Lee, Minseop Choi, Seonyoung Kim 외 arxiv

Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance. Residual binarization enables hardware-friendly, matmul-free in…