A Walsh Hadamard Derived Linear Vector Symbolic Architecture
Vector Symbolic Architectures (VSAs) are one approach to developing Neuro-symbolic AI, where two vectors in $\mathbb{R}^d$ are `bound' together to produce a new vector in the same space. VSAs support the commutativity and associativity of this binding operation, along with an inverse operation, allowing one to construct symbolic-style manipulations over real-valued vectors. Most VSAs were developed before deep learning and automatic differentiation became popular and instead focused on efficacy in hand-designed systems. In this work, we introduce the Hadamard-derived linear Binding (HLB), which is designed to have favorable computational efficiency, and efficacy in classic VSA tasks, and perform well in differentiable systems. Code is available at https://github.com/FutureComputing4AI/Hadamard-derived-Linear-Binding
Code (1)
Tasks
Computational EfficiencySimilar Papers 제목 키워드 기반
Multiallelic Walsh transforms
A closed formula multiallelic Walsh (or Hadamard) transform is introduced. Basic results are derived, and a statistical interpretation of some of the resulting linear forms is discussed.
Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks
In this paper, we propose a novel layer based on fast Walsh-Hadamard transform (WHT) and smooth-thresholding to replace $1\times 1$ convolution layers in deep neural networks. In the WHT domain, we denoise the transform …
Walsh-domain Neural Network for Power Amplifier Behavioral Modelling and Digital Predistortion
This paper investigates the use of Neural Network (NN) nonlinear modelling for Power Amplifier (PA) linearization in the Walsh-Hadamard transceiver architecture. This novel architecture has recently been proposed for ult…
Knowledge DistillationMcKernel: A Library for Approximate Kernel Expansions in Log-linear Time
Kernel Methods Next Generation (KMNG) introduces a framework to use kernel approximates in the mini-batch setting with SGD Optimizer as an alternative to Deep Learning. McKernel is a C++ library for KMNG ML Large-scale. …
CPUGeneral ClassificationSPRIGHT: A Fast and Robust Framework for Sparse Walsh-Hadamard Transform
We consider the problem of computing the Walsh-Hadamard Transform (WHT) of some $N$-length input vector in the presence of noise, where the $N$-point Walsh spectrum is $K$-sparse with $K = {O}(N^{\delta})$ scaling sub-li…