MatrixKAN: Parallelized Kolmogorov-Arnold Network
Kolmogorov-Arnold Networks (KAN) are a new class of neural network architecture representing a promising alternative to the Multilayer Perceptron (MLP), demonstrating improved expressiveness and interpretability. However, KANs suffer from slow training and inference speeds relative to MLPs due in part to the recursive nature of the underlying B-spline calculations. This issue is particularly apparent with respect to KANs utilizing high-degree B-splines, as the number of required non-parallelizable recursions is proportional to B-spline degree. We solve this issue by proposing MatrixKAN, a novel optimization that parallelizes B-spline calculations with matrix representation and operations, thus significantly improving effective computation time for models utilizing high-degree B-splines. In this paper, we demonstrate the superior scaling of MatrixKAN's computation time relative to B-spline degree. Further, our experiments demonstrate speedups of approximately 40x relative to KAN, with significant additional speedup potential for larger datasets or higher spline degrees.
Code (1)
Tasks
Kolmogorov-Arnold NetworksMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
KAT to KANs: A Review of Kolmogorov-Arnold Networks and the Neural Leap Forward
The curse of dimensionality poses a significant challenge to modern multilayer perceptron-based architectures, often causing performance stagnation and scalability issues. Addressing this limitation typically requires va…
Kolmogorov-Arnold NetworksThe Kolmogorov-Arnold representation theorem revisited
There is a longstanding debate whether the Kolmogorov-Arnold representation theorem can explain the use of more than one hidden layer in neural networks. The Kolmogorov-Arnold representation decomposes a multivariate fun…
KAConvNet: Kolmogorov-Arnold Convolutional Networks for Vision Recognition
The Convolutional Neural Networks (CNNs) have been the dominant and effective approach for general computer vision tasks. Recently, Kolmogorov-Arnold neural networks (KANs), based on the Kolmogorov-Arnold representation …
A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting
Capturing complex temporal patterns and relationships within multivariate data streams is a difficult task. We propose the Temporal Kolmogorov-Arnold Transformer (TKAT), a novel attention-based architecture designed to a…
DecoderKolmogorov-Arnold NetworksTime SeriesTime Series ForecastingCombinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networks
For years, many neural networks have been developed based on the Kolmogorov-Arnold Representation Theorem (KART), which was created to address Hilbert's 13th problem. Recently, relying on KART, Kolmogorov-Arnold Networks…
Computational Efficiency