Towards Sampling Data Structures for Tensor Products in Turnstile Streams
This paper studies the computational challenges of large-scale attention-based models in artificial intelligence by utilizing importance sampling methods in the streaming setting. Inspired by the classical definition of the $\ell_2$ sampler and the recent progress of the attention scheme in Large Language Models (LLMs), we propose the definition of the attention sampler. Our approach significantly reduces the computational burden of traditional attention mechanisms. We analyze the effectiveness of the attention sampler from a theoretical perspective, including space and update time. Additionally, our framework exhibits scalability and broad applicability across various model architectures and domains.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Turnstile $\ell_p$ leverage score sampling with applications
The turnstile data stream model offers the most flexible framework where data can be manipulated dynamically, i.e., rows, columns, and even single entries of an input matrix can be added, deleted, or updated multiple tim…
regressionNon-Adaptive Adaptive Sampling on Turnstile Streams
Adaptive sampling is a useful algorithmic tool for data summarization problems in the classical centralized setting, where the entire dataset is available to the single processor performing the computation. Adaptive samp…
ClusteringData SummarizationThree-dimensional signal processing: a new approach in dynamical sampling via tensor products
The dynamical sampling problem is centered around reconstructing signals that evolve over time according to a dynamical process, from spatial-temporal samples that may be noisy. This topic has been thoroughly explored fo…
SeKron: A Decomposition Method Supporting Many Factorization Structures
While convolutional neural networks (CNNs) have become the de facto standard for most image processing and computer vision applications, their deployment on edge devices remains challenging. Tensor decomposition methods …
Model CompressionTensor DecompositionTensor Products of Fell Bundles over Groups
We extend the theory of tensor products of C*-algebras to the larger category of Fell bundles over locally compact groups. We prove that, like in the case of C*-algebras, there exist maximal and minimal tensor products. …