Tensor Decompositions in Deep Learning
The paper surveys the topic of tensor decompositions in modern machine learning applications. It focuses on three active research topics of significant relevance for the community. After a brief review of consolidated works on multi-way data analysis, we consider the use of tensor decompositions in compressing the parameter space of deep learning models. Lastly, we discuss how tensor methods can be leveraged to yield richer adaptive representations of complex data, including structured information. The paper concludes with a discussion on interesting open research challenges.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningDeep LearningSimilar Papers 제목 키워드 기반
Tensor Decompositions: A New Concept in Brain Data Analysis?
Matrix factorizations and their extensions to tensor factorizations and decompositions have become prominent techniques for linear and multilinear blind source separation (BSS), especially multiway Independent Component …
blind source separationClassificationClusteringDimensionality Reduction+2Efficient Contraction of Large Tensor Networks for Weighted Model Counting through Graph Decompositions
Constrained counting is a fundamental problem in artificial intelligence. A promising new algebraic approach to constrained counting makes use of tensor networks, following a reduction from constrained counting to the pr…
Tensor NetworksMode-wise Tensor Decompositions: Multi-dimensional Generalizations of CUR Decompositions
Low rank tensor approximation is a fundamental tool in modern machine learning and data science. In this paper, we study the characterization, perturbation analysis, and an efficient sampling strategy for two primary ten…
A tensor network formalism for neuro-symbolic AI
The unification of neural and symbolic approaches to artificial intelligence remains a central open challenge. In this work, we introduce a tensor network formalism, which captures sparsity principles originating in the …
TenExp: Mixture-of-Experts-Based Tensor Decomposition Structure Search Framework
Recently, tensor decompositions continue to emerge and receive increasing attention. Selecting a suitable tensor decomposition to exactly capture the low-rank structures behind the data is at the heart of the tensor deco…