paper-with-me

홈 › Papers

Regression for matrix-valued data via Kronecker products factorization

2024-04-30 · Yin-Jen Chen, Minh Tang

We study the matrix-variate regression problem $Y_i = \sum_{k} \beta_{1k} X_i \beta_{2k}^{\top} + E_i$ for $i=1,2\dots,n$ in the high dimensional regime wherein the response $Y_i$ are matrices whose dimensions $p_{1}\times p_{2}$ outgrow both the sample size $n$ and the dimensions $q_{1}\times q_{2}$ of the predictor variables $X_i$ i.e., $q_{1},q_{2} \ll n \ll p_{1},p_{2}$. We propose an estimation algorithm, termed KRO-PRO-FAC, for estimating the parameters $\{\beta_{1k}\} \subset \Re^{p_1 \times q_1}$ and $\{\beta_{2k}\} \subset \Re^{p_2 \times q_2}$ that utilizes the Kronecker product factorization and rearrangement operations from Van Loan and Pitsianis (1993). The KRO-PRO-FAC algorithm is computationally efficient as it does not require estimating the covariance between the entries of the $\{Y_i\}$. We establish perturbation bounds between $\hat{\beta}_{1k} -\beta_{1k}$ and $\hat{\beta}_{2k} - \beta_{2k}$ in spectral norm for the setting where either the rows of $E_i$ or the columns of $E_i$ are independent sub-Gaussian random vectors. Numerical studies on simulated and real data indicate that our procedure is competitive, in terms of both estimation error and predictive accuracy, compared to other existing methods.

📄 PDF Abstract BibTeX arXiv:2404.19220

Code (0)

등록된 구현이 없습니다.

Tasks

2kregression

Similar Papers 제목 키워드 기반

Dominant Z-Eigenpairs of Tensor Kronecker Products are Decoupled and Applications to Higher-Order Graph Matching

2020-11-17 · Charles Colley, Huda Nassar, David Gleich

Tensor Kronecker products, the natural generalization of the matrix Kronecker product, are independently emerging in multiple research communities. Like their matrix counterpart, the tensor generalization gives structure…

Graph Matching

A penalized likelihood method for classification with matrix-valued predictors

2016-09-23 · Aaron J. Molstad, Adam J. Rothman

We propose a penalized likelihood method to fit the linear discriminant analysis model when the predictor is matrix valued. We simultaneously estimate the means and the precision matrix, which we assume has a Kronecker p…

ClassificationEEGElectroencephalogram (EEG)General Classification

Compressing Language Models using Doped Kronecker Products

2020-01-24 · Urmish Thakker, Paul N. Whatmough, Zhi-Gang Liu, Matthew Mattina 외

Kronecker Products (KP) have been used to compress IoT RNN Applications by 15-38x compression factors, achieving better results than traditional compression methods. However when KP is applied to large Natural Language P…

Language ModelingLanguage ModellingLarge Language Model

Hybrid Kronecker Product Decomposition and Approximation

2019-12-06 · Chencheng Cai, Rong Chen, Han Xiao

Discovering the underlying low dimensional structure of high dimensional data has attracted a significant amount of researches recently and has shown to have a wide range of applications. As an effective dimension reduct…

Dimensionality Reduction

Compression of Fully-Connected Layer in Neural Network by Kronecker Product

2015-07-21 · Shuchang Zhou, Jia-Nan Wu

In this paper we propose and study a technique to reduce the number of parameters and computation time in fully-connected layers of neural networks using Kronecker product, at a mild cost of the prediction quality. The t…