paper-with-me

홈 › Papers

Few-Shot Data-Driven Algorithms for Low Rank Approximation

2021-12-01 · NeurIPS 2021 12 · Piotr Indyk, Tal Wagner, David Woodruff

Recently, data-driven and learning-based algorithms for low rank matrix approximation were shown to outperform classical data-oblivious algorithms by wide margins in terms of accuracy. Those algorithms are based on the optimization of sparse sketching matrices, which lead to large savings in time and memory during testing. However, they require long training times on a large amount of existing data, and rely on access to specialized hardware and software. In this work, we develop new data-driven low rank approximation algorithms with better computational efficiency in the training phase, alleviating these drawbacks. Furthermore, our methods are interpretable: while previous algorithms choose the sketching matrix either at random or by black-box learning, we show that it can be set (or initialized) to clearly interpretable values extracted from the dataset. Our experiments show that our algorithms, either by themselves or in combination with previous methods, achieve significant empirical advantage over previous work, improving training times by up to an order of magnitude toward achieving the same target accuracy.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Computational Efficiency

Similar Papers 제목 키워드 기반

Learning Sparsity and Randomness for Data-driven Low Rank Approximation

2022-12-15 · Tiejin Chen, Yicheng Tao

Learning-based low rank approximation algorithms can significantly improve the performance of randomized low rank approximation with sketch matrix. With the learned value and fixed non-zero positions for sketch matrices …

Low-Rank Dynamic Mode Decomposition: An Exact and Tractable Solution

2016-10-10 · Patrick Héas, Cédric Herzet

This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solv…

On Low-Rank Hankel Matrix Denoising

2020-12-14 · Mingzhou Yin, Roy S. Smith

The low-complexity assumption in linear systems can often be expressed as rank deficiency in data matrices with generalized Hankel structure. This makes it possible to denoise the data by estimating the underlying struct…

Denoising

Neural Language Model Pruning for Automatic Speech Recognition

2023-10-05 · Leonardo Emili, Thiago Fraga-Silva, Ernest Pusateri, Markus Nußbaum-Thom 외

We study model pruning methods applied to Transformer-based neural network language models for automatic speech recognition. We explore three aspects of the pruning frame work, namely criterion, method and scheduler, ana…

Automatic Speech RecognitionLanguage ModelingLanguage Modellingspeech-recognition+1

Algorithms for $\ell_p$ Low-Rank Approximation

2017-08-01 · ICML 2017 8 · Flavio Chierichetti, Sreenivas Gollapudi, Ravi Kumar, Silvio Lattanzi 외

We consider the problem of approximating a given matrix by a low-rank matrix so as to minimize the entrywise $\ell_p$-approximation error, for any $p \geq 1$; the case $p = 2$ is the classical SVD problem. We obtain…