Revisiting Matching Pursuit: Beyond Approximate Submodularity
We study the problem of selecting a subset of vectors from a large set, to obtain the best signal representation over a family of functions. Although greedy methods have been widely used for tackling this problem and many of those have been analyzed under the lens of (weak) submodularity, none of these algorithms are explicitly devised using such a functional property. Here, we revisit the vector-selection problem and introduce a function which is shown to be submodular in expectation. This function does not only guarantee near-optimality through a greedy algorithm in expectation, but also alleviates the existing deficiencies in commonly used matching pursuit (MP) algorithms. We further show the relation between the single-point-estimate version of the proposed greedy algorithm and MP variants. Our theoretical results are supported by numerical experiments for the angle of arrival estimation problem, a typical signal representation task; the experiments demonstrate the benefits of the proposed method with respect to the traditional MP algorithms.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On Additive Approximate Submodularity
A real-valued set function is (additively) approximately submodular if it satisfies the submodularity conditions with an additive error. Approximate submodularity arises in many settings, especially in machine learning, …
Beyond Adaptive Submodularity: Approximation Guarantees of Greedy Policy with Adaptive Submodularity Ratio
We propose a new concept named adaptive submodularity ratio to study the greedy policy for sequential decision making. While the greedy policy is known to perform well for a wide variety of adaptive stochastic optimizati…
Decision Makingfeature selectionSequential Decision MakingStochastic OptimizationThe IMP game: Learnability, approximability and adversarial learning beyond $Σ^0_1$
We introduce a problem set-up we call the Iterated Matching Pennies (IMP) game and show that it is a powerful framework for the study of three problems: adversarial learnability, conventional (i.e., non-adversarial) lear…
Higher order Matching Pursuit for Low Rank Tensor Learning
Low rank tensor learning, such as tensor completion and multilinear multitask learning, has received much attention in recent years. In this paper, we propose higher order matching pursuit for low rank tensor learning pr…
Performance of $\ell_1$ Regularization for Sparse Convex Optimization
Despite widespread adoption in practice, guarantees for the LASSO and Group LASSO are strikingly lacking in settings beyond statistical problems, and these algorithms are usually considered to be a heuristic in the conte…
feature selection