Sparse Polyak with optimal thresholding operators for high-dimensional M-estimation
We propose and analyze a variant of Sparse Polyak for high dimensional M-estimation problems. Sparse Polyak proposes a novel adaptive step-size rule tailored to suitably estimate the problem's curvature in the high-dimensional setting, guaranteeing that the algorithm's performance does not deteriorate when the ambient dimension increases. However, convergence guarantees can only be obtained by sacrificing solution sparsity and statistical accuracy. In this work, we introduce a variant of Sparse Polyak that retains its desirable scaling properties with respect to the ambient dimension while obtaining sparser and more accurate solutions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Between hard and soft thresholding: optimal iterative thresholding algorithms
Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enfor…
Sparse Polyak: an adaptive step size rule for high-dimensional M-estimation
We propose and study Sparse Polyak, a variant of Polyak's adaptive step size, designed to solve high-dimensional statistical estimation problems where the problem dimension is allowed to grow much faster than the sample …
Stochastic Natural Thresholding Algorithms
Sparse signal recovery is one of the most fundamental problems in various applications, including medical imaging and remote sensing. Many greedy algorithms based on the family of hard thresholding operators have been de…
Computational EfficiencyNewton-Type Optimal Thresholding Algorithms for Sparse Optimization Problems
Sparse signals can be possibly reconstructed by an algorithm which merges a traditional nonlinear optimization method and a certain thresholding technique. Different from existing thresholding methods, a novel thresholdi…
Compressive SensingVocal Bursts Type PredictionLearning optimal nonlinearities for iterative thresholding algorithms
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding fun…