paper-with-me

Papers

On Sharp Stochastic Zeroth Order Hessian Estimators over Riemannian Manifolds

2022-01-26 · Tianyu Wang

We study Hessian estimators for functions defined over an $n$-dimensional complete analytic Riemannian manifold. We introduce new stochastic zeroth-order Hessian estimators using $O (1)$ function evaluations. We show that, for an analytic real-valued function $f$, our estimator achieves a bias bound of order $ O \left( \gamma \delta^2 \right) $, where $ \gamma $ depends on both the Levi-Civita connection and function $f$, and $\delta$ is the finite difference step size. To the best of our knowledge, our results provide the first bias bound for Hessian estimators that explicitly depends on the geometry of the underlying Riemannian manifold. We also study downstream computations based on our Hessian estimators. The supremacy of our method is evidenced by empirical evaluations.

📄 PDF Abstract BibTeX arXiv:2201.10780

Code (1)

wangt1anyu/zeroth-order-Riemann-Hess-code 공식 구현

Similar Papers 제목 키워드 기반

Stochastic Zeroth Order Gradient and Hessian Estimators: Variance Reduction and Refined Bias Bounds

2022-05-29 · Yasong Feng, Tianyu Wang

We study stochastic zeroth order gradient and Hessian estimators for real-valued functions in $\mathbb{R}^n$. We show that, via taking finite difference along random orthogonal directions, the variance of the stochastic …

Stochastic Zeroth-Order Optimization under Strongly Convexity and Lipschitz Hessian: Minimax Sample Complexity

2024-06-28 · Qian Yu, Yining Wang, Baihe Huang, Qi Lei 외

Optimization of convex functions under stochastic zeroth-order feedback has been a major and challenging question in online learning. In this work, we consider the problem of optimizing second-order smooth and strongly c…

Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization

2020-03-25 · Jiaxiang Li, Krishnakumar Balasubramanian, Shiqian Ma

We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problem with only noisy objective function evaluations. Toward…

Riemannian optimization

Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens

2026-05-29 · Junbin Qiu, Zhaowei Hong, Renzhe Xu, Yao Shu arxiv

Accurate Zeroth-Order (ZO) Hessian estimation is a cornerstone of derivative-free methods, essential for tasks such as bilevel optimization, Bayesian inference, and uncertainty quantification. However, obtaining a comple…

Bilevel OptimizationBayesian Inference

FedZeN: Towards superlinear zeroth-order federated learning via incremental Hessian estimation

2023-09-29 · Alessio Maritan, Subhrakanti Dey, Luca Schenato

Federated learning is a distributed learning framework that allows a set of clients to collaboratively train a model under the orchestration of a central server, without sharing raw data samples. Although in many practic…

Federated LearningPrivacy Preserving