paper-with-me

홈 › Papers

Gauss-Newton Natural Gradient Descent for Shape Learning

2026-01-24 · James King, Arturs Berzins, Siddhartha Mishra, Marius Zeinhofer arxiv

We explore the use of the Gauss-Newton method for optimization in shape learning, including implicit neural surfaces and geometry-informed neural networks. The method addresses key challenges in shape learning, such as the ill-conditioning of the underlying differential constraints and the mismatch between the optimization problem in parameter space and the function space where the problem is naturally posed. This leads to significantly faster and more stable convergence than standard first-order methods, while also requiring far fewer iterations. Experiments across benchmark shape optimization tasks demonstrate that the Gauss-Newton method consistently improves both training speed and final solution accuracy.

📄 PDF Abstract BibTeX arXiv:2602.00099

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

First-order and second-order variants of the gradient descent in a unified framework

2018-10-18 · Thomas Pierrot, Nicolas Perrin, Olivier Sigaud

In this paper, we provide an overview of first-order and second-order variants of the gradient descent method that are commonly used in machine learning. We propose a general framework in which 6 of these variants can be…

BIG-bench Machine Learning

Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks

2025-05-27 · Anas Jnini, Flavio Vella

Natural-gradient methods markedly accelerate the training of Physics-Informed Neural Networks (PINNs), yet their Gauss--Newton update must be solved in the parameter space, incurring a prohibitive $O(n^3)$ time complexit…

GPU

Modified Gauss-Newton Algorithms under Noise

2023-05-18 · Krishna Pillutla, Vincent Roulet, Sham Kakade, Zaid Harchaoui

Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing. Their nonsmooth counterparts, modified Gauss-Newton or prox-linear algorithms, can lead to contrasting ou…

Structured Prediction

Error whitening: Why Gauss-Newton outperforms Newton

2026-05-11 · Maricela Best McKay, Nathan P. Lawrence, Brian Wetton, R. Bhushan Gopaluni arxiv

The Gauss-Newton matrix is widely viewed as a positive semidefinite approximation of the Hessian, yet mounting empirical evidence shows that Gauss-Newton descent outperforms Newton's method. We adopt a function space per…

Natural Gradient Methods: Perspectives, Efficient-Scalable Approximations, and Analysis

2023-03-06 · Rajesh Shrestha

Natural Gradient Descent, a second-degree optimization method motivated by the information geometry, makes use of the Fisher Information Matrix instead of the Hessian which is typically used. However, in many cases, the …