Hybrid Least Squares/Gradient Descent Methods for DeepONets

2025-08-21 · Jun Choi, Chang-Ock Lee, Minam Moon arxiv

We propose an efficient hybrid least squares/gradient descent method to accelerate DeepONet training. Since the output of DeepONet can be viewed as linear with respect to the last layer parameters of the branch network, these parameters can be optimized using a least squares (LS) solve, and the remaining hidden layer parameters are updated by means of gradient descent form. However, building the LS system for all possible combinations of branch and trunk inputs yields a prohibitively large linear problem that is infeasible to solve directly. To address this issue, our method decomposes the large LS system into two smaller, more manageable subproblems $\unicode{x2014}$ one for the branch network and one for the trunk network $\unicode{x2014}$ and solves them separately. This method is generalized to a broader type of $L^2$ loss with a regularization term for the last layer parameters, including the case of unsupervised learning with physics-informed loss.

📄 PDF Abstract BibTeX arXiv:2508.15394

Code (0)

등록된 구현이 없습니다.

Hybrid Least Squares/Gradient Descent Methods for MIONets

2026-07-08 · Jun Choi, Chang-Ock Lee, Minam Moon arxiv

In this paper, we propose an efficient hybrid least squares/gradient descent (LSGD) method for MIONets to accelerate training. This method generalizes the LSGD method for DeepONets. Since MIONet is the sum of the entrywi…

Iterative Pre-Conditioning for Expediting the Gradient-Descent Method: The Distributed Linear Least-Squares Problem

2020-08-06 · Kushal Chakrabarti, Nirupam Gupta, Nikhil Chopra

This paper considers the multi-agent linear least-squares problem in a server-agent network. In this problem, the system comprises multiple agents, each having a set of local data points, that are connected to a server. …

Variance reduction in stochastic methods for large-scale regularised least-squares problems

2021-10-15 · Yusuf Pilavci, Pierre-Olivier Amblard, Simon Barthelmé, Nicolas Tremblay

Large dimensional least-squares and regularised least-squares problems are expensive to solve. There exist many approximate techniques, some deterministic (like conjugate gradient), some stochastic (like stochastic gradi…

Point Processes

A Markov Chain Theory Approach to Characterizing the Minimax Optimality of Stochastic Gradient Descent (for Least Squares)

2017-10-25 · Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli 외

This work provides a simplified proof of the statistical minimax optimality of (iterate averaged) stochastic gradient descent (SGD), for the special case of least squares. This result is obtained by analyzing SGD as a st…

On Asymptotic Linear Convergence of Projected Gradient Descent for Constrained Least Squares

2021-12-22 · Trung Vu, Raviv Raich

Many recent problems in signal processing and machine learning such as compressed sensing, image restoration, matrix/tensor recovery, and non-negative matrix factorization can be cast as constrained optimization. Project…

compressed sensing Image Restoration Matrix Completion