Optimization Landscapes Learned: Proxy Networks Boost Convergence in Physics-based Inverse Problems
Solving inverse problems in physics is central to understanding complex systems and advancing technologies in various fields. Iterative optimization algorithms, commonly used to solve these problems, often encounter local minima, chaos, or regions with zero gradients. This is due to their overreliance on local information and highly chaotic inverse loss landscapes governed by underlying partial differential equations (PDEs). In this work, we show that deep neural networks successfully replicate such complex loss landscapes through spatio-temporal trajectory inputs. They also offer the potential to control the underlying complexity of these chaotic loss landscapes during training through various regularization methods. We show that optimizing on network-smoothened loss landscapes leads to improved convergence in predicting optimum inverse parameters over conventional momentum-based optimizers such as BFGS on multiple challenging problems.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning to Evolve for Optimization via Stability-Inducing Neural Unrolling
Evolutionary algorithms serve as a powerful paradigm for tackling optimization challenges, yet their reliance on manually engineered heuristics inherently limits their adaptability across diverse landscapes. However, the…
Towards the Inferrence of Structural Similarity of Combinatorial Landscapes
One of the most common problem-solving heuristics is by analogy. For a given problem, a solver can be viewed as a strategic walk on its fitness landscape. Thus if a solver works for one problem instance, we expect it wil…
Combinatorial OptimizationFeature-metric Loss for Self-supervised Learning of Depth and Egomotion
Photometric loss is widely used for self-supervised depth and egomotion estimation. However, the loss landscapes induced by photometric differences are often problematic for optimization, caused by plateau landscapes for…
Depth EstimationMonocular Depth EstimationSelf-Supervised LearningVisual OdometryLightweight Geometric Adaptation for Training Physics-Informed Neural Networks
Physics-Informed Neural Networks (PINNs) often suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations due to the anisotropic and rapidly varying geometry of…
Proxy Anchor Loss for Deep Metric Learning
Existing metric learning losses can be categorized into two classes: pair-based and proxy-based losses. The former class can leverage fine-grained semantic relations between data points, but slows convergence in general …
Fine-Grained Image ClassificationFine-Grained Vehicle ClassificationMetric Learning