Natural Gradient for Combined Loss Using Wavelets
Natural gradients have been widely used in optimization of loss functionals over probability space, with important examples such as Fisher-Rao gradient descent for Kullback-Leibler divergence, Wasserstein gradient descent for transport-related functionals, and Mahalanobis gradient descent for quadratic loss functionals. This note considers the situation in which the loss is a convex linear combination of these examples. We propose a new natural gradient algorithm by utilizing compactly supported wavelets to diagonalize approximately the Hessian of the combined loss. Numerical results are included to demonstrate the efficiency of the proposed algorithm.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Union is strength in lossy image compression
In this work, we present a comparison between different techniques of image compression. First, the image is divided in blocks which are organized according to a certain scan. Later, several compression techniques are ap…
Image CompressionVideo Text Localization using Wavelet and Shearlet Transforms
Text in video is useful and important in indexing and retrieving the video documents efficiently and accurately. In this paper, we present a new method of text detection using a combined dictionary consisting of wavelets…
ClusteringText DetectionGabor wavelets combined with volumetric fractal dimension applied to texture analysis
Texture analysis and classification remain as one of the biggest challenges for the field of computer vision and pattern recognition. On this matter, Gabor wavelets has proven to be a useful technique to characterize dis…
General ClassificationTexture ClassificationWavelet Design in a Learning Framework
Wavelets have proven to be highly successful in several signal and image processing applications. Wavelet design has been an active field of research for over two decades, with the problem often being approached from an …
Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks
Physics-informed neural networks (PINNs) train a single neural approximation by minimizing multiple physics- and data-derived losses, but the gradients of these losses often interfere and can stall optimization. Existing…