paper-with-me

Papers

The Hypervolume Indicator Hessian Matrix: Analytical Expression, Computational Time Complexity, and Sparsity

2022-11-08 · André H. Deutz, Michael T. M. Emmerich, Hao Wang

The problem of approximating the Pareto front of a multiobjective optimization problem can be reformulated as the problem of finding a set that maximizes the hypervolume indicator. This paper establishes the analytical expression of the Hessian matrix of the mapping from a (fixed size) collection of $n$ points in the $d$-dimensional decision space (or $m$ dimensional objective space) to the scalar hypervolume indicator value. To define the Hessian matrix, the input set is vectorized, and the matrix is derived by analytical differentiation of the mapping from a vectorized set to the hypervolume indicator. The Hessian matrix plays a crucial role in second-order methods, such as the Newton-Raphson optimization method, and it can be used for the verification of local optimal sets. So far, the full analytical expression was only established and analyzed for the relatively simple bi-objective case. This paper will derive the full expression for arbitrary dimensions ($m\geq2$ objective functions). For the practically important three-dimensional case, we also provide an asymptotically efficient algorithm with time complexity in $O(n\log n)$ for the exact computation of the Hessian Matrix' non-zero entries. We establish a sharp bound of $12m-6$ for the number of non-zero entries. Also, for the general $m$-dimensional case, a compact recursive analytical expression is established, and its algorithmic implementation is discussed. Also, for the general case, some sparsity results can be established; these results are implied by the recursive expression. To validate and illustrate the analytically derived algorithms and results, we provide a few numerical examples using Python and Mathematica implementations. Open-source implementations of the algorithms and testing data are made available as a supplement to this paper.

📄 PDF Abstract BibTeX arXiv:2211.04171

Code (1)

wangronin/hypervolumederivatives 공식 구현

Tasks

Multiobjective OptimizationSecond-order methods

Similar Papers 제목 키워드 기반

Learning to Approximate: Auto Direction Vector Set Generation for Hypervolume Contribution Approximation

2022-01-18 · Ke Shang, Tianye Shu, Hisao Ishibuchi

Hypervolume contribution is an important concept in evolutionary multi-objective optimization (EMO). It involves in hypervolume-based EMO algorithms and hypervolume subset selection algorithms. Its main drawback is that …

Hypervolume-based Multi-objective Bayesian Optimization with Student-t Processes

2016-12-01 · Joachim van der Herten, Ivo Couckuyt, Tom Dhaene

Student-$t$ processes have recently been proposed as an appealing alternative non-parameteric function prior. They feature enhanced flexibility and predictive variance. In this work the use of Student-$t$ processes are e…

Bayesian OptimizationGaussian Processes

Analytical Gradient and Hessian Evaluation for System Identification using State-Parameter Transition Tensors

2024-07-03 · Premjit Saha, Tarunraj Singh

In this work, the Einstein notation is utilized to synthesize state and parameter transition matrices, by solving a set of ordinary differential equations. Additionally, for the system identification problem, it has been…

Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training

2020-06-16 · Diego Granziol, Stefan Zohren, Stephen Roberts

We study the effect of mini-batching on the loss landscape of deep neural networks using spiked, field-dependent random matrix theory. We demonstrate that the magnitude of the extremal values of the batch Hessian are lar…

Second-order methods

Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form

2017-09-16 · Maxim Naumov

In this paper we focus on the linear algebra theory behind feedforward (FNN) and recurrent (RNN) neural networks. We review backward propagation, including backward propagation through time (BPTT). Also, we obtain a new …

Form