Min-Max-Plus Neural Networks
We present a new model of neural networks called Min-Max-Plus Neural Networks (MMP-NNs) based on operations in tropical arithmetic. In general, an MMP-NN is composed of three types of alternately stacked layers, namely linear layers, min-plus layers and max-plus layers. Specifically, the latter two types of layers constitute the nonlinear part of the network which is trainable and more sophisticated compared to the nonlinear part of conventional neural networks. In addition, we show that with higher capability of nonlinearity expression, MMP-NNs are universal approximators of continuous functions, even when the number of multiplication operations is tremendously reduced (possibly to none in certain extreme cases). Furthermore, we formulate the backpropagation algorithm in the training process of MMP-NNs and introduce an algorithm of normalization to improve the rate of convergence in training.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Squareplus: A Softplus-Like Algebraic Rectifier
We present squareplus, an activation function that resembles softplus, but which can be computed using only algebraic operations: addition, multiplication, and square-root. Because squareplus is ~6x faster to evaluate th…
CPUSurplus sharing with coherent utility functions
We use the theory of coherent measures to look at the problem of surplus sharing in an insurance business. The surplus share of an insured is calculated by the surplus premium in the contract. The theory of coherent risk…
The Division of Surplus and the Burden of Proof
A surplus must be divided between a principal and an agent. Only the agent knows the surplus' true size and decides how much of it to reveal initially. Both parties can exert costly effort to conclusively prove the surpl…
AdaPlus: Integrating Nesterov Momentum and Precise Stepsize Adjustment on AdamW Basis
This paper proposes an efficient optimizer called AdaPlus which integrates Nesterov momentum and precise stepsize adjustment on AdamW basis. AdaPlus combines the advantages of AdamW, Nadam, and AdaBelief and, in particul…
image-classificationImage ClassificationLanguage ModelingLanguage ModellingFitted Q-Iteration via Max-Plus-Linear Approximation
In this study, we consider the application of max-plus-linear approximators for Q-function in offline reinforcement learning of discounted Markov decision processes. In particular, we incorporate these approximators to p…