Maxout

2000년 도입 · 논문 50편에서 사용

The Maxout Unit is a generalization of the ReLU and the leaky ReLU functions. It is a piecewise linear function that returns the maximum of the inputs, designed to be used in conjunction with dropout. Both ReLU and leaky ReLU are special cases of Maxout. $$f\left(x\right) = \max\left(w^{T}\_{1}x + b\_{1}, w^{T}\_{2}x + b\_{2}\right)$$ The main drawback of Maxout is that it is computationally expensive as it doubles the number of parameters for each neuron.

출처: Maxout Networks

소개 논문: Maxout Networks