SimpleNet

2000년 도입 · 논문 6편에서 사용

SimpleNet is a convolutional neural network with 13 layers. The network employs a homogeneous design utilizing 3 × 3 kernels for convolutional layer and 2 × 2 kernels for pooling operations. The only layers which do not use 3 × 3 kernels are 11th and 12th layers, these layers, utilize 1 × 1 convolutional kernels. Feature-map down-sampling is carried out using nonoverlaping 2 × 2 max-pooling. In order to cope with the problem of vanishing gradient and also over-fitting, SimpleNet also uses batch-normalization with moving average fraction of 0.95 before any ReLU non-linearity.

출처: Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures

소개 논문: Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures