Quantized Minimum Error Entropy Criterion
Comparing with traditional learning criteria, such as mean square error (MSE), the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyis entropy estimator, called information potential (IP), is a popular MEE cost in information theoretic learning (ITL). The computational complexity of IP is however quadratic in terms of sample number due to double summation. This creates computational bottlenecks especially for large-scale datasets. To address this problem, in this work we propose an efficient quantization approach to reduce the computational burden of IP, which decreases the complexity from O(N*N) to O (MN) with M << N. The new learning criterion is called the quantized MEE (QMEE). Some basic properties of QMEE are presented. Illustrative examples are provided to verify the excellent performance of QMEE.
Code (0)
등록된 구현이 없습니다.
Tasks
QuantizationSimilar Papers 제목 키워드 기반
Generalized Minimum Error with Fiducial Points Criterion for Robust Learning
The conventional Minimum Error Entropy criterion (MEE) has its limitations, showing reduced sensitivity to error mean values and uncertainty regarding error probability density function locations. To overcome this, a MEE…
Acoustic echo cancellationGeneralized Minimum Error Entropy for Adaptive Filtering
Error entropy is a important nonlinear similarity measure, and it has received increasing attention in many practical applications. The default kernel function of error entropy criterion is Gaussian kernel function, howe…
Restricted Minimum Error Entropy Criterion for Robust Classification
The minimum error entropy (MEE) criterion has been verified as a powerful approach for non-Gaussian signal processing and robust machine learning. However, the implementation of MEE on robust classification is rather a v…
ClassificationDimensionality ReductionGeneral ClassificationQuantization+1Nonlinear Equalization for TDMR Channels Using Neural Networks
This paper presents new structure and adaptation criterion for equalization of two-dimensional magnetic recording channels, as opposed to typical linear equalizer with minimum mean square error (MMSE) as adaptation crite…
Adaptively Solving the Local-Minimum Problem for Deep Neural Networks
This paper aims to overcome a fundamental problem in the theory and application of deep neural networks (DNNs). We propose a method to solve the local minimum problem in training DNNs directly. Our method is based on the…