Entropy Estimations Using Correlated Symmetric Stable Random Projections
Methods for efficiently estimating the Shannon entropy of data streams have important applications in learning, data mining, and network anomaly detections (e.g., the DDoS attacks). For nonnegative data streams, the method of Compressed Counting (CC) based on maximally-skewed stable random projections can provide accurate estimates of the Shannon entropy using small storage. However, CC is no longer applicable when entries of data streams can be below zero, which is a common scenario when comparing two streams. In this paper, we propose an algorithm for entropy estimation in general data streams which allow negative entries. In our method, the Shannon entropy is approximated by the finite difference of two correlated frequency moments estimated from correlated samples of symmetric stable random variables. Our experiments confirm that this method is able to substantially better approximate the Shannon entropy compared to the prior state-of-the-art.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Asymmetric Correntropy for Robust Adaptive Filtering
In recent years, correntropy has been seccessfully applied to robust adaptive filtering to eliminate adverse effects of impulsive noises or outliers. Correntropy is generally defined as the expectation of a Gaussian kern…
Learning with Correntropy-induced Losses for Regression with Mixture of Symmetric Stable Noise
In recent years, correntropy and its applications in machine learning have been drawing continuous attention owing to its merits in dealing with non-Gaussian noise and outliers. However, theoretical understanding of corr…
regressionSVR-based Observer Design for Unknown Linear Systems: Complexity and Performance
In this paper we consider estimating the system parameters and designing stable observer for unknown noisy linear time-invariant (LTI) systems. We propose a Support Vector Regression (SVR) based estimator to provide adju…
A non-extensive entropy feature and its application to texture classification
This paper proposes a new probabilistic non-extensive entropy feature for texture characterization, based on a Gaussian information measure. The highlights of the new entropy are that it is bounded by finite limits and t…
General ClassificationTexture ClassificationAsymmetric Gained Deep Image Compression With Continuous Rate Adaptation
With the development of deep learning techniques, the combination of deep learning with image compression has drawn lots of attention. Recently, learned image compression methods had exceeded their classical counterparts…
Image CompressionMS-SSIMOpen-Ended Question AnsweringSSIM