On Suspicious Coincidences and Pointwise Mutual Information
Barlow (1985) hypothesized that the co-occurrence of two events $A$ and $B$ is "suspicious" if $P(A,B) \gg P(A) P(B)$. We first review classical measures of association for $2 \times 2$ contingency tables, including Yule's $Y$ (Yule, 1912), which depends only on the odds ratio $\lambda$, and is independent of the marginal probabilities of the table. We then discuss the mutual information (MI) and pointwise mutual information (PMI), which depend on the ratio $P(A,B)/P(A)P(B)$, as measures of association. We show that, once the effect of the marginals is removed, MI and PMI behave similarly to $Y$ as functions of $\lambda$. The pointwise mutual information is used extensively in some research communities for flagging suspicious coincidences, but it is important to bear in mind the sensitivity of the PMI to the marginals, with increased scores for sparser events.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On the Properties and Estimation of Pointwise Mutual Information Profiles
The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precise…
Mutual Information EstimationUncertainty QuantificationWeakly Supervised Object Detection with Pointwise Mutual Information
In this work a novel approach for weakly supervised object detection that incorporates pointwise mutual information is presented. A fully convolutional neural network architecture is applied in which the network learns o…
Objectobject-detectionObject DetectionWeakly Supervised Object DetectionImproving Pointwise Mutual Information (PMI) by Incorporating Significant Co-occurrence
Streaming Pointwise Mutual Information
Recent work has led to the ability to perform space efficient, approximate counting over large vocabularies in a streaming context. Motivated by the existence of data structures of this type, we explore the computation …
ArticlesRelevant and Informative Response Generation using Pointwise Mutual Information
A sequence-to-sequence model tends to generate generic responses with little information for input utterances. To solve this problem, we propose a neural model that generates relevant and informative responses. Our model…
DecoderResponse Generation