Towards a Measure Theory of Semantic Information
A classic account of the quantification of semantic information is that of Bar-Hiller and Carnap. Their account proposes an inverse relation between the informativeness of a statement and its probability. However, their approach assigns the maximum informativeness to a contradiction: which Floridi refers to as the Bar-Hillel-Carnap paradox. He developed a novel theory founded on a distance metric and parabolic relation, designed to remove this paradox. Unfortunately is approach does not succeed in that aim. In this paper I critique Floridi's theory of strongly semantic information on its own terms and show where it succeeds and fails. I then present a new approach based on the unit circle (a relation that has been the basis of theories from basic trigonometry to quantum theory). This is used, by analogy with von Neumann's quantum probability to construct a measure space for informativeness that meets all the requirements stipulated by Floridi and removes the paradox. In addition, while contradictions and tautologies have zero informativeness, it is found that messages which are contradictory to each other are equally informative. The utility of this is explained by means of an example.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On Quantum Generalizations of Information-Theoretic Measures and their Contribution to Distributional Semantics
Information-theoretic measures such as relative entropy and correlation are extremely useful when modeling or analyzing the interaction of probabilistic systems. We survey the quantum generalization of 5 such measures an…
Semantic Information G Theory for Range Control with Tradeoff between Purposiveness and Efficiency
Recent advances in deep learning suggest that we need to maximize and minimize two different kinds of information simultaneously. The Information Max-Min (IMM) method has been used in deep learning, reinforcement learnin…
Data CompressionDeep Learningreinforcement-learningReinforcement LearningReviewing Evolution of Learning Functions and Semantic Information Measures for Understanding Deep Learning
A new trend in deep learning, represented by Mutual Information Neural Estimation (MINE) and Information Noise Contrast Estimation (InfoNCE), is emerging. In this trend, similarity functions and Estimated Mutual Informat…
Deep LearningMulti-Label LearningForget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs
Despite the empirical successes of Large Language Models (LLMs), the prevailing paradigm is heuristic and experiment-driven, tethered to massive compute and data, while a first-principles theory remains absent. This trea…
Causal InferenceA Novel Information Theoretic Framework for Finding Semantic Similarity in WordNet
Information content (IC) based measures for finding semantic similarity is gaining preferences day by day. Semantics of concepts can be highly characterized by information theory. The conventional way for calculating IC …
Semantic SimilaritySemantic Textual Similarity