Measuring Knowledge for Recognition and Knowledge Entropy
People employ their knowledge to recognize things. This paper is concerned with how to measure people's knowledge for recognition and how it changes. The discussion is based on three assumptions. Firstly, we construct two evolution process equations, of which one is for uncertainty and knowledge, and the other for uncertainty and ignorance. Secondly, by solving the equations, formulas for measuring the levels of knowledge and the levels of ignorance are obtained in two particular cases. Thirdly, a new concept of knowledge entropy is introduced. Its similarity with Boltzmann's entropy and its difference with Shannon's Entropy are examined. Finally, it is pointed out that the obtained formulas of knowledge and knowledge entropy reflect two fundamental principles: (1) The knowledge level of a group is not necessarily a simple sum of the individuals' knowledge levels; and (2) An individual's knowledge entropy never increases if the individual's thirst for knowledge never decreases.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Measuring and Modifying Factual Knowledge in Large Language Models
Large Language Models (LLMs) store an extensive amount of factual knowledge obtained from vast collections of text. To effectively utilize these models for downstream tasks, it is crucial to have reliable methods for mea…
In-Context LearningExtracting and Measuring Uncertain Biomedical Knowledge from Scientific Statements
Purpose: This study aims to develop a novel approach to extracting and measuring uncertain biomedical knowledge from scientific statements. Design/methodology/approach: Taking cardiovascular research publications in Chin…
DiagnosticScalable Precise Computation of Shannon Entropy
Quantitative information flow analyses (QIF) are a class of techniques for measuring the amount of confidential information leaked by a program to its public outputs. Shannon entropy is an important method to quantify th…
Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures
Large Language Models (LLMs) define probability measures on text. By considering the implicit knowledge question of what it means for an LLM to know such a measure and what it entails algorithmically, we are naturally le…
Anomaly DetectioncounterfactualQuantifying the Knowledge in GNNs for Reliable Distillation into MLPs
To bridge the gaps between topology-aware Graph Neural Networks (GNNs) and inference-efficient Multi-Layer Perceptron (MLPs), GLNN proposes to distill knowledge from a well-trained teacher GNN into a student MLP. Despite…