Decoding Knowledge Claims: The Evaluation of Scientific Publication Contributions through Semantic Analysis
The surge in scientific publications challenges the use of publication counts as a measure of scientific progress, requiring alternative metrics that emphasize the quality and novelty of scientific contributions rather than sheer quantity. This paper proposes the use of Relaxed Word Mover's Distance (RWMD), a semantic text similarity measure, to evaluate the novelty of scientific papers. We hypothesize that RWMD can more effectively gauge the growth of scientific knowledge. To test such an assumption, we apply RWMD to evaluate seminal papers, with Hirsch's H-Index paper as a primary case study. We compare RWMD results across three groups: 1) H-Index-related papers, 2) scientometric studies, and 3) unrelated papers, aiming to discern redundant literature and hype from genuine innovations. Findings suggest that emphasizing knowledge claims offers a deeper insight into scientific contributions, marking RWMD as a promising alternative method to traditional citation metrics, thus better tracking significant scientific breakthroughs.
Code (0)
등록된 구현이 없습니다.
Tasks
text similarityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The Repeated-Stimulus Confound in Electroencephalography
In neural-decoding studies, recordings of participants' responses to stimuli are used to train models. In recent years, there has been an explosion of publications detailing applications of innovations from deep-learning…
Can LLMs extract scientific consensus? A case study in high-temperature superconductivity
Scientific knowledge is increasingly dispersed across vast and heterogeneous scientific literature, where important claims are often implicit, evolving, and internally debated. While large language models (LLMs) have sho…
Information ExtractionSFBench: The SciFy Scientific Feasibility Benchmark
We present SFBench, a benchmark dataset for evaluating systems that assess the feasibility of scientific claims. SFBench includes 197 claims in materials science, each annotated with a ground-truth feasibility score on a…
Claim Extraction in Biomedical Publications using Deep Discourse Model and Transfer Learning
Claims are a fundamental unit of scientific discourse. The exponential growth in the number of scientific publications makes automatic claim extraction an important problem for researchers who are overwhelmed by this inf…
SentenceTransfer LearningSCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables
Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. We present SCITAB, a challenging evaluation dataset …
Claim VerificationFact Checking