The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD
UKB is an open source collection of programs for performing, among other tasks, knowledge-based Word Sense Disambiguation (WSD). Since it was released in 2009 it has been often used out-of-the-box in sub-optimal settings. We show that nine years later it is the state-of-the-art on knowledge-based WSD. This case shows the pitfalls of releasing open source NLP software without optimal default settings and precise instructions for reproducibility.
Code (0)
등록된 구현이 없습니다.
Tasks
Word Sense DisambiguationSimilar Papers 제목 키워드 기반
An LLM-based Quantitative Framework for Evaluating High-Stealthy Backdoor Risks in OSS Supply Chains
In modern software development workflows, the open-source software supply chain contributes significantly to efficient and convenient engineering practices. With increasing system complexity, using open-source software a…
VeriContaminated: Assessing LLM-Driven Verilog Coding for Data Contamination
Large Language Models (LLMs) have revolutionized code generation, achieving exceptional results on various established benchmarking frameworks. However, concerns about data contamination - where benchmark data inadverten…
BenchmarkingCode GenerationFairnessDetecting Continuous Integration Skip : A Reinforcement Learning-based Approach
The software industry is experiencing a surge in the adoption of Continuous Integration (CI) practices, both in commercial and open-source environments. CI practices facilitate the seamless integration of code changes by…
Binary ClassificationDeep Reinforcement Learningreinforcement-learningReinforcement LearningBotHawk: An Approach for Bots Detection in Open Source Software Projects
Social coding platforms have revolutionized collaboration in software development, leading to using software bots for streamlining operations. However, The presence of open-source software (OSS) bots gives rise to proble…
SECBENCH: A Database of Real Security Vulnerabilities
Currently, to satisfy the high number of system requirements, complex software is created which turns its development cost-intensive and more susceptible to security vulnerabilities. In software security testing, empiric…
software testing