paper-with-me

홈 › Papers

How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance

2025-05-22 · Desiree Heim, Lars-Peter Meyer, Markus Schröder, Johannes Frey, Andreas Dengel

When using Large Language Models (LLMs) to support Knowledge Graph Engineering (KGE), one of the first indications when searching for an appropriate model is its size. According to the scaling laws, larger models typically show higher capabilities. However, in practice, resource costs are also an important factor and thus it makes sense to consider the ratio between model performance and costs. The LLM-KG-Bench framework enables the comparison of LLMs in the context of KGE tasks and assesses their capabilities of understanding and producing KGs and KG queries. Based on a dataset created in an LLM-KG-Bench run covering 26 open state-of-the-art LLMs, we explore the model size scaling laws specific to KGE tasks. In our analyses, we assess how benchmark scores evolve between different model size categories. Additionally, we inspect how the general score development of single models and families of models correlates to their size. Our analyses revealed that, with a few exceptions, the model size scaling laws generally also apply to the selected KGE tasks. However, in some cases, plateau or ceiling effects occurred, i.e., the task performance did not change much between a model and the next larger model. In these cases, smaller models could be considered to achieve high cost-effectiveness. Regarding models of the same family, sometimes larger models performed worse than smaller models of the same family. These effects occurred only locally. Hence it is advisable to additionally test the next smallest and largest model of the same family.

📄 PDF Abstract BibTeX arXiv:2505.16276

Code (1)

aksw/llm-kg-bench 공식 구현

Tasks

Language ModelingLanguage ModellingLarge Language Model

Similar Papers 제목 키워드 기반

Towards Neural Scaling Laws on Graphs

2024-02-03 · Jingzhe Liu, Haitao Mao, Zhikai Chen, Tong Zhao 외

Deep graph models (e.g., graph neural networks and graph transformers) have become important techniques for leveraging knowledge across various types of graphs. Yet, the neural scaling laws on graphs, i.e., how the perfo…

Graph ClassificationLink PredictionNode Classification

Data-driven Progressive Discovery of Physical Laws

2026-03-14 · Mingkun Xia, Weiwei Zhang arxiv

Symbolic regression is a powerful tool for knowledge discovery, enabling the extraction of interpretable mathematical expressions directly from data. However, conventional symbolic discovery typically follows an end-to-e…

EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

2026-07-06 · Deyao Zhu, Xin Zhou, Shengling Qin, Xuekai Zhu 외 arxiv

Pretraining scaling laws reveal that model capability improves predictably with data and compute. But learning from real world environments after deployment remains far less understood. Analyzing roughly 38,000 hours of …

Is the Number of Trainable Parameters All That Actually Matters?

2021-09-24 · NeurIPS Workshop ICBINB 2021 12 · Amélie Chatelain, Amine Djeghri, Daniel Hesslow, Julien Launay 외

Recent work has identified simple empirical scaling laws for language models, linking compute budget, dataset size, model size, and autoregressive modeling loss. The validity of these simple power laws across orders of m…

All

On the Scaling Laws of Geographical Representation in Language Models

2024-02-29 · Nathan Godey, Éric de la Clergerie, Benoît Sagot

Language models have long been shown to embed geographical information in their hidden representations. This line of work has recently been revisited by extending this result to Large Language Models (LLMs). In this pape…