Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference
Small language models fine-tuned for graph property estimation have demonstrated strong in-distribution performance, yet their generalization capabilities beyond training conditions remain poorly understood. In this work, we systematically investigate the boundaries of structural inference in fine-tuned small language models along two generalization axes - graph size and graph family distribution - and assess domain-learning capability on real-world graph benchmarks. Using a controlled experimental setup with three instruction-tuned models in the 3-4B parameter class and two graph serialization formats, we evaluate performance on graphs substantially larger than the training range and across held-out random graph families. Our results show that fine-tuned models maintain strong ordinal consistency across structurally distinct graph families and continue to rank graphs by structural properties on inputs substantially larger than those seen during training, with distinct architecture-specific degradation profiles. These findings delineate where fine-tuned small language models generalize reliably, providing empirical grounding for their use in graph-based reasoning tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Case Study of Cross-Lingual Zero-Shot Generalization for Classical Languages in LLMs
Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across diverse tasks and languages. In this study, we focus on natural language understanding in three classical languages -- Sanskrit…
Machine Translationnamed-entity-recognitionNamed Entity RecognitionNatural Language Understanding+3Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
Large language models are usually fine-tuned to align with human preferences. However, fine-tuning a large language model can be challenging. In this work, we introduce $\textit{weak-to-strong search}$, framing the align…
Instruction FollowingLanguage ModelingLanguage ModellingLarge Language ModelCluster-Aware Neural Collapse Prompt Tuning for Long-Tailed Generalization of Vision-Language Models
Prompt learning has emerged as an efficient alternative to fine-tuning pre-trained vision-language models (VLMs). Despite its promise, current methods still struggle to maintain tail-class discriminability when adapting …
Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guarantees
We consider fine-tuning a pretrained deep neural network on a target task. We study the generalization properties of fine-tuning to understand the problem of overfitting, which has often been observed (e.g., when the tar…
Generalization Boundsimage-classificationImage ClassificationTransfer LearningQuestion: How do Large Language Models perform on the Question Answering tasks? Answer:
Large Language Models (LLMs) have been showing promising results for various NLP-tasks without the explicit need to be trained for these tasks by using few-shot or zero-shot prompting techniques. A common NLP-task is que…
ArticlesInstruction FollowingQuestion Answering