LLASP: Fine-tuning Large Language Models for Answer Set Programming

2024-07-26 · Erica Coppolillo, Francesco Calimeri, Giuseppe Manco, Simona Perri, Francesco Ricca

Recently, Large Language Models (LLMs) have showcased their potential in various natural language processing tasks, including code generation. However, while significant progress has been made in adapting LLMs to generate code for several imperative programming languages and tasks, there remains a notable gap in their application to declarative formalisms, such as Answer Set Programming (ASP). In this paper, we move a step towards exploring the capabilities of LLMs for ASP code generation. First, we perform a systematic evaluation of several state-of-the-art LLMs. Despite their power in terms of number of parameters, training data and computational resources, empirical results demonstrate inadequate performances in generating correct ASP programs. Therefore, we propose LLASP, a fine-tuned lightweight model specifically trained to encode fundamental ASP program patterns. To this aim, we create an ad-hoc dataset covering a wide variety of fundamental problem specifications that can be encoded in ASP. Our experiments demonstrate that the quality of ASP programs generated by LLASP is remarkable. This holds true not only when compared to the non-fine-tuned counterpart but also when compared to the majority of eager LLM candidates, particularly from a semantic perspective. All the code and data used to perform the experiments are publicly available at https://anonymous.4open.science/r/LLASP-D86C/.

📄 PDF Abstract BibTeX arXiv:2407.18723

Code (0)

등록된 구현이 없습니다.

Tasks

Code Generation

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Fine-Tuning Large Language Models for Answering Programming Questions with Code Snippets

2023-06-26 · ICCS: International Conference on Computational Science 2023 6 · Vadim Lomshakov, Sergey Kovalchuk, Maxim Omelchenko, Sergey Nikolenko 외

We study the ability of pretrained large language models (LLM) to answer questions from online question answering fora such as Stack Overflow. We consider question-answer pairs where the main part of the answer consists …

Code Generation Language Modelling Program Synthesis Prompt Engineering+2

Question answering system of bridge design specification based on large language model

2024-08-26 · Leye Zhang, Xiangxiang Tian, Hongjun Zhang

This paper constructs question answering system for bridge design specification based on large language model. Three implementation schemes are tried: full fine-tuning of the Bert pretrained model, parameter-efficient fi…

Language Modeling Language Modelling Large Language Model parameter-efficient fine-tuning+2

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models

2024-02-17 · Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi 외

Despite their success at many natural language processing (NLP) tasks, large language models still struggle to effectively leverage knowledge for knowledge-intensive tasks, manifesting limitations such as generating inco…

Medical Question Answering Question Answering

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models

2023-12-31 · Qianxi Li, Yingyue Cao, Jikun Kang, Tianpei Yang 외

Fine-tuning Large Language Models (LLMs) adapts a trained model to specific downstream tasks, significantly improving task-specific performance. Supervised Fine-Tuning (SFT) is a common approach, where an LLM is trained …

Question Answering

Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning

2024-01-25 · Yanda Chen, Chandan Singh, Xiaodong Liu, Simiao Zuo 외

Large language models (LLMs) often generate convincing, fluent explanations. However, different from humans, they often generate inconsistent explanations on different inputs. For example, an LLM may generate the explana…