Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models
The wide applicability and adaptability of generative large language models (LLMs) has enabled their rapid adoption. While the pre-trained models can perform many tasks, such models are often fine-tuned to improve their performance on various downstream applications. However, this leads to issues over violation of model licenses, model theft, and copyright infringement. Moreover, recent advances show that generative technology is capable of producing harmful content which exacerbates the problems of accountability within model supply chains. Thus, we need a method to investigate how a model was trained or a piece of text was generated and what their pre-trained base model was. In this paper we take the first step to address this open problem by tracing back the origin of a given fine-tuned LLM to its corresponding pre-trained base model. We consider different knowledge levels and attribution strategies, and find that we can correctly trace back 8 out of the 10 fine tuned models with our best method.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fingerprinting Fine-tuned Language Models in the Wild
There are concerns that the ability of language models (LMs) to generate high quality synthetic text can be misused to launch spam, disinformation, or propaganda. Therefore, the research community is actively working on …
AttributeArtificial Interrogation for Attributing Language Models
This paper presents solutions to the Machine Learning Model Attribution challenge (MLMAC) collectively organized by MITRE, Microsoft, Schmidt-Futures, Robust-Intelligence, Lincoln-Network, and Huggingface community. The …
Machine TranslationMulti Class Text Classificationtext-classificationText Classification+1Improving LLM-based Ontology Matching with fine-tuning on synthetic data
Large Language Models (LLMs) are increasingly being integrated into various components of Ontology Matching pipelines. This paper investigates the capability of LLMs to perform ontology matching directly on ontology modu…
Analyzing how BERT performs entity matching
State-of-the-art Entity Matching (EM) approaches rely on transformer architectures, such as BERT, for generating highly contex-tualized embeddings of terms. The embeddings are then used to predict whether pairs of entity…
Entity ResolutionSemantic SimilaritySemantic Textual SimilarityLeveraging Language Models for Automated Patient Record Linkage
Objective: Healthcare data fragmentation presents a major challenge for linking patient data, necessitating robust record linkage to integrate patient records from diverse sources. This study investigates the feasibility…
BlockingData IntegrationSentence Embeddings